{"id":"3f746acf-c9dd-4917-a452-b50a125dac41","arxiv_id":"2501.13697","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"SafeOpt-type safe Bayesian optimization is shown to be practically unsafe under common heuristics and unavailable RKHS norm bounds, and the authors propose Lipschitz-only LoSBO as a safer alternative.","lead":"This paper argues that popular safe Bayesian optimization algorithms can fail to be safe in practice because their safety bounds depend on a quantity that engineers usually cannot estimate. It introduces a new variant that needs only a Lipschitz constant and bounded noise, and it reports numerical evidence of safety violations in the existing approach.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"LoSBO's safety rests on a finite global Lipschitz constant, but the paper's motivating control performance functions are typically unbounded near stability boundaries, so such L may not exist or be far too conservative.","rationale":"The paper makes a valuable and largely convincing case that heuristic β constants in SafeOpt invalidate the theoretical safety guarantees, and that the theoretically valid bounds require an RKHS-norm bound that is hard to obtain. The proposal to replace this by a Lipschitz bound is natural, and the authors are transparent that applicability hinges on the assumptions. However, the central positive claim — that LoSBO can be safe in practice in control — requires more than a known constant: it requires that a finite global Lipschitz constant actually exists and is small enough to be usable. In controller tuning, the objective is often an infinite-horizon cost or tracking error, which grows without bound as parameters approach the stability boundary; the gradient also blows up. Therefore, on any domain that reaches the boundary, no finite L exists, and on the safe sublevel set the required L can be enormous near the threshold. The Lipschitz lower bound is only valid along straight-line distances; if the segment between a queried point and a safe seed passes through a steep or unstable region, the bound can be violated and unsafe points can be accepted. The paper's numerical studies are on synthetic smooth functions, where a modest L is plausible; they do not demonstrate that a usable L is available for the motivating control problems. This concern is a sharpened form of the reader's weakest assumption (validity of L and ε): the issue is not only that the practitioner might supply a wrong value, but that no finite value may exist in the target regime. I therefore retain the conditional verdict; the condition should explicitly include a demonstration that a usable, valid Lipschitz constant exists for a realistic control tuning problem.","tokens_in":6082,"tokens_out":11021,"duration_ms":106695,"concrete_test":"On a benchmark LQR or automotive tracking controller, compute the infinite-horizon cost J(k) on a fine grid of controller gains that includes points approaching the stability boundary. Estimate the maximal local slope (or Lipschitz constant) on the admissible sublevel set {J ≤ M} for decreasing thresholds M approaching the boundary. If the required L grows without bound, or exceeds any physically motivated prior estimate by orders of magnitude, then LoSBO's safety certificate cannot be certified with a practical L in the targeted control setting.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The proposed remedy replaces the RKHS-norm bound with a global Lipschitz constant L and a noise bound ε, and claims this suffices for safety. The load-bearing weakness is that in the paper's own motivating application — tuning controller parameters with safety constraints, e.g., infinite-horizon LQR cost (Section 1) — the performance function is typically unbounded at the stability boundary. A function that tends to ±∞ at the boundary is not Lipschitz on any domain containing that boundary; moreover, the Lipschitz constant on the safe sublevel set can be arbitrarily large as the threshold approaches the boundary. LoSBO's safety certificate ℓ(x)=max_i(y_i−ε−L‖x−x_i‖) can then be invalid for a candidate x whose straight-line path to an observed safe point crosses a steep or unstable region, so the algorithm may declare an unsafe parameter safe. The paper acknowledges 'the applicability of the proposed algorithm clearly hinges on these assumptions' (Section 3), but does not address that a finite, usable L may simply not exist for the control applications used to motivate the method. Thus the central claim that LoSBO overcomes practical safety issues is not established for those applications.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that SafeOpt-type safe Bayesian optimization algorithms have two significant practical obstacles: (i) typical implementations replace theoretically justified uncertainty bounds by heuristics that are not guaranteed to be valid, and (ii) the theoretically valid uncertainty bounds require an upper bound on the RKHS norm of the target function, which the authors claim cannot be reliably obtained from standard engineering prior knowledge. The authors propose Lipschitz-only Safe Bayesian Optimization (LoSBO), which uses only a known Lipschitz bound and bounded noise for safety, and a grid-free variant LoS-GP-UCB for higher-dimensional problems. The manuscript is explicitly an extended abstract that disseminates results from the authors' journal paper [7], and it states that all plots and results are taken verbatim from [7].","tokens_in":6304,"tokens_out":5351,"duration_ms":46240,"significance":"If the central claims are correct, the paper identifies a real and important gap in the practical applicability of SafeOpt-type algorithms and offers a conceptually simple alternative with more interpretable assumptions. The critique of heuristic uncertainty bounds and the RKHS-norm requirement is timely and could influence practice in learning-based control. However, the manuscript's contribution is largely a summary of prior work, and the practical value of LoSBO depends crucially on whether the required Lipschitz and noise bounds can be supplied in the very applications used to motivate the method.","major_comments":[{"comment":"The safety certificate of LoSBO, ℓ(x) = max_i(y_i − ε − L‖x − x_i‖), requires the target function to be L-Lipschitz continuous on the entire search space D. The paper motivates the need for safe BO with control applications such as LQR tuning, where the performance measure is typically unbounded near the stability boundary. A function that is unbounded on D is not Lipschitz continuous on D, and no finite L exists. The paper acknowledges that 'the applicability of the proposed algorithm clearly hinges on these assumptions,' but it does not address the mismatch between the motivating applications and the existence of a finite, usable global L. This is load-bearing because the central claim is that LoSBO overcomes the practical safety issues of SafeOpt-type algorithms; for the examples in Section 1, a finite L may not exist and LoSBO is not applicable as stated. The authors should clarify the domain over which L is assumed, discuss whether the safe sublevel set can be chosen to avoid non-Lipschitz regions, and provide guidance on how a practitioner can verify L.","section":"Section 3 (LoSBO)"},{"comment":"The manuscript explicitly states that it is an extended abstract that disseminates results from the journal paper [7] and that all plots and results are taken verbatim from [7]. As a submission to a serious journal, the paper does not contain the derivations, pseudocode, or experimental protocols required to independently verify the central claims. In particular, the claim in Section 2 that 'as long as the safety of the algorithm relies on a quantitative upper bound on the RKHS norm of the target function, it is at present not possible to ensure that SafeOpt-type algorithms are safe in practice' is not substantiated in this manuscript; the supporting numerical experiments are only referenced, not presented. This lack of self-containedness prevents a full assessment of correctness and of the relationship between the stated Lipschitz assumptions and the algorithm's behavior. The authors should include the necessary technical content or restructure the submission to make clear that it is a research summary rather than a full journal paper.","section":"Title page / preface"}],"minor_comments":[{"comment":"The sentence 'they have to be judge on a case-by-case base by practitioners' contains a typo; it should read 'they have to be judged on a case-by-case basis by practitioners.'","section":"Section 3"},{"comment":"The caption uses 'LosBO' while the text uses 'LoSBO'; the spelling should be made consistent.","section":"Figure 2 caption"},{"comment":"The text refers to 'the constant E' in the caption of Figure 2, but E is never defined in the body. Please define the notation or remove it.","section":"Section 3 / Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is essentially a summary of the authors' journal paper [7] and contains no new technical content beyond what is published there. If the journal does not normally accept extended abstracts, the fit is questionable. The review is based on the manuscript as submitted, and the major comments above reflect concerns that the authors should address regardless of the manuscript's format."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear —,\n\nQuick take: this is an honest, useful extended abstract that makes a real point about SafeOpt-type algorithms, but as a standalone document it is a summary of the authors' journal paper, not a verifiable paper in itself.\n\nWhat's actually new: the paper identifies two concrete reasons existing safe BO methods may not be safe in practice — implementations replace the theoretically motivated beta_t with heuristics, and the rigorous bounds hinge on an RKHS norm that engineers generally cannot bound. That second point is a legitimate and important criticism of the SafeOpt literature. The proposed fix, LoSBO, replaces the RKHS-norm assumption with a Lipschitz constant and bounded noise, and the LoS-GP-UCB variant removes gridding. The safety mechanism is standard set-membership reasoning (Milanese-Novara), and the authors are upfront that everything numeric comes from [7]. Good.\n\nSoft spots. First, this version contains none of the derivations or experimental protocols. The reader has to trust [7]. That is fine for a workshop poster, but it limits what a referee can evaluate here. Second, the central negative claim — that no reliable RKHS-norm bound can be obtained from engineering knowledge — is stated strongly and supported mainly by a discussion in [7]; it would be stronger with at least one concrete control example where a valid bound is provably unavailable or impractical. Third, the stress-test concern about Lipschitz constants is real. In the paper's motivating control setting (e.g., LQR tuning), the performance function is often unbounded near the stability boundary, so a finite global Lipschitz constant may not exist, and on the safe sublevel set it can be arbitrarily large. The authors acknowledge that applicability hinges on the assumptions, but they do not address whether their motivating applications actually satisfy them. This doesn't sink the method — it means the method's scope is narrower than the framing suggests.\n\nOverall: the critique is credible, the replacement is sensible under its assumptions, and the paper is honest about its own limitations. I'd recommend sending this to review only if the full journal version is the object of review; for this extended abstract, the right move is to read [7] and judge the full evidence. If it's submitted as a standalone paper, I'd insist on including at least the LoSBO safety proof and one application where the Lipschitz bound can be certified.","headline":"A credible critique of SafeOpt-style safety and a sensible Lipschitz-based alternative, but this extended abstract is a summary of the journal paper and the Lipschitz assumption is shaky for the motivating control problems.","tokens_in":6860,"tokens_out":3114,"would_cite":false,"duration_ms":30196,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that SafeOpt-type safe Bayesian optimization algorithms cannot currently be guaranteed safe in practice, and proposes LoSBO, whose safety rests on a known Lipschitz bound and bounded noise.","keywords":["Bayesian optimization","safe Bayesian optimization","SafeOpt","Lipschitz-only Safe Bayesian Optimization (LoSBO)","kernel methods","learning-based control","reproducing kernel Hilbert space norm","controller parameter tuning"],"falsifier":"Run Real-$\\beta$-SafeOpt with a correctly evaluated rigorous bound on a benchmark where the target function's true RKHS norm is computable and the only prior knowledge is of the kind an engineer would normally have; a reliable bracketing of the true norm from that prior knowledge would undercut the paper's claim that such bounds are presently unobtainable. Separately, for LoSBO, take any $L$-Lipschitz function with noise bounded by $\\varepsilon$ and check every queried point: with valid constants no $f_*(x_t) < h$ should occur, so one such violation would refute the algorithm's safety guarantee.","tokens_in":5868,"feed_emoji":"🛡️","tokens_out":10812,"duration_ms":93458,"temperature":0.7,"pith_summary":"Safe Bayesian optimization algorithms of the SafeOpt family are widely used to tune controller parameters online, where evaluating an unsafe setting can destabilize the plant. This paper argues that their safety guarantees do not survive practical implementation, because the uncertainty bounds the theory requires are either replaced by heuristics or depend on an upper bound on the reproducing kernel Hilbert space norm of the target function, which engineering prior knowledge cannot currently provide. The paper proposes LoSBO, a safe Bayesian optimization algorithm whose safety mechanism uses only a known Lipschitz bound and bounded noise, and LoS-GP-UCB, a variant that avoids gridding the search space. If the argument is right, safety-critical tuning can be made practically safe under assumptions a control engineer can actually verify, instead of under a kernel norm that cannot be bounded.","feed_headline":"SafeOpt-type algorithms can't be guaranteed safe in practice","feed_subtitle":"A known Lipschitz bound and noise bound replace the hard-to-get kernel norm bound, restoring practical safety for control tuning.","key_machinery":"The central mechanism is the Lipschitz cone safety certificate. From a safe observation $(x_i, y_i)$ with noise bounded by $\\varepsilon$ and a target function $f_*$ known to be $L$-Lipschitz, every point $x$ satisfying $y_i - \\varepsilon - L\\|x - x_i\\| \\ge h$ is guaranteed safe, because $f_*(x) \\ge f_*(x_i) - L\\|x - x_i\\| \\ge y_i - \\varepsilon - L\\|x - x_i\\|$. LoSBO uses the union of such certified sets as its safety mechanism, leaving the Gaussian process mean and variance to guide exploration, so the scaling factors $\\beta_t$ are ordinary hyperparameters rather than safety-critical quantities. LoS-GP-UCB keeps the same certificate but optimizes the acquisition function with local search and random restarts, removing the grid that limits SafeOpt-type algorithms to low dimensions.","core_discovery":"The paper's central claim is that SafeOpt-type safe Bayesian optimization cannot currently be made safe in practice. The first obstacle is that the high-probability confidence bounds on which SafeOpt rests are hard to evaluate, and all implementations known to the authors replace them with heuristics such as $\\beta_t = 2$; numerical experiments show that these heuristic bounds are violated even in relatively benign settings. The second obstacle is that the theoretically valid bounds require a known upper bound on the reproducing kernel Hilbert space norm of the target function, a function-space smoothness measure that the paper argues cannot be reliably bounded from established engineering prior knowledge; an invalid bound produces safety violations. The proposed remedy, LoSBO, decouples exploration from safety: a Gaussian process model drives optimization, while safety is enforced by a deterministic lower bound built from a known Lipschitz constant and noise bound. The paper concludes that as long as safety relies on a quantitative RKHS norm bound, SafeOpt-type safety cannot be ensured in practice, whereas Lipschitz-only safety rests on assumptions with clear interpretations in control applications.","pith_inferences":["The paper leaves implicit that the same Lipschitz-cone certificate could replace the confidence-bound safety mechanism in other safe-exploration schemes, such as stagewise or global-optimization variants, potentially giving them practical safety without any RKHS norm bound.","An obvious next question, not addressed here, is how to maintain a valid Lipschitz bound and noise bound over time; any scheme that adapts $L$ or $\\varepsilon$ from data would have to keep the bounds conservative enough to preserve the guarantee.","The critique suggests a concrete audit for earlier safe-BO applications: if the implementation used a constant $\\beta_t$ instead of a rigorously evaluated bound, reported safe behavior should not be credited to the algorithm's theoretical safety guarantee.","In high-dimensional control problems a single global Lipschitz constant may be conservative, so the practical reach of LoSBO is likely to depend on local or dimension-aware Lipschitz bounds; that is a testable extension, not a result of the paper."],"forward_implications":["SafeOpt-type algorithms should not be treated as safe for online controller tuning unless a valid upper bound on the RKHS norm of the target function is available; heuristic choices like $\\beta_t = 2$ forfeit the theoretical guarantee.","Safety and exploration can be decoupled: the Gaussian process model can drive optimization while a Lipschitz-based lower bound enforces safety, turning the GP scaling factors into tunable hyperparameters.","For continuous parameter spaces in moderate dimensions, LoS-GP-UCB offers a gridding-free route to safe optimization, which matters for controller tuning in practice.","If the critique is correct, published results reporting safe behavior of SafeOpt-type algorithms with heuristic bounds should be re-examined, since their safety was not guaranteed by the algorithm's theory.","A practical precondition for safe online tuning is that a valid Lipschitz constant and noise bound can be identified from physical knowledge or high-fidelity simulation."],"supporting_citations":[{"why":"Introduces SafeOpt and its GP-based high-probability safety framework, the object of the paper's critique.","marker":"[21]"},{"why":"Supplies the GP confidence bound that SafeOpt-type algorithms rely on and that implementations replace with heuristics.","marker":"[19]"},{"why":"Gives an improved confidence bound with smaller constants whose practical evaluation is difficult.","marker":"[6]"},{"why":"Provides a practical rigorous uncertainty bound whose validity requires a known upper bound on the RKHS norm.","marker":"[8]"},{"why":"Supplies improvements used to make the rigorous bound evaluable in Real-$\\beta$-SafeOpt.","marker":"[1]"},{"why":"Shows how Lipschitz continuity plus bounded noise yields deterministic set-membership uncertainty bounds, the basis of LoSBO safety.","marker":"[14]"},{"why":"The underlying journal paper whose numerical results and plots this extended abstract adapts and reports.","marker":"[7]"},{"why":"Provides the local-search acquisition optimization used by the gridding-free LoS-GP-UCB variant.","marker":"[2]"}],"fun_headline_variants":["Safe Bayesian optimization isn't safe—here's a fix","Lipschitz bounds rescue safe Bayesian optimization","Why SafeOpt fails in practice and how to fix it","Ditching kernel norm bound restores safety in BO","Practical safety for control tuning via Lipschitz-only BO"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"LoSBO's safety guarantee collapses if the practitioner does not actually have a valid Lipschitz constant and noise bound for the real plant; the paper itself acknowledges that applicability hinges on these assumptions.","fun_headline_variants_meta":{"raw":{"variants":["Safe Bayesian optimization isn't safe—here's a fix","Lipschitz bounds rescue safe Bayesian optimization","Why SafeOpt fails in practice and how to fix it","Ditching kernel norm bound restores safety in BO","Practical safety for control tuning via Lipschitz-only BO"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000767,"raw_usage":{"total_tokens":3432,"prompt_tokens":1010,"completion_tokens":2422,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":626,"completion_tokens_details":{"reasoning_tokens":2344}},"tokens_in":626,"tokens_out":2422,"duration_ms":14208,"temperature":1.0,"reasoning_tokens":2344,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:40:48.157191+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Real-$\\beta$-SafeOpt with a correctly evaluated rigorous bound on a benchmark where the target function's true RKHS norm is computable and the only prior knowledge is of the kind an engineer would normally have; a reliable bracketing of the true norm from that prior knowledge would undercut the paper's claim that such bounds are presently unobtainable. Separately, for LoSBO, take any $L$-Lipschitz function with noise bounded by $\\varepsilon$ and check every queried point: with valid constants no $f_*(x_t) < h$ should occur, so one such violation would refute the algorithm's safety guarantee.","supporting_citations":[{"cited_title":"In: International conference on machine learning, pp","cited_arxiv_id":null,"evidence_quote":"Introduces SafeOpt and its GP-based high-probability safety framework, the object of the paper's critique."},{"cited_title":"27th International Confer- ence on Machine Learning pp","cited_arxiv_id":null,"evidence_quote":"Supplies the GP confidence bound that SafeOpt-type algorithms rely on and that implementations replace with heuristics."},{"cited_title":"34th Interna- tional Conference on Machine Learning (ICML) 2, 1397–1422 (2017)","cited_arxiv_id":null,"evidence_quote":"Gives an improved confidence bound with smaller constants whose practical evaluation is difficult."},{"cited_title":"Proceedings of the AAAI conference on artificial intelligence 35 (2021)","cited_arxiv_id":null,"evidence_quote":"Provides a practical rigorous uncertainty bound whose validity requires a known upper bound on the RKHS norm."},{"cited_title":"Doctoral thesis, University of Alberta (2013)","cited_arxiv_id":null,"evidence_quote":"Supplies improvements used to make the rigorous bound evaluable in Real-$\\beta$-SafeOpt."},{"cited_title":"Au- tomatica 40(6), 957–975 (2004)","cited_arxiv_id":null,"evidence_quote":"Shows how Lipschitz continuity plus bounded noise yields deterministic set-membership uncertainty bounds, the basis of LoSBO safety."},{"cited_title":"Advances in neural information processing systems 33, 21,524–21,538 (2020)","cited_arxiv_id":null,"evidence_quote":"Provides the local-search acquisition optimization used by the gridding-free LoS-GP-UCB variant."}],"review_version":1}