REVIEW 3 major objections 3 minor 1 cited by
On Some Tunable Multi-fidelity Bayesian Optimization Frameworks
T0 review · 3 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A proximity-based acquisition function is claimed to control high-fidelity usage in multi-fidelity Bayesian optimization with one tunable rule.
desk verdict A plausible tunable multi-fidelity BO paper whose headline claim rests on an undefined proximity criterion; send to review but ask for definitions. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the proximity-based multi-fidelity acquisition function: a single scalar rule that scores a candidate point by the proximity of the current Gaussian process posterior to a reference value, and uses that score both to select the next evaluation and to choose the fidelity level. Its role in the argument is to replace the usual stack of acquisition functions, one per fidelity. The second piece of machinery is the pairing of multi-fidelity UCB acquisition with multi-fidelity Gaussian processes, which lets exploration-exploitation trade-offs be expressed at every fidelity rather than only at the highest. Together the two pieces are what make high-fidelity usage a tunable, continuously adjustable quantity.
What would settle it
On the paper's own benchmarks, vary the proximity threshold from very tight to very loose and record both the number of high-fidelity evaluations and the final objective error. If the high-fidelity count does not respond monotonically to the knob, or if moderate settings produce worse optima than low-fidelity-only optimization, the claimed consistent control over high-fidelity usage fails.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that fidelity selection can be driven by how close the current Gaussian process posterior stands to a reference value rather than by a portfolio of fidelity-specific acquisition functions. The proximity signal is used both to propose the next query and to decide when a low-fidelity evaluation suffices, so that high-fidelity calls are spent only when the surrogate is far from the reference. Implemented with multi-fidelity GPs, the same proximity rule extends to Upper Confidence Bound acquisition, yielding a tunable strategy whose high-fidelity usage remains controllable across tasks. The illustrative applications to homogeneous and heterogeneous chemical kinetics, including a dynamic catalysis model for ammonia production, support the claim that this control survives in practically relevant simulators.
Load-bearing premise
The method assumes that the proximity between the surrogate's prediction and a reference value reliably indicates when low-fidelity evaluations are sufficient, so that skipping high-fidelity calls at close points does not silently degrade the optimum.
Editorial extensions
If this is right
- One knob replaces per-fidelity acquisition design: a practitioner tunes proximity sensitivity instead of writing separate acquisition functions for cheap and expensive simulators.
- Multi-fidelity UCB becomes usable with multi-fidelity GPs, so exploration-exploitation balance can be set once and applied across fidelities.
- Expensive simulation campaigns, such as chemical kinetic models for ammonia catalysis, can budget high-fidelity evaluations explicitly while preserving convergence.
- The same acquisition rule should extend to other multi-fidelity GP formulations, since the proximity signal is defined on the posterior rather than on a specific covariance structure.
Reading between the lines
- A natural extension would be to replace the fixed reference value with an adaptive target, such as the current best posterior mean, which could make the proximity rule self-tuning rather than dependent on a user-supplied reference.
- If the proximity criterion is a reliable need-for-information signal, it could be combined with information-theoretic acquisition functions to yield a multi-fidelity entropy search with the same one-knob property.
- A direct test outside the paper's examples: on a fixed benchmark, sweep the proximity threshold and record high-fidelity count versus final error; the claim predicts a monotone trade-off curve that remains usable across settings.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes two tunable multi-fidelity Bayesian optimization frameworks: a proximity-based acquisition function intended to replace the need for separate acquisition functions at each fidelity level, and an extension of multi-fidelity Upper Confidence Bound (UCB) strategies to multi-fidelity Gaussian processes. The authors report benchmarking these approaches against other multi-fidelity acquisition strategies, with particular emphasis on 'consistent control over high-fidelity usage while maintaining convergence efficiency.' The examples include multi-fidelity chemical kinetic models, both homogeneous and heterogeneous.
Significance. If the claims hold, the proximity-based acquisition function would be a practical simplification of multi-fidelity Bayesian optimization, removing the need to design separate acquisition functions per fidelity level. The extension of UCB to multi-fidelity GPs also broadens the toolkit for such problems. However, the abstract alone provides no quantitative results, derivations, or error bars, so the empirical significance cannot be assessed from the submitted text. The framing of the problem and the proposed unification are conceptually interesting, but the evidence needed to support the central claim is not visible in the abstract.
major comments (3)
- [Abstract] The abstract states that the proximity-based acquisition function 'delivers consistent control over high-fidelity usage while maintaining convergence efficiency,' but it does not define the proximity metric. Without specifying whether proximity is measured in input space, function-value space, or via posterior uncertainty, and without stating how the proximity threshold is normalized across problems with different objective scales, the central claim is not testable from the provided text.
- [Abstract] The abstract reports benchmark outcomes without presenting any quantitative results, such as regret curves, high-fidelity budgets, or error bars, and without a comparison to a trivial baseline such as always querying the low-fidelity model. The claim of 'maintaining convergence efficiency' is only meaningful relative to such a baseline, and its absence leaves the central empirical claim unsupported.
- [Abstract] The proximity-based fidelity selection rests on the assumption that the nearness of the current posterior to a reference value (for example, a previous high-fidelity observation) indicates when additional high-fidelity information is unnecessary. The abstract provides no derivation or justification for this proxy, which is load-bearing because an unreliable proxy could either waste high-fidelity queries or cause premature convergence.
minor comments (3)
- [Title] The title 'On Some Tunable Multi-fidelity Bayesian Optimization Frameworks' is vague; a more specific title mentioning the proximity-based acquisition function and the multi-fidelity UCB extension would better convey the contributions.
- [Abstract] The phrase 'rather than the standard GPs typically used' would benefit from clarification: it is unclear which particular GP model is considered standard and how the multi-fidelity GP differs from it in this context.
- [Abstract] The abstract mentions 'hyperparameter tunability' as a benchmark dimension, but it does not indicate how tunability is measured or compared across methods; a brief clarification would help readers interpret the claimed advantage.
Circularity Check
No circularity found in abstract-only evidence.
full rationale
The available text is the abstract only; no equations, derivations, or parameter-fitting descriptions are present. The proximity-based acquisition function is presented as a new algorithmic construction, not as a derived prediction. There is no quoted step in which a fitted input is renamed as a prediction, no self-citation invoked as load-bearing evidence, and no definitional equivalence between inputs and outputs. A possible concern that the proximity threshold is tunable to benchmark tasks would be a question of external validity or overfitting, not circularity, and cannot be assessed without the full text. Under the hard rules requiring quotation of the specific reduction, no circular step can be identified. Hence score 0.
Assumptions & free parameters
free parameters (2)
- Proximity threshold or schedule
- UCB exploration coefficient
assumptions (2)
- domain assumption Low-fidelity models are substantially cheaper than high-fidelity models and are positively correlated with them.
- domain assumption Gaussian process surrogates provide reliable uncertainty estimates that drive the acquisition functions.
Cite this review
Pith. "Pith review of On Some Tunable Multi-fidelity Bayesian Optimization Frameworks." pith.science (2026). https://pith.science/paper/CPUTB4BD
@misc{pith2026250801013,
author = {Pith},
title = {Pith review of: On Some Tunable Multi-fidelity Bayesian Optimization Frameworks},
year = {2026},
howpublished = {\url{https://pith.science/paper/CPUTB4BD}},
note = {Machine review of arXiv:2508.01013}
}
read the original abstract
Multi-fidelity optimization employs surrogate models that integrate information from varying levels of fidelity to guide efficient exploration of complex design spaces while minimizing the reliance on (expensive) high-fidelity objective function evaluations. To advance Gaussian Process (GP)-based multi-fidelity optimization, we implement a proximity-based acquisition strategy that simplifies fidelity selection by eliminating the need for separate acquisition functions at each fidelity level. We also enable multi-fidelity Upper Confidence Bound (UCB) strategies by combining them with multi-fidelity GPs rather than the standard GPs typically used. We benchmark these approaches alongside other multi-fidelity acquisition strategies (including fidelity-weighted approaches) comparing their performance, reliance on high-fidelity evaluations, and hyperparameter tunability in representative optimization tasks. The results highlight the capability of the proximity-based multi-fidelity acquisition function to deliver consistent control over high-fidelity usage while maintaining convergence efficiency. Our illustrative examples include multi-fidelity chemical kinetic models, both homogeneous and heterogeneous (dynamic catalysis for ammonia production).
Forward citations
Cited by 1 Pith paper
-
Frugal, Flexible, Faithful: Causal Data Simulation via Frengression
Frengression is a deep generative realization of the frugal parameterization that models the joint distribution of covariates, treatments, and outcomes and allows direct sampling from user-specified interventional dis...
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.