Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Information-theoretic Bayesian Optimization: Survey and Tutorial

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This survey shows that every information-theoretic acquisition function in Bayesian optimization is the same objective—maximize expected information gain about the optimum—with differences only in what is tracked and how the entropy is…

desk verdict A useful roadmap to information-theoretic BO, but a missing minus sign in the conditional entropy definition breaks the central mutual-information identity every acquisition function relies on. read the letter →

arxiv 2502.06789 v1 pith:5CL3GG6S submitted 2025-01-22 cs.LG cs.AIcs.ITmath.ITstat.ML

Eduardo C. Garrido-Merchán This is my paper · ORCID
classification cs.LGcs.AIcs.ITmath.ITstat.ML
keywords Bayesianoptimizationinformation-theoreticacquisitionfunctionsentropysearchmutualinformationGaussianprocessespredictivemax-valuejoint
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This survey and tutorial argues that the many information-theoretic acquisition functions in Bayesian optimization form a single coherent family rather than a bag of separate tricks: each chooses the next point to evaluate by maximizing the expected information gain about the solution of the problem, either its location, its value, or both. The paper starts from basic information theory and Gaussian process surrogates, then presents the major methods in historical order—IAGO, Entropy Search, PES, MES, FITBO, JES, and their extensions—showing that each one is an approximation of the same mutual-information objective. A sympathetic reader would care because this unified view makes an intimidating literature accessible: once the common objective is seen, the adaptations to constrained, multi-objective, parallel, asynchronous, and multi-fidelity settings become variations on a single recipe. The payoff, if the survey is right, is that a practitioner can understand, compare, and even design information-theoretic acquisition functions from one principle.

What carries the argument

The central object is the mutual-information acquisition function, $\alpha(x) = I(\{x,y\}; S \mid D)$, where $S$ is the random variable describing the solution (optimum location $x^\star$, optimum value $f^\star$, or both). This objective is intractable in closed form; the load-bearing machinery is the set of approximations that make it computable: the expectation propagation algorithm for the Gaussian integrals arising from optimality factors, Monte Carlo sampling of optima or max values, and moment matching for truncated normal conditionals. The paper's explanatory work is to show how every surveyed acquisition function arises from this same objective plus one of these approximation choices.

What would settle it

Take a low-dimensional benchmark (for instance a two-dimensional test function) where the exact expected entropy reduction of every candidate point can be computed by numerical integration on a fine grid. Then compare the ranking of candidates produced by the exact computation with the rankings produced by Entropy Search, PES, MES, and JES using their standard approximations. If the approximate rankings put clearly suboptimal points at the top of the list—reversing the exact ranking on the points the exact method would select—then the central practical claim that these approximations preserve information-gain ranking would be falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that every information-theoretic acquisition function for Bayesian optimization can be written as the expected reduction of entropy (equivalently, a mutual information) between a candidate evaluation and some random variable that encodes the optimum of the black-box function—the minimizer location $x^\star$, the maximum value $f^\star$, or the joint pair $(x^\star, f^\star)$. The tutorial traces this claim through the main methods: IAGO and Entropy Search track the optimum location, predictive entropy search uses the symmetry of mutual information to simplify the approximation, max-value entropy search tracks the scalar optimum value to make the objective univariate, and joint entropy search tracks location and value together for richer information. The paper then argues that the differences among methods are approximation choices for the intractable entropy terms—expectation propagation for ES and PES, Monte Carlo sampling of optima for MES, and moment matching of truncated normals for JES—rather than differences in the underlying objective. Having established this, it extends the recipe to complex scenarios by adding non-Gaussian factors for constraints, Pareto-set membership, or batches, and approximating those factors in the same way.

Load-bearing premise

The survey's practical usefulness rests on the fidelity of the approximate inference procedures used by the surveyed acquisition functions (expectation propagation, Monte Carlo sampling of optima, and moment matching of truncated normals), since the paper provides no experiments or error bounds showing that these approximations preserve the true ranking of candidate points by information gain.

Editorial extensions

If this is right

  • A newcomer can implement a working information-theoretic BO loop by choosing a target random variable (location, value, or joint) and an approximation scheme, instead of learning each method from scratch.
  • The unified objective explains why these acquisition functions are global rather than local: they use the whole posterior predictive distribution, so they avoid the exploitation bias of expected improvement.
  • Adaptations to constraints, multiple objectives, and parallel batches follow a standard recipe—add the relevant indicator factors and approximate them with EP or a Laplace method—making the extended literature much easier to read.
  • Comparisons between methods can be reframed as a trade-off between approximation error and computational cost, clarifying when to prefer PES-style accuracy over MES-style speed.
  • The paper's historical narrative implies that further improvements are most likely to come from better approximations (e.g., power expectation propagation) or generalized entropies rather than from entirely new acquisition heuristics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I would take the taxonomy as a design space: any random variable summarizing the solution could define a new acquisition function, and the paper's framework suggests that the objective family is closed under such choices.
  • The 'typically outperforms' claim in the abstract is an empirical generalization; whether it holds on a given problem likely depends on the quality of the approximation and the evaluation budget, and a controlled study would be needed to separate the objective's merits from the approximation's errors.
  • If a substantially better approximation to the same mutual-information objective were developed, it would improve all methods in this family at once, which suggests the approximations—not the objectives—are the key bottleneck.
  • The survey supports a modular software design where the information objective is separated from the approximation engine, so practitioners could mix and match targets and approximations without reimplementing the whole pipeline.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper is a tutorial-and-survey of information-theoretic acquisition functions for Bayesian optimization. Sections 2 and 3 recap Gaussian-process-based BO and the information-theoretic concepts used later. Section 4 presents, in chronological order, IAGO, Entropy Search, Predictive Entropy Search, Max-value Entropy Search, FITBO, Joint Entropy Search, and Alpha Entropy Search, together with adaptations to constrained, multi-objective, constrained multi-objective, parallel, multi-fidelity, high-dimensional, multi-agent, and multi-task settings. Section 5 lists future research directions. The stated contribution is to make the technical literature on information-theoretic BO accessible and to explain why mutual-information objectives such as Eqs. (13), (16), (19), and (21)-(23) work, and how their intractable terms are approximated.

Significance. A reliable survey and tutorial of this kind would be genuinely useful: the primary literature is technical, recent, and heavily reliant on approximate inference. The paper's strengths are its chronological organization, its separation of the exact information-theoretic objective from the approximation step (expectation propagation, Monte Carlo sampling, and truncated-normal moment matching), and its systematic pointers to the original papers for implementation details. It also covers recent methods such as JES and AES, and it identifies open problems such as power EP and generalized entropies. However, the tutorial value is currently undermined by a load-bearing sign error in the definition of conditional entropy (Section 3), which is the quantity on which every acquisition function in Section 4 is built. The performance claim in the Abstract ('typically outperforms the rest of acquisition functions') is asserted rather than evidenced, but that is acceptable for a survey provided it is attributed to the cited empirical literature. Once the technical errors are fixed, the paper can serve as an entry point to the field.

major comments (4)
  1. [Section 3] The definition of conditional differential entropy is missing the leading minus sign: the text gives H(X|Y) = ∫∫ p(x,y) log p(x|y) dx dy, whereas the correct expression is H(X|Y) = −∫∫ p(x,y) log p(x|y) dx dy. This is not cosmetic, because the paper explicitly builds on the identity I(X;Y) = H(X) − H(X|Y) and on the statement that I(X;Y) = 0 for independent variables. Under the printed definition, independent X and Y would give H(X|Y) = −H(X), so the identity would yield I(X;Y) = 2H(X), contradicting the sentence immediately following Eq. (11). Since Eqs. (13), (16), (19), and (21)-(23) all measure information gain as a reduction of an entropy, the tutorial cannot be followed as written.
  2. [Section 2, Eq. (7)] The expected-improvement formula is malformed: the argument of Φ and ϕ has an unmatched parenthesis, and the second term uses f(x) where the first term uses f(x⋆). As printed, the expression cannot be evaluated. For a tutorial paper this is a serious presentation defect; the equation should read EI(x) = (µ(x) − f(x⋆))Φ((µ(x) − f(x⋆))/σ(x)) + σ(x)ϕ((µ(x) − f(x⋆))/σ(x)) (up to the convention for maximization).
  3. [Section 2] The sentence 'If the assumption about the objective function f(x) belonging to the distribution of functions F is satisfied, f ∈ F, then, using a Gaussian process will guarantee that the Bayesian optimization procedure is successful' is an unconditional guarantee that the manuscript neither proves nor supports. Even with a well-specified GP prior, the success of BO depends on adequate optimization of the acquisition function, on hyperparameters, and on the noise model; the available theoretical results are convergence or regret bounds under specific conditions, not a blanket guarantee. This sentence should be qualified or replaced with a precise citation.
  4. [Abstract and Section 4.2] The Abstract promises coverage of 'non-myopic, multi-fidelity, parallel and asynchronous settings', but Section 4.2 does not discuss non-myopic or asynchronous Bayesian optimization. Freeze-thaw BO concerns partial/interrupted evaluations, multi-fidelity MES concerns variable evaluation cost, and the multi-agent method is a parallel/decentralized variant; none is presented as non-myopic or asynchronous BO. The scope mismatch should be fixed either by adding the missing material or by removing the unsubstantiated terms from the Abstract and the introductory scope statements.
minor comments (5)
  1. [Section 3, Eq. (11)] The mutual information is written as I(X,Y) throughout, whereas the standard notation I(X;Y) is later used implicitly in the identity. Using one consistent notation would avoid confusion.
  2. [Section 4] The text refers to 'Section 4.2.2' for the robotics adaptation of Entropy Search, but the paper has no Section 4.2.2; the material appears in the unnumbered paragraph before Section 4.1. The cross-reference should be corrected.
  3. [Section 4.2] The claim that 'All the approaches seen until now work successfully for less than 8 dimensions' is unsupported and vague: no definition of 'successfully' is given, and no citation accompanies the dimensional threshold. It should either be removed or replaced with a citation to a comparative study.
  4. [Section 4.1.1] In the constrained BO example, Eq. (25) states the problem as max f(x) subject to c_i(x) ≤ 0, but the caption of Figure 6 says the solution is the point that minimizes the objective such that the parabola is positive. The sign conventions should be aligned or explained.
  5. [Throughout] The manuscript contains numerous typographical errors ('iteraiton', 'orequivalently', 'beggining', 'henc,e', 'Entorpy', 'rigurous') and inconsistent punctuation. A careful language edit is needed before publication.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is a survey/tutorial that restates externally sourced methods; no prediction reduces to a fitted input or self-citation chain.

full rationale

This paper is a survey and tutorial, not a derivation of new results. It does not fit parameters to data and then present them as predictions, and it does not construct an acquisition function from the quantity it later claims to predict. The equations in Sections 2-4 are standard results restated with citations to the original literature: the Gaussian process predictive equations, the mutual information identity, and the ES/PES/MES/JES acquisition functions are all attributed to their source papers. No equation in the paper forces its own conclusion by definition. The author's own prior works (e.g., [13], [14], [18], [19], [20]) are cited as sources for specific methods and receive favorable descriptions, but they are not used as the sole justification of a uniqueness theorem, nor are they invoked to forbid alternative approaches. The survey's central expository content is independently sourced from the cited primary literature. One notable defect is in Section 3, where the conditional differential entropy is written as H(X|Y) = ∫∫ p(x,y) log p(x|y) dx dy without the leading minus sign; this is internally inconsistent with the immediately following identity I(X;Y) = H(X) - H(X|Y). However, this is a mathematical sign error and a correctness risk, not a circularity: it does not make any claim equivalent to its own input. No circular step can be exhibited with a specific quote-and-reduction, so the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The survey introduces no new free parameters or entities; it compiles methods from the cited literature. The main imported assumptions are the GP surrogate, Gaussian noise, and the fidelity of approximate inference used by the surveyed acquisition functions. These assumptions are load-bearing for the practical claims but are not verified by the paper.

assumptions (3)
  • domain assumption The objective function is modeled as a sample from a Gaussian process with chosen mean, kernel, and hyperparameters; the paper states this guarantees BO success if f is in the GP class.
    Invoked throughout Section 2 (Eqs. 2-5) and Algorithm 1. The guarantee statement is stronger than what the cited theory supports, so the assumption is load-bearing for the tutorial's framing.
  • domain assumption Observations are noisy with y = f(x) + epsilon, where epsilon is zero-mean Gaussian noise with known variance.
    Used in Eqs. (3)-(5) and Algorithm 1; this is a standard BO assumption inherited from the Gaussian process literature.
  • domain assumption Approximate inference procedures used by the surveyed acquisition functions (expectation propagation, Monte Carlo sampling of optima, moment matching of truncated normals) preserve the ranking of candidate points by true information gain.
    Relied on throughout Section 4 (e.g., Eqs. 15, 20, 23); the survey does not test or bound this approximation error.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Information-theoretic Bayesian Optimization: Survey and Tutorial." pith.science (2026). https://pith.science/paper/5CL3GG6S

@misc{pith2026250206789,
  author       = {Pith},
  title        = {Pith review of: Information-theoretic Bayesian Optimization: Survey and Tutorial},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5CL3GG6S}},
  note         = {Machine review of arXiv:2502.06789}
}
read the original abstract

Several scenarios require the optimization of non-convex black-box functions, that are noisy expensive to evaluate functions with unknown analytical expression, whose gradients are hence not accessible. For example, the hyper-parameter tuning problem of machine learning models. Bayesian optimization is a class of methods with state-of-the-art performance delivering a solution to this problem in real scenarios. It uses an iterative process that employs a probabilistic surrogate model, typically a Gaussian process, of the objective function to be optimized computing a posterior predictive distribution of the black-box function. Based on the information given by this posterior predictive distribution, Bayesian optimization includes the computation of an acquisition function that represents, for every input space point, the utility of evaluating that point in the next iteraiton if the objective of the process is to retrieve a global extremum. This paper is a survey of the information theoretical acquisition functions, whose performance typically outperforms the rest of acquisition functions. The main concepts of the field of information theory are also described in detail to make the reader aware of why information theory acquisition functions deliver great results in Bayesian optimization and how can we approximate them when they are intractable. We also cover how information theory acquisition functions can be adapted to complex optimization scenarios such as the multi-objective, constrained, non-myopic, multi-fidelity, parallel and asynchronous settings and provide further lines of research.

Figures

Figures reproduced from arXiv: 2502.06789 by the authors.

Figure 1
Figure 1. Gaussian process posterior distribution of an objective function and [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Shannon information content (blue) and probabilities (yellow) associ [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Entropy of two discrete random variables. The entropy is the expec [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Empirical distribution of the minimizer p(x ⋆ ) of a black-box function according to the information given by the posterior distribution of the condi￾tioned Gaussian process on previous observations. Each sample consists on a path of the Gaussian process model that is …
Figure 5
Figure 5. Figure 5: Visual example of the distribution of the optimum value of the problem [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]
Figure 6
Figure 6. Figure 6: Constrained Black-box optimization problem. The constraint is drawn [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Visual example of the Pareto frontier, red crosses, of a two black [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: Batch Bayesian optimization of a one-dimensional objective function [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Generative AI for Industrial Contour Detection: A Language-Guided Vision System

    cs.CV 2025-08 reject novelty 4.0 of 10

    A GAN-plus-VLM pipeline improves industrial remnant contour extraction, with GPT-image-1 outperforming Gemini 2.0 Flash on SSIM, LPIPS, and Hausdorff distance.

Reference graph

Works this paper leans on

51 extracted references · 44 canonical work pages · cited by 1 Pith paper

  1. [1]

    Agrell and K

    C. Agrell and K. R. Dahl. Sequential bayesian optimal experimental design for structural reliability analysis.Statistics and Computing, 31(3):27, 2021

    work page 2021

  2. [2]

    J. M. Amigó, S. G. Balogh, and S. Hernández. A brief review of generalized entropies. Entropy, 20(11):813, 2018

    work page 2018

  3. [3]

    S. Bai, J. Wang, F. Chen, and B. Englot. Information-theoretic exploration with bayesian optimization. In2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1816–1822. IEEE, 2016

    work page 2016

  4. [4]

    Belakaria, A

    S. Belakaria, A. Deshwal, and J. R. Doppa. Max-value entropy search for multi-objective bayesian optimization.Advances in neural information processing systems, 32, 2019. 22

    work page 2019

  5. [5]

    Belakaria, A

    S. Belakaria, A. Deshwal, and J. R. Doppa. Max-value entropy search for multi-objective bayesian optimization with constraints.arXiv preprint arXiv:2009.01721, 2020

    arXiv 2009

  6. [6]

    Belakaria, A

    S. Belakaria, A. Deshwal, and J. R. Doppa. Multi-fidelity multi-objective bayesian optimization: An output space entropy search approach. InPro- ceedings of the AAAI Conference on artificial intelligence, volume 34, pages 10035–10043, 2020

    work page 2020

  7. [7]

    Belakaria, A

    S. Belakaria, A. Deshwal, and J. R. Doppa. Output space entropy search framework for multi-objective bayesian optimization.Journal of artificial intelligence research, 72:667–715, 2021

    work page 2021

  8. [8]

    H. Bian, J. Tian, J. Yu, and H. Yu. Bayesian co-evolutionary optimization based entropy search for high-dimensional many-objective optimization. Knowledge-Based Systems, 274:110630, 2023

    work page 2023

Show all 51 references
  1. [9]

    D. M. Blei, A. Kucukelbir, and J. D. McAuliffe. Variational inference: A review for statisticians. Journal of the American statistical Association, 112(518):859–877, 2017

    2017

  2. [10]

    B. Brevini. Black boxes, not green: Mythologizing artificial intelligence and omitting the environment. Big Data & Society, 7(2):2053951720935141, 2020

    2020

  3. [11]

    T. M. Cover.Elements of information theory. John Wiley & Sons, 1999

    1999

  4. [12]

    J. P. Cunningham, P. Hennig, and S. Lacoste-Julien. Gaussian probabilities and expectation propagation.arXiv preprint arXiv:1111.6832, 2011

    2011 arXiv

  5. [13]

    Fernández-Sánchez, E

    D. Fernández-Sánchez, E. C. Garrido-Merchán, and D. Hernández-Lobato. Improved max-value entropy search for multi-objective bayesian optimiza- tion with constraints.Neurocomputing, 546:126290, 2023

    2023

  6. [14]

    Fernández-Sánchez, E

    D. Fernández-Sánchez, E. C. Garrido-Merchán, and D. Hernández-Lobato. Alpha entropy search for new information-based bayesian optimization. arXiv preprint arXiv:2411.16586, 2024

    2024 arXiv

  7. [15]

    Fröhlich, E

    L. Fröhlich, E. Klenske, J. Vinogradska, C. Daniel, and M. Zeilinger. Noisy- input entropy search for efficient robust bayesian optimization. InInterna- tional Conference on Artificial Intelligence and Statistics, pages 2262–2272. PMLR, 2020

    2020

  8. [16]

    Fukuoka, Y

    H. Fukuoka, Y. Tsukada, T. Koyama, M. Shiga, I. Takeuchi, and M. Kara- suyama. Multi-fidelity bayesian optimization with max-value entropy search. In International Conference on Machine Learning, pages 9334–

  9. [17]

    R. Garnett. Bayesian optimization. Cambridge University Press, 2023. 23

    2023

  10. [18]

    E. C. Garrido Merchán. Advanced methods for bayesian optimization in complex scenarios. PhD Thesis, 2021

    2021

  11. [19]

    E. C. Garrido-Merchán, D. Fernández-Sánchez, and D. Hernández-Lobato. Parallel predictive entropy search for multi-objective bayesian optimiza- tion with constraints applied to the tuning of machine learning algorithms. Expert Systems with Applications, 215:119328, 2023

    2023

  12. [20]

    E. C. Garrido-Merchán and D. Hernández-Lobato. Predictive entropy search for multi-objective bayesian optimization with constraints. Neu- rocomputing, 361:50–68, 2019

    2019

  13. [21]

    Geman and B

    D. Geman and B. Jedynak. An active testing model for tracking roads in satellite images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(1):1–14, 1996

    1996

  14. [22]

    Hennig and C

    P. Hennig and C. J. Schuler. Entropy search for information-efficient global optimization. Journal of Machine Learning Research, 13(6), 2012

    2012

  15. [23]

    Hernández-Lobato, J

    D. Hernández-Lobato, J. Hernandez-Lobato, A. Shah, and R. Adams. Pre- dictive entropy search for multi-objective bayesian optimization. InInter- national conference on machine learning, pages 1492–1501. PMLR, 2016

    2016

  16. [24]

    J. M. Hernández-Lobato, M. Gelbart, M. Hoffman, R. Adams, and Z. Ghahramani. Predictive entropy search for bayesian optimization with unknown constraints. In International conference on machine learning, pages 1699–1707. PMLR, 2015

    2015

  17. [25]

    J. M. Hernández-Lobato, M. W. Hoffman, and Z. Ghahramani. Predic- tive entropy search for efficient global optimization of black-box functions. Advances in neural information processing systems, 27, 2014

    2014

  18. [26]

    M. W. Hoffman and Z. Ghahramani. Output-space predictive entropy search for flexible global optimization. In NIPS workshop on Bayesian Optimization, pages 1–5, 2015

    2015

  19. [27]

    Hvarfner, F

    C. Hvarfner, F. Hutter, and L. Nardi. Joint entropy search for maximally- informed bayesian optimization.Advances in Neural Information Process- ing Systems, 35:11494–11506, 2022

    2022

  20. [28]

    James and S

    T. James and S. Thomas. Multi objective regionalized bayesian optimiza- tion via entropy search. InOPT 2024: Optimization for Machine Learning, 2024

    2024

  21. [29]

    D. R. Jones, M. Schonlau, and W. J. Welch. Efficient global optimization of expensive black-box functions.Journal of Global optimization, 13:455–492, 1998

    1998

  22. [30]

    R. Li, Y. Shi, Y. Han, Y. Shao, M. Qi, and B. Li. Active and compact entropy search for high-dimensional bayesian optimization.IEEE Transac- tions on Knowledge and Data Engineering, 35(1):59–70, 2021. 24

    2021

  23. [31]

    H. Ma, T. Zhang, Y. Wu, F. P. Calmon, and N. Li. Gaussian max-value entropy search for multi-agent bayesian optimization. In2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 10028–10035. IEEE, 2023

    2023

  24. [32]

    Information theory, inference and learning algorithms

    D.J.MacKay. Information theory, inference and learning algorithms. Cam- bridge university press, 2003

    2003

  25. [33]

    T. Minka. Power ep. Technical report, Technical report, Microsoft Re- search, Cambridge, 2004

    2004

  26. [34]

    T. P. Minka.A family of algorithms for approximate Bayesian inference. PhD thesis, Massachusetts Institute of Technology, 2001

    2001

  27. [35]

    Onbayesianmethodsforseekingtheextremum

    J.Močkus. Onbayesianmethodsforseekingtheextremum. In Optimization techniques IFIP technical conference: Novosibirsk, July 1–7, 1974, pages 400–404. Springer, 1975

    1974

  28. [36]

    H. B. Moss, D. S. Leslie, and P. Rayson. Mumbo: Multi-task max-value bayesian optimization. InMachine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2020, Ghent, Belgium, September 14–18, 2020, Proceedings, Part III, pages 447–462. Springer, 2021

    2020

  29. [37]

    Neiswanger, L

    W. Neiswanger, L. Yu, S. Zhao, C. Meng, and S. Ermon. Generalizing bayesianoptimizationwithdecision-theoreticentropies. Advances in Neural Information Processing Systems, 35:21016–21029, 2022

    2022

  30. [38]

    Q. P. Nguyen, B. K. H. Low, and P. Jaillet. Rectified max-value entropy search for bayesian optimization.arXiv preprint arXiv:2202.13597, 2022

    2022 arXiv

  31. [39]

    Q. P. Nguyen, Z. Wu, B. K. H. Low, and P. Jaillet. Trusted-maximizers en- tropy search for efficient bayesian optimization. InUncertainty in Artificial Intelligence, pages 1486–1495. PMLR, 2021

    2021

  32. [40]

    Perrone, I

    V. Perrone, I. Shcherbatyi, R. Jenatton, C. Archambeau, and M. Seeger. Constrained bayesian optimization with max-value entropy search.arXiv preprint arXiv:1910.07003, 2019

    1910 arXiv

  33. [41]

    B. Ru, M. A. Osborne, M. McLeod, and D. Granziol. Fast information- theoretic bayesian optimisation. InInternational Conference on Machine Learning, pages 4384–4392. PMLR, 2018

    2018

  34. [42]

    Shah and Z

    A. Shah and Z. Ghahramani. Parallel predictive entropy search for batch global optimization of expensive objective functions. Advances in neural information processing systems, 28, 2015

    2015

  35. [43]

    Shahriari, Z

    B. Shahriari, Z. Wang, M. W. Hoffman, A. Bouchard-Côté, and N. de Fre- itas. An entropy search portfolio for bayesian optimization.arXiv preprint arXiv:1406.4625, 2014. 25

    2014 arXiv

  36. [44]

    C. E. Shannon. A mathematical theory of communication.The Bell system technical journal, 27(3):379–423, 1948

    1948

  37. [45]

    Snoek, H

    J. Snoek, H. Larochelle, and R. P. Adams. Practical bayesian optimization of machine learning algorithms.Advances in neural information processing systems, 25, 2012

    2012

  38. [46]

    Suzuki, S

    S. Suzuki, S. Takeno, T. Tamura, K. Shitara, and M. Karasuyama. Multi- objective bayesian optimization using pareto-frontier entropy. InInterna- tional conference on machine learning, pages 9279–9288. PMLR, 2020

    2020

  39. [47]

    Swersky, J

    K. Swersky, J. Snoek, and R. P. Adams. Freeze-thaw bayesian optimization. arXiv preprint arXiv:1406.3896, 2014

    2014 arXiv

  40. [48]

    B. Tu, A. Gandy, N. Kantas, and B. Shafei. Joint entropy search for multi-objective bayesian optimization. Advances in Neural Information Processing Systems, 35:9922–9938, 2022

    2022

  41. [49]

    Villemonteix, E

    J. Villemonteix, E. Vazquez, and E. Walter. An informational approach to the global optimization of expensive-to-evaluate functions.Journal of Global Optimization, 44:509–534, 2009

    2009

  42. [50]

    Wang and S

    Z. Wang and S. Jegelka. Max-value entropy search for efficient bayesian optimization. In International Conference on Machine Learning, pages 3627–3635. PMLR, 2017

    2017

  43. [51]

    C. K. Williams and C. E. Rasmussen. Gaussian processes for machine learning, volume 2. MIT press Cambridge, MA, 2006. 26

    2006

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.