Pith. sign in

REVIEW 4 major objections 6 minor 19 references

Non-Myopic Multi-Objective Bayesian Optimization

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read This paper introduces the first non-myopic acquisition functions for multi-objective Bayesian optimization, deriving them from a lower bound on the Bellman equation built on hypervolume improvement.

desk verdict First non-myopic MOBO formulation with an HVI reward, but the implemented NMMO-Joint replaces the derived expectation with a mean plug-in, so the headline gains cannot yet be attributed to the non-myopic lower bound. read the letter →

arxiv 2412.08085 v2 pith:MPXJMOOS submitted 2024-12-11 cs.LG cs.AI

classification cs.LGcs.AI
keywords multi-objectiveBayesianoptimizationnon-myopicacquisitionfunctionhypervolumeimprovementBellmanequationlowerboundfinite-horizonsequentialexperimentaldesignexpectedParetofrontapproximationlookaheadpolicy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multi-objective optimization of expensive black-box functions, such as selecting candidate materials for lab measurement, is normally driven by myopic acquisition functions that value only the next experiment. This paper argues that the Bellman optimality principle behind single-objective lookahead planning carries over to multi-objective problems when the reward is defined as hypervolume improvement (HVI), the volume of objective space newly dominated by adding a point to the Pareto front. HVI is scalar, monotone with respect to Pareto dominance, and additive over a sequence of evaluations, so a lower bound on the Bellman equation can be turned into three acquisition functions: Nested, Joint, and BINOM. If this construction is right, finite-horizon planning for multi-objective experiments is practical, and the paper reports that the proposed acquisitions outperform myopic EHVI and JESMO across real-world benchmarks under small budgets.

What carries the argument

Hypervolume improvement (HVI), the increase in dominated objective-space volume when a new outcome joins the current Pareto front, is the scalar reward that makes the Bellman lower bound valid. Because HVI telescopes into the total hypervolume gain over a sequence of evaluations, it satisfies the additivity condition that generic vector rewards fail. The acquisition functions combine the myopic EHVI term with an expected batch EHVI term for the lookahead horizon, and the expectation is made computationally tractable by a one-step lookahead GP model that updates variances while keeping the mean function unchanged.

What would settle it

On a small synthetic problem with a known Pareto front, compute the lookahead term of the acquisition functions by exact Monte Carlo sampling of $y$ from the full posterior conditioned on a new observation $(x,y)$, and compare the induced ranking of candidate inputs with the ranking from the paper's variance-only one-step lookahead proxy; a material difference in rankings would show the lower-bound values are not faithful estimates. In addition, for a horizon of three on a small discrete grid, enumerate the exact Bellman policy and check whether the first input selected by NMMO-Joint matches the optimal first action.

Watch

Extended reading notes

Core claim

The central claim is that a finite-horizon multi-objective sequential experimental design can be solved approximately by scalarizing the reward as hypervolume improvement and maximizing a lower bound on the Bellman recursion, $\alpha_{\mathrm{Nested}}(x|D_t) = \mathrm{EHVI}(x|D_t) + \max_{X'} \mathbb{E}_y[\mathrm{BEHVI}(X'|D_t \cup (x,y))]$. Because $\mathrm{HVI}_t = \mathrm{HV}(Y_t) - \mathrm{HV}(Y_{t-1})$, the total improvement over a horizon telescopes to $\mathrm{HV}(Y_T) - \mathrm{HV}(Y_0)$, which gives the additivity the recursion needs. The paper derives NMMO-Joint and BINOM from this bound to avoid the nested maximization, reports that these non-myopic acquisitions substantially improve hypervolume over myopic EHVI and JESMO on multiple real-world benchmarks, and shows that instantiating the framework with information-gain scalarizations is possible but that HVI generally performs best.

Load-bearing premise

The acquisition values assume that, after a hypothetical new observation, the surrogate model's predictions have the same means and only narrower variances, and the resulting lower-bound scores remain faithful to the true posterior; if this one-step lookahead proxy is wrong, the performance gains may not transfer.

Editorial extensions

If this is right

  • Finite-horizon lookahead becomes feasible for multi-objective Bayesian optimization, and the two scalable variants, Joint and BINOM, outperform myopic EHVI and JESMO on the tested real-world problems.
  • Longer horizons do not automatically help: hypervolume tends to decline as the lookahead horizon grows from 2 to 8, because prediction uncertainty compounds, so the authors recommend short horizons.
  • Any improvement-based scalarization inherits the additivity argument, so the framework can be re-instantiated with different reward definitions; information-gain variants work but HVI typically wins in the ablation.
  • The lower bound contains EHVI as its first term, so optimizing it is at least as tight as the myopic policy in the ideal model setting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the variance-only lookahead proxy is unfaithful when the posterior mean shifts after new data, the acquisition scores could be miscalibrated; a direct check would compare them against Monte Carlo sampling of the full posterior update on a small problem.
  • The additivity of HVI could transfer to other sequential decision problems, such as multi-objective reinforcement learning, wherever a scalar, Pareto-sensitive reward with additive structure is wanted.
  • A stronger stress test is to compare these acquisitions against an exactly solved Bellman policy on a small discrete input grid with known Pareto front; the paper only reports grid-based experiments for the Nested variant.
  • The reported gains are measured under short horizons and fixed budgets, so the method's advantage may shrink or reverse if the surrogate model is accurate enough that myopic choices already track the optimal policy.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper addresses finite-horizon sequential experimental design for multi-objective optimization of expensive black-box functions. It proposes to use hypervolume improvement (HVI) as a scalar reward that satisfies the conditions needed for a Bellman-type decomposition, and derives a lower-bound acquisition function by replacing the nested adaptive maximization in the Bellman recursion with a batch expected hypervolume improvement (BEHVI). From this lower bound the authors obtain three acquisition functions: NMMO-Nested (exact lower-bound computation), NMMO-Joint (joint optimization over the first query and the lookahead batch), and BINOM (a fast batch-EHVI variant). Experiments on six real-world benchmarks plus synthetic problems compare these methods with EHVI, JESMO, MESMO, and HVKG.

Significance. If the theoretical derivation and the practical implementation matched, this would be a useful contribution: it provides a principled way to import the lower-bound Bellman machinery into multi-objective BO, and the proposed methods are evaluated on diverse real-world problems with released code and ablation studies. However, the implemented NMMO-Joint replaces the expectation in Eq. 15 by a deterministic one-step lookahead with unchanged posterior mean, which is not the expectation over the posterior predictive distribution. Because the empirical performance gains are attributed to the non-myopic lower-bound formulation, this gap, together with the modified JESMO baseline, prevents the paper from fully supporting its central claim in its current form.

major comments (4)
  1. [Section 4.2.2, Eq. 15] The 'expectation computation via posterior sampling' substitutes the expectation E_y[BEHVI(X' | D_t ∪ (x,y))] with a one-step lookahead GP model whose posterior mean is unchanged and whose variance is updated by conditioning on the new input. For a GP, this is exactly the fantasy observation y = m(x); it is a single plug-in trajectory, not a sample from the posterior predictive. Since BEHVI is nonlinear in the fantasy values, this plug-in is generally not equal to the expectation, and the acquisition function actually optimized by NMMO-Joint is not the lower bound α_Joint derived in Eq. 15. The manuscript provides no Monte Carlo comparison, no sample sizes, and no error analysis, and Appendix A.8 only discusses tightness of the lower bounds with respect to the optimal policy, not this approximation. This gap is load-bearing because the empirical gains in Section 5.1 are claimed as evidence for the Bellman lower-bound approach.
  2. [Appendix A.7] The JESMO baseline is modified from its default of 10 Pareto sample points to 4 because the default implementation failed. The paper does not report a sensitivity analysis showing that this modification does not disadvantage JESMO. Since JESMO is one of the two main baselines in Figure 1, the claim 'substantially improve performance over the existing myopic AFs' is not fully supported for the JESMO comparison.
  3. [Section 5 and Appendix A.5] NMMO-Nested, the only method that directly implements the exact lower bound in Eq. 14, is absent from the main experimental results and performs worse than the approximate variants at H=2. The paper's central theoretical contribution is therefore not the source of the demonstrated practical improvements; the practical results rest on NMMO-Joint and BINOM, which are approximations. The abstract and contribution list should be revised to make this distinction explicit.
  4. [Section 5.1] The hypervolume results are reported as averaged curves without standard deviations, confidence intervals, or significance tests. Given that all experiments were run for 15 replications, the authors should report the variance or perform a paired test to support the 'substantially improve' claim.
minor comments (6)
  1. [Section 4.2.1, Lemma 1] Lemma 1 is a telescoping identity (HV(Y_T)-HV(Y_0)) and should be described as such; calling it a lemma and devoting an appendix proof gives the impression of a substantive result when the statement is definitionally true.
  2. [Section 4.2.2] The heading 'Expectation computation via posterior sampling' is misleading because the proposed one-step lookahead model is not posterior sampling; consider renaming it, e.g., 'Expectation approximation via a one-step lookahead model.'
  3. [Algorithm 1, line 9] The selection xt = argmax_{x∈X} EHVI(x|Dt) uses a myopic score to select from the batch; if this follows Jiang et al. (2020a), please cite the specific connection in the algorithm description.
  4. [Appendix Table 2] The meaning of the column 'H' for the baseline methods (EHVI, JESMO, HVKG) is not defined in the caption; clarify that it denotes the lookahead horizon only for the proposed methods.
  5. [Figure 1 and appendix figures] It is not clear whether the plotted bands are standard errors or confidence intervals; add a caption note and, if the bands are absent, add them.
  6. [Appendix A.5] The statement that NMMO-Nested 'should theoretically outperform' the other non-myopic baselines is speculative and should be supported or removed, especially since the reported results do not show it.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the Bellman lower-bound derivation is self-contained, and the reported gains are not forced by construction.

full rationale

The paper's derivation chain is self-contained. The Bellman recursion (Eq. 4) and the lower-bound connection between nested and batch maximization (Eqs. 8-10) are taken from prior work and adapted to the multi-objective setting with HVI as the scalarizing reward. The HVI additivity lemma (Eq. 12 and Appendix B) is a telescoping identity: HVI_t is defined as HV(Y_t)-HV(Y_{t-1}) and HVI_total as HV(Y_T)-HV(Y_0), so the sum equals the total by definition. This is a tautology, but it is not a circular prediction: it is used only to verify a necessary condition for the Bellman framework, and it does not smuggle in the acquisition results or the benchmark outcomes. The proposed acquisition functions (Eqs. 14-16) are explicit closed-form expressions in terms of EHVI and BEHVI; no parameter is fitted to the target benchmark hypervolumes and then reported as a predicted improvement. The replacement of the expectation in Eq. 15 by a one-step-lookahead GP variance update (Section 4.2.2) is an approximation, and Appendix A.8 concedes that the tightness and fidelity of the lower bounds remain open; this is a faithfulness and correctness concern, not a circularity, because the implemented quantity is not claimed to be identical to the derived expectation by construction. BINOM is explicitly defined as BEHVI (Eq. 16), an existing batch acquisition function, and is presented as an approximate variant by analogy to prior work rather than as a fresh derivation; this is borrowing or renaming, not circular reduction. The empirical improvements over EHVI and JESMO are measured against external benchmarks and are not entailed by the definitions used to derive the acquisition functions. Self-citations appear only for baselines and ablations, and they are not load-bearing for the central derivation. No step in the paper reduces by construction to its own inputs, so the circularity burden is minimal.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method introduces no fitted constants in the acquisition formulas, but it depends on a user-chosen horizon H and on two unvalidated approximations: the one-step lookahead GP proxy and the tightness of the lower bound. The empirical evaluation also depends on a modified JESMO baseline.

free parameters (3)
  • Lookahead horizon H = H=4 for BINOM, H=4/8 for NMMO-Joint, H=2 for NMMO-Nested
    User-set horizon controls the depth of lookahead; performance degrades for larger H, so reported gains depend on choosing a favorable H.
  • Posterior sample count for expectation estimation = not reported
    The expectation E_y[BEHVI] in Eq 14-15 is estimated by sampling from the GP posterior; the number of samples is not stated and affects the accuracy and stability of the AF.
  • JESMO Pareto sample count = reduced from 10 to 4
    Baseline hyperparameter modified empirically because the standard BoTorch implementation failed on real-world problems; this tuning affects the baseline comparison.
assumptions (4)
  • standard math Finite-horizon dynamic programming with additive rewards (the Bellman equation) applies to the MOO sequential decision process once HVI is used as the scalar reward.
    Invoked in Eq 4 and Section 4.1; a standard theorem, but the state variable and sufficient statistics are not fully formalized.
  • domain assumption The K objective functions are realizations of independent GPs with Matérn 5/2 ARD kernels, so their posteriors correctly quantify uncertainty.
    Section 2 and Section 5 state independent GP models; all acquisition computations rely on these posteriors.
  • ad hoc to paper The one-step lookahead GP approximation (unchanged mean, updated variance) faithfully approximates the expectation over future observations in Eq 14-15.
    Section 4.2.2 substitutes the expectation with this proxy; no empirical validation of the approximation is given.
  • ad hoc to paper The lower bound in Eq 8-10 has a small adaptivity gap, so maximizing it approximates the optimal finite-horizon policy well enough to improve decisions.
    Appendix A.8 acknowledges that the tightness of the lower bound and its fidelity to the optimal policy remain open questions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Non-Myopic Multi-Objective Bayesian Optimization." pith.science (2026). https://pith.science/paper/MPXJMOOS

@misc{pith2026241208085,
  author       = {Pith},
  title        = {Pith review of: Non-Myopic Multi-Objective Bayesian Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MPXJMOOS}},
  note         = {Machine review of arXiv:2412.08085}
}
read the original abstract

We consider the problem of finite-horizon sequential experimental design to solve multi-objective optimization (MOO) of expensive black-box objective functions. This problem arises in many real-world applications, including materials design, where we have a small resource budget to make and evaluate candidate materials in the lab. We solve this problem using the framework of Bayesian optimization (BO) and propose the first set of non-myopic methods for MOO problems. Prior work on non-myopic BO for single-objective problems relies on the Bellman optimality principle to handle the lookahead reasoning process. However, this principle does not hold for most MOO problems because the reward function needs to satisfy some conditions: scalar variable, monotonicity, and additivity. We address this challenge by using hypervolume improvement (HVI) as our scalarization approach, which allows us to use a lower-bound on the Bellman equation to approximate the finite-horizon using a batch expected hypervolume improvement (EHVI) acquisition function (AF) for MOO. Our formulation naturally allows us to use other improvement-based scalarizations and compare their efficacy to HVI. We derive three non-myopic AFs for MOBO: 1) the Nested AF, which is based on the exact computation of the lower bound, 2) the Joint AF, which is a lower bound on the nested AF, and 3) the BINOM AF, which is a fast and approximate variant based on batch multi-objective acquisition functions. Our experiments on multiple diverse real-world MO problems demonstrate that our non-myopic AFs substantially improve performance over the existing myopic AFs for MOBO.

Figures

Figures reproduced from arXiv: 2412.08085 by the authors.

Figure 1
Figure 1. Hypervolume results on real-world problems: Non-myopic methods vs. EHVI and JESMO. [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Hypervolume results for NMMO-Joint with lookahead horizon [PITH_FULL_IMAGE:figures/full_fig_p017_2.png] view at source ↗
Figure 3
Figure 3. Hypervolume results for BINOM with lookahead horizon [PITH_FULL_IMAGE:figures/full_fig_p018_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Hypervolume results for BINOM with MESMO (BINOM-MES) , JESMO (BINOM-JES), and [PITH_FULL_IMAGE:figures/full_fig_p019_4.png]
Figure 5
Figure 5. Figure 5: With respect to the ZDT3 problem, Figure 5a shows the comparison of all baselines, Figure 5b [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: Hypervolume results for variants of the DTLZ problem. [PITH_FULL_IMAGE:figures/full_fig_p020_6.png]
Figure 7
Figure 7. Figure 7: Hypervolume results on real-world benchmarks including additional baselines [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 15 canonical work pages

  1. [1]

    Pareto front-diverse batch multi- objective Bayesian optimization

    Alaleh Ahmadianshalchi, Syrine Belakaria, and Janardhan Rao Doppa. Pareto front-diverse batch multi- objective Bayesian optimization. Proceedings of the AAAI Conference on Artificial Intelligence, 38: 10784–10794, Mar. 2024a. doi: 10.1609/aaai.v38i10.28951. URL https://ojs.aaai.org/index.php/ AAAI/article/view/28951. Alaleh Ahmadianshalchi, Syrine Belakar...

  2. [2]

    NMMO-Nested: O(KN 2H 2M ) Where M is the size of the discretized input space used for the nested optimization

    This method has an additional factor of H due to the joint optimization over the lookahead horizon. NMMO-Nested: O(KN 2H 2M ) Where M is the size of the discretized input space used for the nested optimization. *https://github.com/pytorch/botorch 20 Published in Transactions on Machine Learning Research (04/2025) Table 1: Benchmark details. Problem name d...

  3. [5]

    doi: 10.1038/s41586-019-1798-7

    ISSN 1476-4687. doi: 10.1038/s41586-019-1798-7. URL http: //dx.doi.org/10.1038/s41586-019-1798-7. Eric S Chen, Alaleh Ahmadianshalchi, Sonja S Sparks, Chuchu Chen, Aryan Deshwal, Janardhan R Doppa, and Kaiyan Qiu. Machine learning enabled design and optimization for 3d-printing of high-fidelity presurgical organ models.Advanced Materials Technologies, pp....

  4. [6]

    Learning pareto-frontier resource management policies for heterogeneous socs: An information-theoretic approach

    13 Published in Transactions on Machine Learning Research (04/2025) Aryan Deshwal, Syrine Belakaria, Ganapati Bhat, Janardhan Rao Doppa, and Partha Pratim Pande. Learning pareto-frontier resource management policies for heterogeneous socs: An information-theoretic approach. In (DAC), 2021a. Aryan Deshwal, Syrine Belakaria, and Janardhan Rao Doppa. Bayesia...

  5. [7]

    Due to computational cost, we were not able to run experiments of NMMO-Nested with additional horizon values. As explained in section 5, while the NMMO-Nested method should theoretically outperform the other non-myopic baselines, its optimization is very time-consuming and inefficient in practice, leading to slightly subpar results. MESMO. We included MES...

  6. [8]

    A.3 Alternative scalarization approaches: an ablation study The proof of additivity in Lemma 1 relies on the fact that HVI is an improvement-based scalarization

    Figure 2: Hypervolume results for NMMO-Joint with lookahead horizonH∈{ 2, 4, 6, 8}. A.3 Alternative scalarization approaches: an ablation study The proof of additivity in Lemma 1 relies on the fact that HVI is an improvement-based scalarization. Therefore, this lemma theoretically holds for any improvement-based scalarization. Notably, information gain (I...

  7. [9]

    Metal–organic frameworks (mofs).Chemical Society Reviews, 43(16):5415–5418,

    14 Published in Transactions on Machine Learning Research (04/2025) Susumu Kitagawa et al. Metal–organic frameworks (mofs).Chemical Society Reviews, 43(16):5415–5418,

  8. [12]

    Multi-objective optimization of reram crossbars for robust DNN inferencing under stochastic noise

    Xiaoxuan Yang, Syrine Belakaria, Biresh Kumar Joardar, Huanrui Yang, Janardhan Rao Doppa, Partha Pra- tim Pande, Krishnendu Chakrabarty, and Hai Helen Li. Multi-objective optimization of reram crossbars for robust DNN inferencing under stochastic noise. InIEEE/ACM International Conference On Computer Aided Design, ICCAD 2021, Munich, Germany, November 1-4...

Show all 19 references
  1. [13]

    Design of multi-output switched-capacitor voltage regulator via machine learning

    Zhiyuan Zhou, Syrine Belakaria, Aryan Deshwal, Wookpyo Hong, Janardhan Rao Doppa, Partha Pratim Pande, and Deukhyoun Heo. Design of multi-output switched-capacitor voltage regulator via machine learning. In 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE...

  2. [16]

    The ZDT3 problem is a widely used test in multi-objective optimization literature (Konakovic Lukovic et al., 2020). To demonstrate the effectiveness of our non-myopic methods on problems with a larger number of objectives, we evaluated our most computationally efficient method...

  3. [19]

    as a baseline for comparison in our study. However, we encountered a significant challenge with this approach: the JESMO implementation in BoTorch failed almost immediately when applied to many of our real-world multi-objective problems. This failure was primarily due to the a...

  4. [2000]

    A.2 Analysis of lookahead horizons We include figures illustrating the performance of our non-myopic methods across different horizon values

    16 Published in Transactions on Machine Learning Research (04/2025) A Appendix A.1 Benchmarks details In Table 1, we include a concise overview of all of our MOO benchmarks consisting of the problem names, number of input dimensions (d), number of output dimensions (K), and th...

  5. [2006]

    Roman Garnett

    doi: 10.1109/TEVC.2005.859463. Roman Garnett. Bayesian optimization. Cambridge University Press,

  6. [2014]

    Hypervolume-based multi-objective reinforcement learning

    Kristof Van Moffaert, Madalina M Drugan, and Ann Nowé. Hypervolume-based multi-objective reinforcement learning. InEvolutionary Multi-Criterion Optimization: 7th International Conference, EMO 2013, Sheffield, UK, March 19-22,

  7. [2015]

    Multi-objective Bayesian optimization using pareto-frontier entropy

    15 Published in Transactions on Machine Learning Research (04/2025) Shinya Suzuki, Shion Takeno, Tomoyuki Tamura, Kazuki Shitara, and Masayuki Karasuyama. Multi-objective Bayesian optimization using pareto-frontier entropy. InInternational Conference on Machine Learning, pp. 9...

  8. [2019]

    Max-value entropy search for multi-objective Bayesian optimization with constraints.arXiv preprint arXiv:2009.01721, 2020a

    Syrine Belakaria, Aryan Deshwal, and Janardhan Rao Doppa. Max-value entropy search for multi-objective Bayesian optimization with constraints.arXiv preprint arXiv:2009.01721, 2020a. 12 Published in Transactions on Machine Learning Research (04/2025) Syrine Belakaria, Aryan Des...

  9. [2020]

    Leon Barrett and Srini Narayanan

    URL http://arxiv.org/abs/1910.06403. Leon Barrett and Srini Narayanan. Learning all optimal policies with multiple criteria. InProceedings of the 25th international conference on Machine learning, pp. 41–47,

  10. [2021]

    Bayesian optimization over iterative learners with structured responses: A budget-aware planning approach.CoRR, abs/2206.12708,

    Syrine Belakaria, Rishit Sheth, Janardhan Rao Doppa, and Nicolò Fusi. Bayesian optimization over iterative learners with structured responses: A budget-aware planning approach.CoRR, abs/2206.12708,

  11. [2022]

    Bayesian optimization over high-dimensional combinatorial spaces via dictionary-based embed- dings

    Aryan Deshwal, Sebastian Ament, Maximilian Balandat, Eytan Bakshy, Janardhan Rao Doppa, and David Eriksson. Bayesian optimization over high-dimensional combinatorial spaces via dictionary-based embed- dings. CoRR, abs/2303.01774,

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.