REVIEW 4 major objections 6 minor 19 references
Non-Myopic Multi-Objective Bayesian Optimization
T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read This paper introduces the first non-myopic acquisition functions for multi-objective Bayesian optimization, deriving them from a lower bound on the Bellman equation built on hypervolume improvement.
desk verdict First non-myopic MOBO formulation with an HVI reward, but the implemented NMMO-Joint replaces the derived expectation with a mean plug-in, so the headline gains cannot yet be attributed to the non-myopic lower bound. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Hypervolume improvement (HVI), the increase in dominated objective-space volume when a new outcome joins the current Pareto front, is the scalar reward that makes the Bellman lower bound valid. Because HVI telescopes into the total hypervolume gain over a sequence of evaluations, it satisfies the additivity condition that generic vector rewards fail. The acquisition functions combine the myopic EHVI term with an expected batch EHVI term for the lookahead horizon, and the expectation is made computationally tractable by a one-step lookahead GP model that updates variances while keeping the mean function unchanged.
What would settle it
On a small synthetic problem with a known Pareto front, compute the lookahead term of the acquisition functions by exact Monte Carlo sampling of $y$ from the full posterior conditioned on a new observation $(x,y)$, and compare the induced ranking of candidate inputs with the ranking from the paper's variance-only one-step lookahead proxy; a material difference in rankings would show the lower-bound values are not faithful estimates. In addition, for a horizon of three on a small discrete grid, enumerate the exact Bellman policy and check whether the first input selected by NMMO-Joint matches the optimal first action.
Extended reading notes
Core claim
The central claim is that a finite-horizon multi-objective sequential experimental design can be solved approximately by scalarizing the reward as hypervolume improvement and maximizing a lower bound on the Bellman recursion, $\alpha_{\mathrm{Nested}}(x|D_t) = \mathrm{EHVI}(x|D_t) + \max_{X'} \mathbb{E}_y[\mathrm{BEHVI}(X'|D_t \cup (x,y))]$. Because $\mathrm{HVI}_t = \mathrm{HV}(Y_t) - \mathrm{HV}(Y_{t-1})$, the total improvement over a horizon telescopes to $\mathrm{HV}(Y_T) - \mathrm{HV}(Y_0)$, which gives the additivity the recursion needs. The paper derives NMMO-Joint and BINOM from this bound to avoid the nested maximization, reports that these non-myopic acquisitions substantially improve hypervolume over myopic EHVI and JESMO on multiple real-world benchmarks, and shows that instantiating the framework with information-gain scalarizations is possible but that HVI generally performs best.
Load-bearing premise
The acquisition values assume that, after a hypothetical new observation, the surrogate model's predictions have the same means and only narrower variances, and the resulting lower-bound scores remain faithful to the true posterior; if this one-step lookahead proxy is wrong, the performance gains may not transfer.
Editorial extensions
If this is right
- Finite-horizon lookahead becomes feasible for multi-objective Bayesian optimization, and the two scalable variants, Joint and BINOM, outperform myopic EHVI and JESMO on the tested real-world problems.
- Longer horizons do not automatically help: hypervolume tends to decline as the lookahead horizon grows from 2 to 8, because prediction uncertainty compounds, so the authors recommend short horizons.
- Any improvement-based scalarization inherits the additivity argument, so the framework can be re-instantiated with different reward definitions; information-gain variants work but HVI typically wins in the ablation.
- The lower bound contains EHVI as its first term, so optimizing it is at least as tight as the myopic policy in the ideal model setting.
Reading between the lines
- If the variance-only lookahead proxy is unfaithful when the posterior mean shifts after new data, the acquisition scores could be miscalibrated; a direct check would compare them against Monte Carlo sampling of the full posterior update on a small problem.
- The additivity of HVI could transfer to other sequential decision problems, such as multi-objective reinforcement learning, wherever a scalar, Pareto-sensitive reward with additive structure is wanted.
- A stronger stress test is to compare these acquisitions against an exactly solved Bellman policy on a small discrete input grid with known Pareto front; the paper only reports grid-based experiments for the Nested variant.
- The reported gains are measured under short horizons and fixed budgets, so the method's advantage may shrink or reverse if the surrogate model is accurate enough that myopic choices already track the optimal policy.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper addresses finite-horizon sequential experimental design for multi-objective optimization of expensive black-box functions. It proposes to use hypervolume improvement (HVI) as a scalar reward that satisfies the conditions needed for a Bellman-type decomposition, and derives a lower-bound acquisition function by replacing the nested adaptive maximization in the Bellman recursion with a batch expected hypervolume improvement (BEHVI). From this lower bound the authors obtain three acquisition functions: NMMO-Nested (exact lower-bound computation), NMMO-Joint (joint optimization over the first query and the lookahead batch), and BINOM (a fast batch-EHVI variant). Experiments on six real-world benchmarks plus synthetic problems compare these methods with EHVI, JESMO, MESMO, and HVKG.
Significance. If the theoretical derivation and the practical implementation matched, this would be a useful contribution: it provides a principled way to import the lower-bound Bellman machinery into multi-objective BO, and the proposed methods are evaluated on diverse real-world problems with released code and ablation studies. However, the implemented NMMO-Joint replaces the expectation in Eq. 15 by a deterministic one-step lookahead with unchanged posterior mean, which is not the expectation over the posterior predictive distribution. Because the empirical performance gains are attributed to the non-myopic lower-bound formulation, this gap, together with the modified JESMO baseline, prevents the paper from fully supporting its central claim in its current form.
major comments (4)
- [Section 4.2.2, Eq. 15] The 'expectation computation via posterior sampling' substitutes the expectation E_y[BEHVI(X' | D_t ∪ (x,y))] with a one-step lookahead GP model whose posterior mean is unchanged and whose variance is updated by conditioning on the new input. For a GP, this is exactly the fantasy observation y = m(x); it is a single plug-in trajectory, not a sample from the posterior predictive. Since BEHVI is nonlinear in the fantasy values, this plug-in is generally not equal to the expectation, and the acquisition function actually optimized by NMMO-Joint is not the lower bound α_Joint derived in Eq. 15. The manuscript provides no Monte Carlo comparison, no sample sizes, and no error analysis, and Appendix A.8 only discusses tightness of the lower bounds with respect to the optimal policy, not this approximation. This gap is load-bearing because the empirical gains in Section 5.1 are claimed as evidence for the Bellman lower-bound approach.
- [Appendix A.7] The JESMO baseline is modified from its default of 10 Pareto sample points to 4 because the default implementation failed. The paper does not report a sensitivity analysis showing that this modification does not disadvantage JESMO. Since JESMO is one of the two main baselines in Figure 1, the claim 'substantially improve performance over the existing myopic AFs' is not fully supported for the JESMO comparison.
- [Section 5 and Appendix A.5] NMMO-Nested, the only method that directly implements the exact lower bound in Eq. 14, is absent from the main experimental results and performs worse than the approximate variants at H=2. The paper's central theoretical contribution is therefore not the source of the demonstrated practical improvements; the practical results rest on NMMO-Joint and BINOM, which are approximations. The abstract and contribution list should be revised to make this distinction explicit.
- [Section 5.1] The hypervolume results are reported as averaged curves without standard deviations, confidence intervals, or significance tests. Given that all experiments were run for 15 replications, the authors should report the variance or perform a paired test to support the 'substantially improve' claim.
minor comments (6)
- [Section 4.2.1, Lemma 1] Lemma 1 is a telescoping identity (HV(Y_T)-HV(Y_0)) and should be described as such; calling it a lemma and devoting an appendix proof gives the impression of a substantive result when the statement is definitionally true.
- [Section 4.2.2] The heading 'Expectation computation via posterior sampling' is misleading because the proposed one-step lookahead model is not posterior sampling; consider renaming it, e.g., 'Expectation approximation via a one-step lookahead model.'
- [Algorithm 1, line 9] The selection xt = argmax_{x∈X} EHVI(x|Dt) uses a myopic score to select from the batch; if this follows Jiang et al. (2020a), please cite the specific connection in the algorithm description.
- [Appendix Table 2] The meaning of the column 'H' for the baseline methods (EHVI, JESMO, HVKG) is not defined in the caption; clarify that it denotes the lookahead horizon only for the proposed methods.
- [Figure 1 and appendix figures] It is not clear whether the plotted bands are standard errors or confidence intervals; add a caption note and, if the bands are absent, add them.
- [Appendix A.5] The statement that NMMO-Nested 'should theoretically outperform' the other non-myopic baselines is speculative and should be supported or removed, especially since the reported results do not show it.
Circularity Check
No significant circularity: the Bellman lower-bound derivation is self-contained, and the reported gains are not forced by construction.
full rationale
The paper's derivation chain is self-contained. The Bellman recursion (Eq. 4) and the lower-bound connection between nested and batch maximization (Eqs. 8-10) are taken from prior work and adapted to the multi-objective setting with HVI as the scalarizing reward. The HVI additivity lemma (Eq. 12 and Appendix B) is a telescoping identity: HVI_t is defined as HV(Y_t)-HV(Y_{t-1}) and HVI_total as HV(Y_T)-HV(Y_0), so the sum equals the total by definition. This is a tautology, but it is not a circular prediction: it is used only to verify a necessary condition for the Bellman framework, and it does not smuggle in the acquisition results or the benchmark outcomes. The proposed acquisition functions (Eqs. 14-16) are explicit closed-form expressions in terms of EHVI and BEHVI; no parameter is fitted to the target benchmark hypervolumes and then reported as a predicted improvement. The replacement of the expectation in Eq. 15 by a one-step-lookahead GP variance update (Section 4.2.2) is an approximation, and Appendix A.8 concedes that the tightness and fidelity of the lower bounds remain open; this is a faithfulness and correctness concern, not a circularity, because the implemented quantity is not claimed to be identical to the derived expectation by construction. BINOM is explicitly defined as BEHVI (Eq. 16), an existing batch acquisition function, and is presented as an approximate variant by analogy to prior work rather than as a fresh derivation; this is borrowing or renaming, not circular reduction. The empirical improvements over EHVI and JESMO are measured against external benchmarks and are not entailed by the definitions used to derive the acquisition functions. Self-citations appear only for baselines and ablations, and they are not load-bearing for the central derivation. No step in the paper reduces by construction to its own inputs, so the circularity burden is minimal.
Assumptions & free parameters
free parameters (3)
- Lookahead horizon H =
H=4 for BINOM, H=4/8 for NMMO-Joint, H=2 for NMMO-Nested
- Posterior sample count for expectation estimation =
not reported
- JESMO Pareto sample count =
reduced from 10 to 4
assumptions (4)
- standard math Finite-horizon dynamic programming with additive rewards (the Bellman equation) applies to the MOO sequential decision process once HVI is used as the scalar reward.
- domain assumption The K objective functions are realizations of independent GPs with Matérn 5/2 ARD kernels, so their posteriors correctly quantify uncertainty.
- ad hoc to paper The one-step lookahead GP approximation (unchanged mean, updated variance) faithfully approximates the expectation over future observations in Eq 14-15.
- ad hoc to paper The lower bound in Eq 8-10 has a small adaptivity gap, so maximizing it approximates the optimal finite-horizon policy well enough to improve decisions.
Cite this review
Pith. "Pith review of Non-Myopic Multi-Objective Bayesian Optimization." pith.science (2026). https://pith.science/paper/MPXJMOOS
@misc{pith2026241208085,
author = {Pith},
title = {Pith review of: Non-Myopic Multi-Objective Bayesian Optimization},
year = {2026},
howpublished = {\url{https://pith.science/paper/MPXJMOOS}},
note = {Machine review of arXiv:2412.08085}
}
read the original abstract
We consider the problem of finite-horizon sequential experimental design to solve multi-objective optimization (MOO) of expensive black-box objective functions. This problem arises in many real-world applications, including materials design, where we have a small resource budget to make and evaluate candidate materials in the lab. We solve this problem using the framework of Bayesian optimization (BO) and propose the first set of non-myopic methods for MOO problems. Prior work on non-myopic BO for single-objective problems relies on the Bellman optimality principle to handle the lookahead reasoning process. However, this principle does not hold for most MOO problems because the reward function needs to satisfy some conditions: scalar variable, monotonicity, and additivity. We address this challenge by using hypervolume improvement (HVI) as our scalarization approach, which allows us to use a lower-bound on the Bellman equation to approximate the finite-horizon using a batch expected hypervolume improvement (EHVI) acquisition function (AF) for MOO. Our formulation naturally allows us to use other improvement-based scalarizations and compare their efficacy to HVI. We derive three non-myopic AFs for MOBO: 1) the Nested AF, which is based on the exact computation of the lower bound, 2) the Joint AF, which is a lower bound on the nested AF, and 3) the BINOM AF, which is a fast and approximate variant based on batch multi-objective acquisition functions. Our experiments on multiple diverse real-world MO problems demonstrate that our non-myopic AFs substantially improve performance over the existing myopic AFs for MOBO.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Pareto front-diverse batch multi- objective Bayesian optimization
Alaleh Ahmadianshalchi, Syrine Belakaria, and Janardhan Rao Doppa. Pareto front-diverse batch multi- objective Bayesian optimization. Proceedings of the AAAI Conference on Artificial Intelligence, 38: 10784–10794, Mar. 2024a. doi: 10.1609/aaai.v38i10.28951. URL https://ojs.aaai.org/index.php/ AAAI/article/view/28951. Alaleh Ahmadianshalchi, Syrine Belakar...
-
[2]
This method has an additional factor of H due to the joint optimization over the lookahead horizon. NMMO-Nested: O(KN 2H 2M ) Where M is the size of the discretized input space used for the nested optimization. *https://github.com/pytorch/botorch 20 Published in Transactions on Machine Learning Research (04/2025) Table 1: Benchmark details. Problem name d...
-
[5]
doi: 10.1038/s41586-019-1798-7
ISSN 1476-4687. doi: 10.1038/s41586-019-1798-7. URL http: //dx.doi.org/10.1038/s41586-019-1798-7. Eric S Chen, Alaleh Ahmadianshalchi, Sonja S Sparks, Chuchu Chen, Aryan Deshwal, Janardhan R Doppa, and Kaiyan Qiu. Machine learning enabled design and optimization for 3d-printing of high-fidelity presurgical organ models.Advanced Materials Technologies, pp....
-
[6]
13 Published in Transactions on Machine Learning Research (04/2025) Aryan Deshwal, Syrine Belakaria, Ganapati Bhat, Janardhan Rao Doppa, and Partha Pratim Pande. Learning pareto-frontier resource management policies for heterogeneous socs: An information-theoretic approach. In (DAC), 2021a. Aryan Deshwal, Syrine Belakaria, and Janardhan Rao Doppa. Bayesia...
work page 2025
-
[7]
Due to computational cost, we were not able to run experiments of NMMO-Nested with additional horizon values. As explained in section 5, while the NMMO-Nested method should theoretically outperform the other non-myopic baselines, its optimization is very time-consuming and inefficient in practice, leading to slightly subpar results. MESMO. We included MES...
work page 2019
-
[8]
Figure 2: Hypervolume results for NMMO-Joint with lookahead horizonH∈{ 2, 4, 6, 8}. A.3 Alternative scalarization approaches: an ablation study The proof of additivity in Lemma 1 relies on the fact that HVI is an improvement-based scalarization. Therefore, this lemma theoretically holds for any improvement-based scalarization. Notably, information gain (I...
work page 2016
-
[9]
Metal–organic frameworks (mofs).Chemical Society Reviews, 43(16):5415–5418,
14 Published in Transactions on Machine Learning Research (04/2025) Susumu Kitagawa et al. Metal–organic frameworks (mofs).Chemical Society Reviews, 43(16):5415–5418,
work page 2025
-
[12]
Multi-objective optimization of reram crossbars for robust DNN inferencing under stochastic noise
Xiaoxuan Yang, Syrine Belakaria, Biresh Kumar Joardar, Huanrui Yang, Janardhan Rao Doppa, Partha Pra- tim Pande, Krishnendu Chakrabarty, and Hai Helen Li. Multi-objective optimization of reram crossbars for robust DNN inferencing under stochastic noise. InIEEE/ACM International Conference On Computer Aided Design, ICCAD 2021, Munich, Germany, November 1-4...
work page 2021
Show all 19 references
-
[13]
Design of multi-output switched-capacitor voltage regulator via machine learning
Zhiyuan Zhou, Syrine Belakaria, Aryan Deshwal, Wookpyo Hong, Janardhan Rao Doppa, Partha Pratim Pande, and Deukhyoun Heo. Design of multi-output switched-capacitor voltage regulator via machine learning. In 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE...
2020
-
[16]
The ZDT3 problem is a widely used test in multi-objective optimization literature (Konakovic Lukovic et al., 2020). To demonstrate the effectiveness of our non-myopic methods on problems with a larger number of objectives, we evaluated our most computationally efficient method...
2020
-
[19]
as a baseline for comparison in our study. However, we encountered a significant challenge with this approach: the JESMO implementation in BoTorch failed almost immediately when applied to many of our real-world multi-objective problems. This failure was primarily due to the a...
2025
-
[2000]
A.2 Analysis of lookahead horizons We include figures illustrating the performance of our non-myopic methods across different horizon values
16 Published in Transactions on Machine Learning Research (04/2025) A Appendix A.1 Benchmarks details In Table 1, we include a concise overview of all of our MOO benchmarks consisting of the problem names, number of input dimensions (d), number of output dimensions (K), and th...
2025
-
[2006]
Roman Garnett
doi: 10.1109/TEVC.2005.859463. Roman Garnett. Bayesian optimization. Cambridge University Press,
2005
-
[2014]
Hypervolume-based multi-objective reinforcement learning
Kristof Van Moffaert, Madalina M Drugan, and Ann Nowé. Hypervolume-based multi-objective reinforcement learning. InEvolutionary Multi-Criterion Optimization: 7th International Conference, EMO 2013, Sheffield, UK, March 19-22,
2013
-
[2015]
Multi-objective Bayesian optimization using pareto-frontier entropy
15 Published in Transactions on Machine Learning Research (04/2025) Shinya Suzuki, Shion Takeno, Tomoyuki Tamura, Kazuki Shitara, and Masayuki Karasuyama. Multi-objective Bayesian optimization using pareto-frontier entropy. InInternational Conference on Machine Learning, pp. 9...
2025
-
[2019]
Max-value entropy search for multi-objective Bayesian optimization with constraints.arXiv preprint arXiv:2009.01721, 2020a
Syrine Belakaria, Aryan Deshwal, and Janardhan Rao Doppa. Max-value entropy search for multi-objective Bayesian optimization with constraints.arXiv preprint arXiv:2009.01721, 2020a. 12 Published in Transactions on Machine Learning Research (04/2025) Syrine Belakaria, Aryan Des...
2009 arXiv
-
[2020]
Leon Barrett and Srini Narayanan
URL http://arxiv.org/abs/1910.06403. Leon Barrett and Srini Narayanan. Learning all optimal policies with multiple criteria. InProceedings of the 25th international conference on Machine learning, pp. 41–47,
1910 arXiv
-
[2021]
Bayesian optimization over iterative learners with structured responses: A budget-aware planning approach.CoRR, abs/2206.12708,
Syrine Belakaria, Rishit Sheth, Janardhan Rao Doppa, and Nicolò Fusi. Bayesian optimization over iterative learners with structured responses: A budget-aware planning approach.CoRR, abs/2206.12708,
-
[2022]
Bayesian optimization over high-dimensional combinatorial spaces via dictionary-based embed- dings
Aryan Deshwal, Sebastian Ament, Maximilian Balandat, Eytan Bakshy, Janardhan Rao Doppa, and David Eriksson. Bayesian optimization over high-dimensional combinatorial spaces via dictionary-based embed- dings. CoRR, abs/2303.01774,
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.