{"id":"9466ce8e-61ea-46d5-8892-a51d838cecb0","arxiv_id":"2412.08085","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"The paper proposes the first multi-step lookahead acquisition functions for multi-objective Bayesian optimization, using hypervolume improvement as the reward scalarization, and reports improved Pareto-front hypervolume on real-world benchmarks.","lead":"This paper builds Bayesian optimization methods that plan several experiments ahead instead of choosing each experiment greedily, for problems with multiple competing objectives. The authors show their lookahead methods find better trade-off frontiers within small experimental budgets on real-world design problems.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Implemented NMMO-Joint replaces the expectation in Eq. 15 with a single GP fantasy at the predictive mean; this breaks the derivation of the lower bound and is not validated.","rationale":"The weakest point is not the HVI additivity claim (Appendix B telescopes correctly when Y_t is the nondominated front) nor the lower-bound algebra (Eq. 8-10 is a standard batch ≤ adaptive inequality). The vulnerable step is the implementation of the lookahead term in Eq. 14-15. The paper's Section 4.2.2 describes substituting the expectation with a one-step-lookahead GP whose mean is unchanged; for a GP, the only observation that leaves the mean unchanged is y=m(x), so the substitution is a single deterministic fantasy rather than an expectation over posterior samples. Because BEHVI is nonlinear, this substitution can be biased in either direction. The appendix's own limitation statement (A.8) acknowledges that lower-bound tightness and fidelity are open questions. The proposed check, Monte Carlo expectation versus the proxy, would settle whether the implemented acquisition function is actually optimizing the derived lower bound. If it fails, the experimental superiority claim in Figure 1 cannot be attributed to non-myopic reasoning; if it passes, the paper's conditional accept is justified. BINOM is also essentially BEHVI, so part of its comparison to EHVI may measure batch size rather than lookahead, but the expectation-proxy issue is the more load-bearing concern. I would keep the reader's CONDITIONAL verdict rather than move it, pending this check.","tokens_in":18367,"tokens_out":10661,"duration_ms":117607,"concrete_test":"Reproduce the second term of Eq. 15 exactly for a fixed dataset and candidate x: draw N=256 samples y_j from the posterior predictive p(f(x)|Dt), update the GP with each y_j (e.g., via BoTorch fantasy models), compute BEHVI(X'|Dt∪(x,y_j)), and average. Compare this Monte Carlo acquisition surface to the one-step-lookahead proxy used in the paper on ZDT3 and one real benchmark (MOF) for H=2 and H=3. If the rank correlation of the two acquisition surfaces is below ~0.9 or the argmax inputs disagree in a substantial fraction of trials, the proxy is not faithful and the empirical claims in Figure 1 are not tied to the derived lower bound.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.2.2 (Expectation computation via posterior sampling) states that the expectation E_y[BEHVI(X'|Dt∪(x,y))] in Eq. 14-15 is 'substituted' by a one-step lookahead GP model in which the mean function remains unchanged and only the variance is updated. For a Gaussian process, conditioning on a fantasy observation y = m(x) is exactly what leaves the posterior mean unchanged; conditioning on any other sampled y shifts the mean. The implemented term is therefore not the expectation over the posterior predictive distribution but a single plug-in trajectory at the posterior mean. Since BEHVI is nonlinear in y, the plug-in value need not approximate the expectation, and the acquisition function being optimized is not the lower bound derived in Eq. 10/15. The paper reports no validation of this substitution, no Monte Carlo comparison, no sample sizes, and no error analysis, and Appendix A.8 concedes that the tightness and fidelity of these lower bounds remain open. Because this approximation is used in the reported NMMO-Joint experiments, the observed gains cannot yet be attributed to the claimed non-myopic Bellman lower bound.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses finite-horizon sequential experimental design for multi-objective optimization of expensive black-box functions. It proposes to use hypervolume improvement (HVI) as a scalar reward that satisfies the conditions needed for a Bellman-type decomposition, and derives a lower-bound acquisition function by replacing the nested adaptive maximization in the Bellman recursion with a batch expected hypervolume improvement (BEHVI). From this lower bound the authors obtain three acquisition functions: NMMO-Nested (exact lower-bound computation), NMMO-Joint (joint optimization over the first query and the lookahead batch), and BINOM (a fast batch-EHVI variant). Experiments on six real-world benchmarks plus synthetic problems compare these methods with EHVI, JESMO, MESMO, and HVKG.","tokens_in":18674,"tokens_out":8914,"duration_ms":87730,"significance":"If the theoretical derivation and the practical implementation matched, this would be a useful contribution: it provides a principled way to import the lower-bound Bellman machinery into multi-objective BO, and the proposed methods are evaluated on diverse real-world problems with released code and ablation studies. However, the implemented NMMO-Joint replaces the expectation in Eq. 15 by a deterministic one-step lookahead with unchanged posterior mean, which is not the expectation over the posterior predictive distribution. Because the empirical performance gains are attributed to the non-myopic lower-bound formulation, this gap, together with the modified JESMO baseline, prevents the paper from fully supporting its central claim in its current form.","major_comments":[{"comment":"The 'expectation computation via posterior sampling' substitutes the expectation E_y[BEHVI(X' | D_t ∪ (x,y))] with a one-step lookahead GP model whose posterior mean is unchanged and whose variance is updated by conditioning on the new input. For a GP, this is exactly the fantasy observation y = m(x); it is a single plug-in trajectory, not a sample from the posterior predictive. Since BEHVI is nonlinear in the fantasy values, this plug-in is generally not equal to the expectation, and the acquisition function actually optimized by NMMO-Joint is not the lower bound α_Joint derived in Eq. 15. The manuscript provides no Monte Carlo comparison, no sample sizes, and no error analysis, and Appendix A.8 only discusses tightness of the lower bounds with respect to the optimal policy, not this approximation. This gap is load-bearing because the empirical gains in Section 5.1 are claimed as evidence for the Bellman lower-bound approach.","section":"Section 4.2.2, Eq. 15"},{"comment":"The JESMO baseline is modified from its default of 10 Pareto sample points to 4 because the default implementation failed. The paper does not report a sensitivity analysis showing that this modification does not disadvantage JESMO. Since JESMO is one of the two main baselines in Figure 1, the claim 'substantially improve performance over the existing myopic AFs' is not fully supported for the JESMO comparison.","section":"Appendix A.7"},{"comment":"NMMO-Nested, the only method that directly implements the exact lower bound in Eq. 14, is absent from the main experimental results and performs worse than the approximate variants at H=2. The paper's central theoretical contribution is therefore not the source of the demonstrated practical improvements; the practical results rest on NMMO-Joint and BINOM, which are approximations. The abstract and contribution list should be revised to make this distinction explicit.","section":"Section 5 and Appendix A.5"},{"comment":"The hypervolume results are reported as averaged curves without standard deviations, confidence intervals, or significance tests. Given that all experiments were run for 15 replications, the authors should report the variance or perform a paired test to support the 'substantially improve' claim.","section":"Section 5.1"}],"minor_comments":[{"comment":"Lemma 1 is a telescoping identity (HV(Y_T)-HV(Y_0)) and should be described as such; calling it a lemma and devoting an appendix proof gives the impression of a substantive result when the statement is definitionally true.","section":"Section 4.2.1, Lemma 1"},{"comment":"The heading 'Expectation computation via posterior sampling' is misleading because the proposed one-step lookahead model is not posterior sampling; consider renaming it, e.g., 'Expectation approximation via a one-step lookahead model.'","section":"Section 4.2.2"},{"comment":"The selection xt = argmax_{x∈X} EHVI(x|Dt) uses a myopic score to select from the batch; if this follows Jiang et al. (2020a), please cite the specific connection in the algorithm description.","section":"Algorithm 1, line 9"},{"comment":"The meaning of the column 'H' for the baseline methods (EHVI, JESMO, HVKG) is not defined in the caption; clarify that it denotes the lookahead horizon only for the proposed methods.","section":"Appendix Table 2"},{"comment":"It is not clear whether the plotted bands are standard errors or confidence intervals; add a caption note and, if the bands are absent, add them.","section":"Figure 1 and appendix figures"},{"comment":"The statement that NMMO-Nested 'should theoretically outperform' the other non-myopic baselines is speculative and should be supported or removed, especially since the reported results do not show it.","section":"Appendix A.5"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for TMLR, and the code release and ablation studies are commendable. The main concern is the mismatch between the derived acquisition function and the implemented one; this needs to be resolved in revision, either by validating the one-step lookahead approximation or by reframing the contribution as a heuristic. The JESMO baseline modification also needs careful handling. I would not recommend rejection because the core lower-bound idea is sound and the experiments, if properly validated, could support a useful contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is the first to frame finite-horizon multi-objective BO as a Bellman recursion with hypervolume improvement as an additive scalar reward. That framing is a legitimate step, and the three acquisition functions (Nested, Joint, BINOM) are new combinations of known pieces: EHVI, BEHVI, and the Jiang et al. lower bound. The additivity lemma is a tautology, but it does point at the right property. The experiments cover several real-world benchmarks, include ablations with different scalarizations, and the code is available.\n\nThe soft spots are real and one is load-bearing. In Section 4.2.2 the authors say they 'substitute' the expectation E_y[BEHVI] with a one-step lookahead GP where the mean function is unchanged and only the variance is updated. Conditioning on a fantasy observation at the posterior mean is exactly what leaves the mean unchanged. So the implemented NMMO-Joint optimizes a single plug-in trajectory, not the expectation over the posterior predictive. BEHVI is nonlinear in y, so the plug-in is not the expectation and the acquisition being optimized is not the lower bound derived in Eq. 10/15. The paper does not validate this substitution with a Monte Carlo comparison, and Appendix A.8 concedes the fidelity of these lower bounds remains open. That means the reported gains over myopic baselines cannot be confidently attributed to the non-myopic Bellman lower bound. The JESMO baseline was also modified from 10 to 4 Pareto samples to make it run, and the NMMO-Nested results only appear in the appendix, so the main figure is weaker than the full story.\n\nThis is a useful first step for the MOBO community, but it is not a definitive demonstration. The formulation is worth thinking about; the implementation gap needs to be closed or explicitly characterized. If this came in fresh, I would send it to review because the question is timely and the authors engage with the prior work honestly. But I would expect major revision before the empirical claims can be taken at face value.","headline":"First non-myopic MOBO formulation with an HVI reward, but the implemented NMMO-Joint replaces the derived expectation with a mean plug-in, so the headline gains cannot yet be attributed to the non-myopic lower bound.","tokens_in":19179,"tokens_out":3290,"would_cite":false,"duration_ms":28151,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces the first non-myopic acquisition functions for multi-objective Bayesian optimization, deriving them from a lower bound on the Bellman equation built on hypervolume improvement.","keywords":["multi-objective Bayesian optimization","non-myopic acquisition function","hypervolume improvement","Bellman equation lower bound","finite-horizon sequential experimental design","expected hypervolume improvement","Pareto front approximation","lookahead policy"],"falsifier":"On a small synthetic problem with a known Pareto front, compute the lookahead term of the acquisition functions by exact Monte Carlo sampling of $y$ from the full posterior conditioned on a new observation $(x,y)$, and compare the induced ranking of candidate inputs with the ranking from the paper's variance-only one-step lookahead proxy; a material difference in rankings would show the lower-bound values are not faithful estimates. In addition, for a horizon of three on a small discrete grid, enumerate the exact Bellman policy and check whether the first input selected by NMMO-Joint matches the optimal first action.","tokens_in":18177,"feed_emoji":"📈","tokens_out":9440,"duration_ms":83890,"temperature":0.7,"pith_summary":"Multi-objective optimization of expensive black-box functions, such as selecting candidate materials for lab measurement, is normally driven by myopic acquisition functions that value only the next experiment. This paper argues that the Bellman optimality principle behind single-objective lookahead planning carries over to multi-objective problems when the reward is defined as hypervolume improvement (HVI), the volume of objective space newly dominated by adding a point to the Pareto front. HVI is scalar, monotone with respect to Pareto dominance, and additive over a sequence of evaluations, so a lower bound on the Bellman equation can be turned into three acquisition functions: Nested, Joint, and BINOM. If this construction is right, finite-horizon planning for multi-objective experiments is practical, and the paper reports that the proposed acquisitions outperform myopic EHVI and JESMO across real-world benchmarks under small budgets.","feed_headline":"Lookahead planning beats myopic acquisition for multi-objective BO","feed_subtitle":"A scalar hypervolume reward makes multi-step planning practical and lifts Pareto-front quality on small budgets.","key_machinery":"Hypervolume improvement (HVI), the increase in dominated objective-space volume when a new outcome joins the current Pareto front, is the scalar reward that makes the Bellman lower bound valid. Because HVI telescopes into the total hypervolume gain over a sequence of evaluations, it satisfies the additivity condition that generic vector rewards fail. The acquisition functions combine the myopic EHVI term with an expected batch EHVI term for the lookahead horizon, and the expectation is made computationally tractable by a one-step lookahead GP model that updates variances while keeping the mean function unchanged.","core_discovery":"The central claim is that a finite-horizon multi-objective sequential experimental design can be solved approximately by scalarizing the reward as hypervolume improvement and maximizing a lower bound on the Bellman recursion, $\\alpha_{\\mathrm{Nested}}(x|D_t) = \\mathrm{EHVI}(x|D_t) + \\max_{X'} \\mathbb{E}_y[\\mathrm{BEHVI}(X'|D_t \\cup (x,y))]$. Because $\\mathrm{HVI}_t = \\mathrm{HV}(Y_t) - \\mathrm{HV}(Y_{t-1})$, the total improvement over a horizon telescopes to $\\mathrm{HV}(Y_T) - \\mathrm{HV}(Y_0)$, which gives the additivity the recursion needs. The paper derives NMMO-Joint and BINOM from this bound to avoid the nested maximization, reports that these non-myopic acquisitions substantially improve hypervolume over myopic EHVI and JESMO on multiple real-world benchmarks, and shows that instantiating the framework with information-gain scalarizations is possible but that HVI generally performs best.","pith_inferences":["If the variance-only lookahead proxy is unfaithful when the posterior mean shifts after new data, the acquisition scores could be miscalibrated; a direct check would compare them against Monte Carlo sampling of the full posterior update on a small problem.","The additivity of HVI could transfer to other sequential decision problems, such as multi-objective reinforcement learning, wherever a scalar, Pareto-sensitive reward with additive structure is wanted.","A stronger stress test is to compare these acquisitions against an exactly solved Bellman policy on a small discrete input grid with known Pareto front; the paper only reports grid-based experiments for the Nested variant.","The reported gains are measured under short horizons and fixed budgets, so the method's advantage may shrink or reverse if the surrogate model is accurate enough that myopic choices already track the optimal policy."],"forward_implications":["Finite-horizon lookahead becomes feasible for multi-objective Bayesian optimization, and the two scalable variants, Joint and BINOM, outperform myopic EHVI and JESMO on the tested real-world problems.","Longer horizons do not automatically help: hypervolume tends to decline as the lookahead horizon grows from 2 to 8, because prediction uncertainty compounds, so the authors recommend short horizons.","Any improvement-based scalarization inherits the additivity argument, so the framework can be re-instantiated with different reward definitions; information-gain variants work but HVI typically wins in the ablation.","The lower bound contains EHVI as its first term, so optimizing it is at least as tight as the myopic policy in the ideal model setting."],"supporting_citations":[{"why":"Supplies the lower-bound-on-Bellman recursion and batch-selection approximation that this paper adapts from single-objective to multi-objective settings.","marker":"Jiang et al. (2020a)"},{"why":"Defines the (batch) expected hypervolume improvement used as the myopic first term and as the BEHVI lookahead term in the proposed acquisitions.","marker":"Daulton et al. (2020)"},{"why":"Provides the EHVI computation underlying the acquisition functions and the myopic baseline.","marker":"Emmerich & Klinkenberg (2008)"},{"why":"States the scalar, monotone, additive reward conditions the paper must satisfy to apply the Bellman principle.","marker":"Roijers et al. (2013)"},{"why":"Documents the failure of linear scalarization on non-convex Pareto fronts, motivating the HVI scalarization.","marker":"Van Moffaert & Nowé (2014)"},{"why":"Supports the claim that hypervolume alone is not additive, which HVI fixes.","marker":"Van Moffaert et al. (2013)"},{"why":"Provides the JESMO information-theoretic baseline that the proposed methods are compared against and that is used in ablations.","marker":"Tu et al. (2022)"},{"why":"Supplies the implementations used for all baseline acquisition functions in the experiments.","marker":"Balandat et al. (2020)"},{"why":"Supplies the one-step lookahead HVKG baseline whose decoupled cost-aware setting is adapted for comparison.","marker":"Daulton et al. (2023)"}],"fun_headline_variants":["Lookahead planning boosts multi-objective Bayesian optimization","Hypervolume scalarization unlocks non-myopic multi-objective BO","Three non-myopic acquisition functions beat myopic methods on MOBO","Nested, Joint, and BINOM: lookahead acquisitions for MOBO"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The acquisition values assume that, after a hypothetical new observation, the surrogate model's predictions have the same means and only narrower variances, and the resulting lower-bound scores remain faithful to the true posterior; if this one-step lookahead proxy is wrong, the performance gains may not transfer.","fun_headline_variants_meta":{"raw":{"variants":["Lookahead planning boosts multi-objective Bayesian optimization","Hypervolume scalarization unlocks non-myopic multi-objective BO","Three non-myopic acquisition functions beat myopic methods on MOBO","Nested, Joint, and BINOM: lookahead acquisitions for MOBO"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000843,"raw_usage":{"total_tokens":3735,"prompt_tokens":1070,"completion_tokens":2665,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":686,"completion_tokens_details":{"reasoning_tokens":2590}},"tokens_in":686,"tokens_out":2665,"duration_ms":19734,"temperature":1.0,"reasoning_tokens":2590,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:14:58.383176+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a small synthetic problem with a known Pareto front, compute the lookahead term of the acquisition functions by exact Monte Carlo sampling of $y$ from the full posterior conditioned on a new observation $(x,y)$, and compare the induced ranking of candidate inputs with the ranking from the paper's variance-only one-step lookahead proxy; a material difference in rankings would show the lower-bound values are not faithful estimates. In addition, for a horizon of three on a small discrete grid, enumerate the exact Bellman policy and check whether the first input selected by NMMO-Joint matches the optimal first action.","supporting_citations":[{"cited_title":"Hypervolume-based multi-objective reinforcement learning","cited_arxiv_id":null,"evidence_quote":"Supports the claim that hypervolume alone is not additive, which HVI fixes."}],"review_version":1}