{"id":"83839ba9-e50f-4f0b-8393-2b8523378bc1","arxiv_id":"2505.11281","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"The paper extends REMBO to multiple random embeddings modeled jointly by an index-conditioned product-kernel Gaussian process, but provides no empirical evidence that the method works.","lead":"This paper proposes a method that merges several random low-dimensional embeddings into one Gaussian process by adding an embedding-index variable, with the goal of handling nonstationary high-dimensional optimization. The experimental section offers no quantitative results, so the claimed improvement over existing methods is not demonstrated.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central claim is unverified: no quantitative experiments or new theorem establishes that the index-conditioned GP improves REMBO on nonstationary objectives.","rationale":"The original reader rejected the paper as an unfinished draft with an unsubstantiated empirical claim. I agree with the rejection. The most direct load-bearing gap is the absence of any actual experimental result or new theorem: the only 'experiment' is a figure caption, and the 'theoretical implications' are prose. The coverage condition named by the reader is related, but it is not the only issue; even if every local subspace were covered by some A_z, the paper supplies no comparative data showing adaptive selection helps. The arbitrary-order index kernel is an additional internal defect, and the manuscript's own Section 4.4 acknowledges kernel design and acquisition optimization as unresolved limitations. A single reproduction with a permutation control would settle both the empirical claim and the label-order artifact. Credit is due where the paper is honest about its limitations and cites prior components such as multi-task GPs and multiple-embedding REMBO, but that does not establish the central claim.","tokens_in":7721,"tokens_out":8529,"duration_ms":99964,"concrete_test":"Reproduce Section 5 on the 8-dimensional latent, D=21 Styblinski-Tang function using the Section 4.1 product-kernel GP, with K=4 embeddings, 100 function evaluations, and 20 random seeds; report mean and standard error of best objective for SA-REMBO, standard REMBO, and the K-independent-GP ensemble baseline from [8], and repeat with a random permutation of the embedding indices. If SA-REMBO is not significantly better than the baselines, or if permuting indices changes results materially, the central claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that SA-REMBO improves sample efficiency and robustness on nonstationary high-dimensional objectives. For this to hold, the index-conditioned GP must represent the objective in regions the optimizer visits and must be shown empirically to work. Neither is established. Section 4 defines f(x,z)=f(A_z x), but no theorem or condition guarantees that, at each visited region, some A_z gives a usable local projection; the REMBO results quoted in Theorems 1-2 concern a fixed effective subspace and a single embedding, not a locally varying subspace indexed by z. Section 4.3, titled 'Theoretical Implications,' contains only qualitative statements, with no expressiveness or stability theorem. Section 5's entire experimental content is one paragraph and a figure caption: there is no optimization setup, no baseline configuration, no error bars, and no table of values, so the abstract's claim of empirical demonstration is unsupported. Section 4.4 itself lists kernel design and mixed-variable acquisition optimization as limitations without resolving them. Additionally, kz(zi,zj)=exp(-lambda^2(zi-zj)^2) assigns a metric to arbitrary embedding labels, so the model is not invariant under relabeling of embeddings, which undermines the claimed adaptation. In sum, the central claim rests on unverified assumptions and absent evidence.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript proposes SA-REMBO/cREMBO, an extension of REMBO in which multiple random Gaussian embeddings are used and an embedding index z is treated as an additional discrete input to a single Gaussian process surrogate with a product kernel. The paper claims that this construction yields adaptive embedding selection, improved sample efficiency, and robustness on nonstationary high-dimensional objectives, and it promises both theoretical analysis and empirical demonstrations. In the submitted version, the theoretical analysis consists of restatements of known REMBO results, and the experimental content is a single figure with no quantitative results, so the central claims are not supported by the evidence provided.","tokens_in":8021,"tokens_out":4704,"duration_ms":47240,"significance":"The underlying idea of sharing information across multiple random embeddings through an index-conditioned kernel is potentially interesting, since the failure of a single random projection is a recognized limitation of REMBO. If the method were precisely specified, accompanied by a coverage condition ensuring that at least one embedding captures local structure, and validated with controlled benchmark comparisons, it could be a useful contribution. However, as submitted, the significance remains purely conceptual: no new theorem about the proposed kernel is proved, no algorithm is fully specified, and the experimental section contains no numbers, error bars, or protocol details. The positive feature is that the paper coherently restates the standard REMBO theorems in the supplementary material, but those results are not novel and do not concern the proposed product kernel.","major_comments":[{"comment":"The experimental section contains a single subsection and one figure with no quantitative results. There is no table of final objective values, no number of initial points, no optimization budget, no number of independent runs, no error bars or confidence intervals, and no specification of the baseline methods or their configurations. The abstract's claim that the method 'empirically demonstrate[s] the advantage' over REMBO and other low-rank BO methods is therefore unsupported by anything reported in the manuscript. Because the central claim is an empirical advantage, this omission is load-bearing and cannot be fixed by a local edit.","section":"Section 5, Figure 3"},{"comment":"The section titled 'Theoretical Implications' contains only qualitative statements about sample efficiency and posterior relevance; no theorem, proposition, or derivation is stated for the index-conditioned product kernel. The proofs in the Supplementary Material, equations (4) through (17), are restatements of the original REMBO guarantees for a single random embedding and a fixed effective subspace. They do not analyze the product kernel kx·kz, the adaptive selection mechanism, or the stability of the model under varying embeddings. Thus the theoretical contribution promised in the abstract is absent.","section":"Section 4.3 and Supplementary Material"},{"comment":"The kernel kz(zi,zj) = exp(-lambda^2(zi-zj)^2) assigns an Euclidean metric to the discrete, unordered embedding index. Relabeling the embeddings permutes the GP posterior and the acquisition function arbitrarily, so the claimed 'adaptive selection' behavior could be an artifact of the chosen label order rather than of the objective's structure. The paper neither justifies this ordering nor demonstrates invariance under permutation. In addition, no condition is stated to guarantee that, in every region the optimizer visits, at least one of the K embeddings covers the local effective subspace; without such a coverage condition, the surrogate cannot represent the objective in regions where all projections fail.","section":"Section 4.1, kernel definition for z"},{"comment":"The method description is internally inconsistent. Algorithm 1 and the surrounding text describe cREMBO, which augments a single random projection with an orthogonal projection obtained via Gram-Schmidt, while Section 4.1 introduces K arbitrary random embedding matrices and a product-kernel GP over (x,z). No algorithm is provided for the latter formulation: there is no statement of how the K embeddings are generated, which index values are used during training, how the acquisition function is optimized over z, or how the final recommendation is produced. As a result, the proposed method is underspecified and not reproducible from the manuscript.","section":"Section 4 and Algorithm 1"}],"minor_comments":[{"comment":"The title uses 'Adaptive Linear Embedding,' the abstract introduces 'Self-Adaptive cross embedding REMBO (SA-cREMBO),' and Section 4 calls the method 'cross Random Embedding Bayesian Optimization (cREMBO)'; these names should be unified throughout the paper.","section":"Title and Abstract"},{"comment":"Section 5 ends after a single experimental subsection, and Section 6 is a second heading titled 'Method Description' with no content, followed directly by Section 7. This structural error should be corrected.","section":"Section 5 and Section 6"},{"comment":"The text states that Eriksson et al. [8] and Binois et al. [9] 'proposed using multiple random embedding matrices in parallel,' but reference [8] is TuRBO and reference [9] is a heteroscedastic GP method; these citations do not support the claimed related work.","section":"Related Work, references [8] and [9]"},{"comment":"The figure is difficult to interpret: axis labels and legends are unclear, the name 'Harzmann6' appears to be a typo for 'Hartmann6,' and no uncertainty information is displayed.","section":"Figure 3"},{"comment":"In the proof of Theorem 2, the text refers to 'Proposition 1' and 'Proposition 8,' but no propositions are numbered in the main text; these references should be corrected.","section":"Supplementary proofs"}],"recommendation":"reject","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is an unfinished draft. The only idea with any promise—jointly modeling multiple random embeddings in a single GP via a discrete embedding index and a product kernel—is acknowledged in the paper itself as conceptually aligned with Daxberger et al. [12]. Applying that idea to REMBO is a straightforward extension, not a new framework.\n\nWhat the paper does well: it is honest about its debts and its limitations. The Related Work and Section 4.4 explicitly name kernel design and mixed-variable acquisition optimization as open challenges. If the method worked, it would be a modest practical improvement over running several REMBO embeddings independently. But that conditional is doing heavy lifting.\n\nThe soft spots are not subtle. Section 5 is one paragraph and two figures with no numbers, no baselines, no error bars, no protocol. The abstract promises theoretical analysis of expressiveness and stability; Section 4.3 contains only qualitative statements. The theorems in Section 2 are quoted from REMBO [7], and the supplementary proofs are for those existing theorems, not for anything new. The index kernel kz(zi,zj)=exp(-lambda^2(zi-zj)^2) imposes an arbitrary ordering on discrete embedding labels, so the model is not invariant under permutation of embeddings—a property that matters for any claim of adaptive selection. There are also structural problems: \"Method Description\" appears twice, and the supplementary material sits after the references with no pointer from the body.\n\nThe central claim—that the method improves sample efficiency and robustness on nonstationary high-dimensional objectives—is unverified. No quantitative experiment supports it, and no theorem establishes that the index-conditioned GP can represent locally varying effective subspaces. The paper is not circular in the \"fit labeled as prediction\" sense, but it is not reproducible or checkable as submitted.\n\nWho is this for? Possibly a workshop poster if the authors add honest experiments and compare against a Daxberger-style joint model. As it stands, I would not cite it or bring it to a reading group. If I were the editor, I would desk-reject and invite a resubmission once the experiments and theory are actually written.\n\nRecommendation: reject without peer review.","headline":"Unfinished draft whose one new idea is explicitly acknowledged as known mixed-variable BO, with no quantitative experiments or new theory to back the central claim.","tokens_in":8475,"tokens_out":2347,"would_cite":false,"duration_ms":23480,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a product kernel over an embedding index lets Bayesian optimization switch linear projections region by region.","keywords":["Bayesian optimization","random linear embeddings","Gaussian process surrogate","product kernel","adaptive subspace selection","nonstationary objective","high-dimensional optimization","heteroscedasticity"],"falsifier":"Build a synthetic objective whose active directions change by region (e.g., in a 20-dimensional box, only coordinates 1-5 matter when x_1<0 and only coordinates 6-10 matter when x_1>0), run SA-cREMBO with K random embeddings small enough that some regions are almost surely uncovered, and record the selected index at each evaluation. If the surrogate still reduces uncertainty and finds the optimum where no embedding contains the local active coordinates, the local-coverage assumption is not the bottleneck; if performance collapses or the selected embedding is uncorrelated with the region's active subspace, the paper's central mechanism fails.","tokens_in":7559,"feed_emoji":"📐","tokens_out":10261,"duration_ms":96354,"temperature":0.7,"pith_summary":"REMBO (Random Embedding Bayesian Optimization) assumes one random linear projection into a fixed low-dimensional space, which is too rigid for objectives whose effective low-dimensional structure changes from region to region. The paper proposes self-adaptive cross-embedding REMBO (SA-cREMBO), which keeps K random Gaussian embeddings A_1,...,A_K and models the objective as f(A_z x), where z is a discrete embedding index. The GP surrogate uses a product kernel on (x,z), with a smooth exponential kernel on the index, so one model shares information across all embeddings while learning which embedding is reliable where. If correct, this lets the optimizer adaptively select the projection at each acquisition step, improving sample efficiency and robustness on nonstationary, heteroscedastic, high-dimensional landscapes where a single global embedding fails.","feed_headline":"One shared GP lets Bayesian optimization switch projections per region","feed_subtitle":"A single GP shares data across embeddings to tame nonstationary high-dimensional problems.","key_machinery":"The load-bearing object is the index-conditioned product kernel K((x_i,z_i),(x_j,z_j)) = K_x(x_i,x_j) * K_z(z_i,z_j) on the augmented space of low-dimensional points x and embedding indices z. Here K_x is an ARD kernel (Matern or squared exponential) over the latent optimization variable, and K_z is a smooth exponential kernel exp(-$lambda^{2}$ (z_i-z_j)^2) over the discrete embedding index. The product form is what lets one GP share data across K random Gaussian embeddings: it treats z as an extra input, so the surrogate can learn which embeddings correlate with which regions and weight them accordingly at acquisition time. The paper also motivates an orthogonalized 'cross embedding' variant obtained by orthogonalizing the random projection matrix, but the adaptive kernel is the mechanism that carries the central claim.","core_discovery":"The central discovery is that the embedding itself can be treated as a model choice rather than a fixed preprocessing step. SA-cREMBO defines the surrogate as f(x,z):=f(A_z x) over the augmented space R^d times {1,...,K} and factorizes the GP kernel as K((x_i,z_i),(x_j,z_j)) = K_x(x_i,x_j) * K_z(z_i,z_j), with K_z(z_i,z_j)=exp(-$lambda^{2}$ (z_i-z_j)^2). Because all observations feed a single GP, information from every embedding contributes to every prediction, and the learned correlations in the kernel let the model up-weight embeddings that locally capture the objective's active subspace. The paper states this formulation enables adaptive selection among multiple subspaces and mitigates the limitation of a single projection in standard REMBO, and presents it as a way to capture locally varying effective dimensionality, nonstationarity, and heteroscedasticity.","pith_inferences":["A testable extension, not stated in the paper: replace the smooth scalar kernel on z with learned latent vectors for each embedding, which would remove the arbitrary ordering that the exponential kernel exp(-lambda^2 (z_i-z_j)^2) imposes on indices that are actually unordered.","If local coverage is the true bottleneck, the method's gain should scale with how the K embeddings are drawn: a greedy scheme that adds a new embedding when the GP's predictive variance stays high inside a region could outperform fixed random draws, though that is an inference beyond the paper.","The framework suggests a bridge to heteroscedastic GP modeling: the index dimension can act as a discrete 'regime' label, so the same kernel machinery could model nonstationary variance, though the paper does not demonstrate this directly.","A natural stress test would be a benchmark where each region's active subspace is known; then one can measure whether the learned K_z actually up-weights the embedding whose projection contains that subspace, which the paper does not report."],"forward_implications":["Under one GP, every objective evaluation updates the model for all K embeddings, so SA-cREMBO should reach a given regret with fewer total evaluations than K independent REMBO runs.","The learned index kernel provides a location-dependent posterior over embedding relevance, so the acquisition step can switch projections without a separate model-selection loop.","Standard REMBO is recovered as a special case (K=1 or a delta kernel over z), making the framework a strict generalization rather than a competing algorithm.","On objectives whose active directions differ across regions—the nonstationary and heteroscedastic cases the paper targets—the method should outperform both single-embedding REMBO and ensemble approaches that train separate surrogates.","Because bad embeddings are down-weighted rather than discarded, the optimizer gains robustness against the failure of any single random projection."],"supporting_citations":[{"why":"Supplies the random-embedding setup, the effective-subspace decomposition, and the subspace-intersection guarantee that SA-cREMBO generalizes from a single projection to K projections.","marker":"[7]"},{"why":"Cited as an ensemble-style method that explores multiple embeddings in parallel but trains separate surrogates, the baseline whose computational cost and information fragmentation the paper addresses.","marker":"[8]"},{"why":"Cited as part of the ensemble-embedding motivation for robustness through multiple parallel explorations, which the paper contrasts with joint modeling.","marker":"[9]"},{"why":"Introduced joint GP modeling with a discrete embedding identifier, which the paper formalizes inside REMBO as the index-conditioned surrogate.","marker":"[12]"},{"why":"Motivates the product-kernel treatment of discrete latent inputs by showing how correlations across tasks can be captured in a single GP.","marker":"[13]"},{"why":"Identifies the boundary over-exploration problem and notes that scaling embeddings worsens it, supporting the paper's motivation for structured or multiple embeddings rather than one scaled projection.","marker":"[14]"},{"why":"Shows that REMBO does not guarantee the global optimum lies in the projected bounded box, one reason a single fixed embedding can fail and multiple adaptive embeddings are needed.","marker":"[15]"},{"why":"Provides the matrix condition-number bound used in the paper's proof that the random projection preserves the optimizer with high probability.","marker":"[16]"}],"fun_headline_variants":["Adaptive embeddings let Bayesian optimization switch subspaces on the fly","One GP learns which linear projection fits each region","SA-REMBO: Bayesian optimization adapts its embedding per location","Shared Gaussian process picks the best subspace for each point","Kernel chooses embedding locally for nonstationary Bayesian optimization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's central assumption is that in every region the optimizer visits, at least one of the K random linear projections happens to align with the directions that actually matter there, and that the discrete embedding index can be treated by a smooth exponential kernel even though the index is unordered; the paper does not analyze the probability of either condition.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive embeddings let Bayesian optimization switch subspaces on the fly","One GP learns which linear projection fits each region","SA-REMBO: Bayesian optimization adapts its embedding per location","Shared Gaussian process picks the best subspace for each point","Kernel chooses embedding locally for nonstationary Bayesian optimization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000609,"raw_usage":{"total_tokens":2831,"prompt_tokens":938,"completion_tokens":1893,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":554,"completion_tokens_details":{"reasoning_tokens":1812}},"tokens_in":554,"tokens_out":1893,"duration_ms":13438,"temperature":1.0,"reasoning_tokens":1812,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:53:56.018400+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a synthetic objective whose active directions change by region (e.g., in a 20-dimensional box, only coordinates 1-5 matter when x_1<0 and only coordinates 6-10 matter when x_1>0), run SA-cREMBO with K random embeddings small enough that some regions are almost surely uncovered, and record the selected index at each evaluation. If the surrogate still reduces uncertainty and finds the optimum where no embedding contains the local active coordinates, the local-coverage assumption is not the bottleneck; if performance collapses or the selected embedding is uncorrelated with the region's active subspace, the paper's central mechanism fails.","supporting_citations":[{"cited_title":"Bayesian optimization in high dimensions via random embeddings,","cited_arxiv_id":null,"evidence_quote":"Supplies the random-embedding setup, the effective-subspace decomposition, and the subspace-intersection guarantee that SA-cREMBO generalizes from a single projection to K projections."},{"cited_title":"Scalable global optimization via local bayesian optimization,","cited_arxiv_id":null,"evidence_quote":"Cited as an ensemble-style method that explores multiple embeddings in parallel but trains separate surrogates, the baseline whose computational cost and information fragmentation the paper addresses."},{"cited_title":"Practical heteroscedastic gaussian process modeling for large simulation experiments,","cited_arxiv_id":null,"evidence_quote":"Cited as part of the ensemble-embedding motivation for robustness through multiple parallel explorations, which the paper contrasts with joint modeling."},{"cited_title":"Multi-task bayesian optimization,","cited_arxiv_id":null,"evidence_quote":"Motivates the product-kernel treatment of discrete latent inputs by showing how correlations across tasks can be captured in a single GP."},{"cited_title":"Bayesian optimization in a billion dimensions via random embeddings,","cited_arxiv_id":null,"evidence_quote":"Identifies the boundary over-exploration problem and notes that scaling embeddings worsens it, supporting the paper's motivation for structured or multiple embeddings rather than one scaled projection."},{"cited_title":"Smoothed analysis of the condition numbers and growth factors of matrices,","cited_arxiv_id":null,"evidence_quote":"Provides the matrix condition-number bound used in the paper's proof that the random projection preserves the optimizer with high probability."}],"review_version":1}