Bayesian Optimization for Enhanced Language Models: Optimizing Acquisition Functions

Changyutao Qiu; Yibo Liu; Zishuo Bao

REVIEW 4 major objections 5 minor 22 references

Bayesian Optimization for Enhanced Language Models: Optimizing Acquisition Functions

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Pairing EI and UCB acquisition functions in nested Bayesian optimization loops improves language-model fine-tuning by 2.7 percent.

desk verdict The paper reports a 2.7% GLUE gain from a bilevel EI-UCB BO scheme, but the method is never coherently described: Section 5.3 describes only single-level BO with EI, and Sections 3, 4, and 6 disagree on which acquisition function sits in which loop. read the letter →

arxiv 2505.17151 v1 pith:WMOWH2MR submitted 2025-05-22 cs.CL cs.AI

Zishuo Bao , Yibo Liu , Changyutao Qiu This is my paper · ORCID

classification cs.CLcs.AI

keywords Bayesianoptimizationacquisitionfunctionbilevellanguagemodelfine-tuningfusionGLUEbenchmarkRoBERTahyperparameter

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the choice and arrangement of acquisition functions inside a bilevel Bayesian optimization loop materially change how well a fine-tuned language model generalizes. It claims that pairing Expected Improvement (EI) in the inner loop with Upper Confidence Bound (UCB) in the outer loop yields an average GLUE score of 76.82 for RoBERTa-base, beating standard fine-tuning's 74.80 by 2.7 percent. A sympathetic reader would care because fine-tuning is expensive and hyperparameter choice is usually treated as a generic search problem, not as a design decision about exploration versus exploitation. If the pairing claim holds, it gives practitioners a concrete, low-cost lever for improving fine-tuning without changing model architecture.

What carries the argument

The load-bearing mechanism is a bilevel Bayesian optimization loop in which two different acquisition functions are assigned to two nested levels: an inner loop that minimizes training loss and an outer loop that optimizes validation metric. Expected Improvement (EI), which prefers regions with high improvement potential, and Upper Confidence Bound (UCB), which adds an uncertainty term $\kappa\sigma(\theta)$ to the predicted mean, are the two acquisition functions combined into configurations such as EI-UCB. A Gaussian Process surrogate models the validation-performance landscape, and the nested structure is meant to let the inner loop refine model parameters while the outer loop gathers better hyperparameters, with the acquisition-function pairing determining how exploration and exploitation are balanced at each level. This machinery carries the argument because the reported gains are attributed specifically to the interaction of EI and UCB in the two loops.

What would settle it

Re-run the four GLUE tasks with a single-level GP-EI search (as Section 5.3 describes) and compare the average with the reported 76.82; if single-level EI reaches the same score, the benefit attributed to the EI-UCB pairing disappears. Alternatively, inspect the implementation to confirm that two nested acquisition-function loops exist and that the reported numbers come from that procedure.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that Bilevel-BO-SWA—a framework that couples model fusion with bilevel Bayesian optimization—improves language-model fine-tuning, and that the configuration using EI in the inner optimization loop and UCB in the outer loop achieves the best results: an average score of 76.82 on RTE, MRPC, CoLA, and STS-B, 2.7 percent above standard fine-tuning. The paper further claims that this EI-UCB arrangement also produces the lowest average loss (0.76) across the four tasks, and that the search space of the best configuration is 'notably larger' than alternatives, which it ties to more thorough exploration. The authors present this as validation that acquisition-function design, not just the choice of hyperparameters, is a meaningful axis of optimization for large language models.

Load-bearing premise

The claim depends on the assumption that the experiments actually ran a two-level optimization with EI in one loop and UCB in the other, rather than a single standard Bayesian optimization search with one acquisition function.

Editorial extensions

If this is right

If the central claim is right, then acquisition-function pairing is a transferable design choice for fine-tuning other transformer models.
The 2.7 percent gain implies that replacing a single acquisition function with a complementary pair could improve downstream task performance at no architectural cost.
The bilevel scheme's success on four GLUE tasks suggests it is worth testing on the full nine-task GLUE suite and on larger models.
Because the best configuration also has the lowest average loss, the pairing may improve both optimization and generalization simultaneously.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper does not report standard deviations or multiple seeds, so a natural next test is whether the 2.7 percent gap is stable across random seeds; the fixed-seed setup leaves the improvement vulnerable to noise.
The description in Section 5.3 of a standard GP-EI search sits uneasily with the bilevel EI-UCB narrative; if the implementation is in fact single-level, the paper's contribution would reduce to a conventional BO fine-tuning study with a different framing.
The SWA component is introduced but not isolated; a reader could test whether the gain comes from weight averaging rather than from the acquisition-function pairing by running EI-UCB without SWA.
The authors themselves note (Section 7) that only RoBERTa was tested and that the overhead of a GP model can exceed the benefit when only one or two hyperparameters matter; both caveats bound the scope of the 2.7 percent claim.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

What you should know: this paper is not a coherent report of a bilevel EI-UCB procedure. The central empirical result—76.82 average on four GLUE tasks versus 74.80 for fine-tuning—is real in the sense that the table exists, but the method that supposedly produced it is never pinned down, and the experimental description contradicts the headline claim.

What is genuinely new: the idea of pairing EI and UCB in separate nested loops for LLM hyperparameter search is a reasonable extension of standard BO. The paper also compares several configurations (EI, UCB, EI-UCB, UCB-EI, single-level) on a small GLUE subset, which is a useful check on whether acquisition-function choice matters at all. That is the extent of the credit.

The soft spots are load-bearing. Section 3.1 says EI goes in the outer loop and UCB in the inner loop; Section 3.2 says UCB is suitable for the inner loop; Section 4 says the inner-EI/outer-UCB configuration performs best; Section 6 defines "EI-UCB" as EI in the inner loop and UCB in the outer loop. Section 5.3, the only concrete setup, says: "We use a Gaussian Process surrogate with Expected Improvement as the acquisition function, running 50 trials." No inner loop, no UCB, no SWA. SWA is in the title and abstract but never described as a fusion step anywhere in the experiments. There is no algorithm or pseudocode and no code release. Results are from a single seed with no error bars, and early stopping uses the dev set that is then reported as the outcome—an optimistic-bias risk the paper does not address. The limitations section is honestly worded, but it concedes that traditional grid search may beat BO when only a few hyperparameters are tuned, which further undercuts the claimed advantage.

Who this is for: a reader interested in the broad question of acquisition-function selection in BO for NLP fine-tuning could skim the comparative table for inspiration. As a contribution, though, it does not hold together; a serious referee would have to reconstruct the method from contradictory text, which is not an acceptable starting point.

I would desk-reject this one. The underlying question is worth a better paper, but this manuscript is not ready for review.

Referee Report

4 major / 5 minor

Summary. The paper proposes Bilevel-BO-SWA, a bilevel Bayesian optimization framework for fine-tuning RoBERTa-base on GLUE tasks, in which different acquisition functions (Expected Improvement and Upper Confidence Bound) are assigned to inner and outer optimization loops, combined with stochastic weight averaging. The central empirical claim is that the EI-UCB configuration achieves an average GLUE score of 76.82 versus 74.80 for standard fine-tuning, an improvement of 2.7%, reported in the abstract, Section 6, and Table 1. The paper does not provide pseudocode, an algorithmic description of the nested loops, or a description of how SWA is integrated.

Significance. If the described method were well defined and the experiments reproducible, the question of how acquisition functions should be paired across nested optimization loops for language-model fine-tuning would be a legitimate and reasonably interesting contribution. However, as written, the paper does not define the claimed bilevel EI-UCB procedure, does not describe the SWA component, presents results from a single fixed seed without error bars, and contains direct contradictions between the method description and the experimental setup. These problems concern the central claim rather than presentation, so the paper in its current form does not provide evidence for its headline result.

major comments (4)

[Section 3 vs. Section 6] The assignment of acquisition functions to loops is internally inconsistent. Section 3.1 states that EI is applied to the outer optimization loop and Section 3.2 states that UCB is suitable for the inner loop, while Section 6 defines 'EI-UCB' as EI in the inner loop and UCB in the outer loop (and 'UCB-EI' as the reverse). The paper never resolves this contradiction or states which assignment was actually used in the reported experiments.
[Section 5.3] The only concrete experimental description says: 'We use a Gaussian Process surrogate with Expected Improvement as the acquisition function, running 50 trials.' This is a standard single-level BO setup; there is no inner loop, no UCB acquisition function, no bilevel structure, and no SWA or model-fusion step. Consequently, Table 1's rows labeled UCB-EI and EI-UCB cannot be attributed to the proposed bilevel method, and the central claim that pairing EI and UCB across nested loops improves fine-tuning is unsupported by the described experiments.
[Section 4 and throughout] The name 'Bilevel-BO-SWA' promises stochastic weight averaging as a model-fusion component, but SWA is never formally defined or described anywhere in the manuscript. There is no equation, algorithm, or experimental detail showing how SWA is combined with the Bayesian optimization procedure, so the experiments are not reproducible from the text.
[Section 6, Table 1] All runs use a fixed random seed and no variance or confidence information is reported. Many of the differences between configurations are small (e.g., EI at 75.52 versus UCB-EI at 75.45, or RTE values of 70.8 versus 70.9), and without multiple seeds or error bars it is impossible to know whether the reported 2.7% improvement over fine-tuning is statistically meaningful. In addition, the 'single-level' baseline row is not described anywhere in the experimental setup.

minor comments (5)

[Abstract and Section 1] The text contains numerous grammatical and typographical errors, for example 'down stream tasks Model gets messy' in the abstract and 'Bilievel-BO' in the keywords; the manuscript would benefit from careful proofreading.
[Table 1] The 'Imp. Rate (%)' column is computed inconsistently: for the single-level row, 1.18 is the absolute improvement in average score, while for EI-UCB, 2.70 is the relative percentage improvement; the column should use one consistent definition.
[References] The reference list contains duplicates (Devlin et al. appears as [8] and [13]; Wang et al. as [11] and [17]; Snoek et al. as [9] and [21]) and reference [6] is cited for the UCB formula but is actually a paper on predictive entropy search; the citations need to be cleaned up.
[Figure 3] The caption says 'Cumulative Maximum Accuracy Comparison,' but the surrounding text says the figure tracks loss and accuracy across trials; the figure itself is not described in enough detail to interpret the convergence claim.
[Section 2] The sentence 'RditionalBO relies on acquisition functions' appears to be a typo for 'Traditional BO'; in addition, the claim that SWA is less effective in NLP 'due to differences in loss functions like cross-entropy in NLP versus softmax loss used in vision tasks' is not substantiated by a citation or analysis.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim is an empirical benchmark comparison and the acquisition-function formulas are standard, not fitted to the result.

full rationale

The paper's central assertion (EI-UCB reaches 76.82 average GLUE score vs. 74.80 for fine-tuning) is a direct experimental result reported in Table 1, not a derivation from its own equations. The EI and UCB formulas given in Sections 3.1 and 3.2 are standard definitions quoted from the literature and contain no fitted constants or target scores. No parameter is fitted to the reported accuracy, and no self-citation is load-bearing: the cited works are external references (e.g., Snoek et al. for BO, Liu et al. for LLM-assisted BO, Wang et al. for GLUE). The paper's internal inconsistency about whether EI-UCB means EI in the inner loop or outer loop (compare Section 3.1, Section 4, and Section 6) is a methodological clarity or correctness problem, not circularity: even if the naming is confused, the empirical comparison does not reduce by construction to its inputs. Therefore the circularity score is 0.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

No new physical or conceptual entities are introduced; Bilevel-BO-SWA is a method label rather than a new entity. The framework's load-bearing parts rest on standard BO assumptions and an unspecified fusion step.

free parameters (4)

UCB exploration weight kappa = unspecified
Controls the exploration-exploitation trade-off in UCB; the paper defines UCB = mu + kappa sigma but never gives the value used in experiments, despite the central claim depending on the acquisition function behavior.
Number of BO trials = 50
Hand-chosen; affects the best found hyperparameters and hence the reported improvement.
Early stopping patience = 3
Hand-chosen; interacts with the reported dev-set scores.
Search space bounds (learning rate, batch size, weight decay) = lr [1e-6,1e-5], batch {8,32}, weight decay [0.0,0.1]
Hand-chosen ranges; the central result is conditional on this search space, and weight decay is uniform [0.0,0.1], which affects the BO trajectory.

assumptions (3)

domain assumption Gaussian process surrogate accurately models validation performance
The entire BO framework assumes the GP posterior is a faithful surrogate for the validation metric in the searched hyperparameter space; no validation of this assumption is given.
ad hoc to paper Bilevel decomposition (outer metric, inner loss) is a valid and correctly implemented optimization structure
The paper gives a generic bilevel formulation but does not specify how the inner and outer loops are instantiated, and Section 5.3 contradicts it by describing a GP with EI as the acquisition function.
ad hoc to paper SWA/model fusion improves fine-tuning in this setting
The method is named Bilevel-BO-SWA, but the paper never describes how stochastic weight averaging is applied, which checkpoints are fused, or how fusion interacts with the BO loop.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Bayesian Optimization for Enhanced Language Models: Optimizing Acquisition Functions." pith.science (2026). https://pith.science/paper/WMOWH2MR

@misc{pith2026250517151,
  author       = {Pith},
  title        = {Pith review of: Bayesian Optimization for Enhanced Language Models: Optimizing Acquisition Functions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WMOWH2MR}},
  note         = {Machine review of arXiv:2505.17151}
}

read the original abstract

With the rise of different language model architecture, fine-tuning is becoming even more important for down stream tasks Model gets messy, finding proper hyperparameters for fine-tuning. Although BO has been tried for hyperparameter tuning, most of the existing methods are oblivious to the fact that BO relies on careful choices of acquisition functions, which are essential components of BO that guide how much to explore versus exploit during the optimization process; Different acquisition functions have different levels of sensitivity towards training loss and validation performance; existing methods often just apply an acquisition function no matter if the training and validation performance are sensitive to the acquisition function or not. This work introduces{Bilevel - BO - SWA}, a model fusion approach coupled with a bilevel BO strategy to improve the fine - tunning of large language models. Our work on mixture of acquisition functions like EI and UCB into nested opt loops, where inner loop perform minimization of training loss while outer loops optimized w.r.t. val metric. Experiments on GLUE tasks using RoBERTA - base show that when using EI and UCB, there is an improvement in generalization, and fine - tuning can be improved by up to 2.7%.

Figures

Figures reproduced from arXiv: 2505.17151 by the authors.

**Figure 2.** Schematic illustration of the Bilevel Bayesian Optimization framework. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** Cumulative Maximum Accuracy Comparison [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 16 canonical work pages

[1]

Jang, C., Lee, H., Kim, J., Lee, J.: Model Fusion through Bayesian Optimization in Language Model Fine-Tuning.arXiv preprint arXiv:2411.06710v2(2024)

work page Pith review arXiv 2024
[2]

Liu, T., Astorga, N., Seedat, N., van der Schaar, M.: Large Language Models to Enhance Bayesian Optimization.arXiv preprint arXiv:2402.03921v2(2024)

arXiv 2024
[3]

Zhang, Y., Khanduri, P., Tsaknakis, I., Yao, Y., Hong, M., Liu, S.: An Introduction to Bilevel Optimization: Foundations and Applications in Signal Processing and Machine Learning.IEEE Transactions on Signal Processing(2023)

work page 2023
[4]

Shen, H., Chen, P.-Y., Das, P., Chen, T.: SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection.arXiv preprint arXiv:2410.07471v2(2024)

arXiv 2024
[5]

Jiang, D., Ren, X., Lin, B.Y.: LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion.arXiv preprint arXiv:2306.02561v3 (2023)

arXiv 2023
[6]

Hernández-Lobato, D., Hernández-Lobato, J.M., Shah, A., Adams, R.P.: Predic- tive Entropy Search for Multi-objective Bayesian Optimization.arXiv preprint arXiv:1511.05467v3(2016)

work page Pith review arXiv 2016
[7]

Daulton, S., Eriksson, D., Balandat, M., Bakshy, E.: Multi-Objective Bayesian Optimization over High-Dimensional Search Spaces.arXiv preprint arXiv:2109.10964v4(2022)

arXiv 2022
[8]

In:Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp

Devlin,J.,Chang,M.W.,Lee,K.,Toutanova,K.:BERT:Pre-trainingofDeepBidi- rectional Transformers for Language Understanding. In:Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4171–4186 (2019)

work page 2019

Show all 22 references

[9]

In:Advances in Neural Information Processing Systems (NeurIPS), vol

Snoek, J., Larochelle, H., Adams, R.P.: Practical Bayesian Optimization of Ma- chine Learning Algorithms. In:Advances in Neural Information Processing Systems (NeurIPS), vol. 25, pp. 2951–2959 (2012)

2012
[10]

In:Automated Machine Learning, Springer, Cham, pp

Feurer, M., Hutter, F.: Hyperparameter Optimization. In:Automated Machine Learning, Springer, Cham, pp. 3–33 (2019)

2019
[11]

In: International Conference on Learning Representations (ICLR)(2019)

Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., Bowman, S.R.: GLUE: A Multi- Task Benchmark and Analysis Platform for Natural Language Understanding. In: International Conference on Learning Representations (ICLR)(2019)

2019
[12]

In:Conference on Uncertainty in Artificial Intelligence (UAI)(2018)

Izmailov, P., Podoprikhin, D., Garipov, T., Vetrov, D., Wilson, A.G.: Averaging Weights Leads to Wider Optima and Better Generalization. In:Conference on Uncertainty in Artificial Intelligence (UAI)(2018)

2018
[13]

Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In:Proceedings of the 2019 Conference of the North American Chapter of the Association for Compu- tational Linguistics: Human Language Technologi...

2019
[14]

In:arXiv:1907.11692 [cs.CL](2019) 12 Zishuo Bao 1,†, Yibo Liu2,†, Changyutao Qiu3,†

Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., Stoyanov, V.: RoBERTa: A Robustly Optimized BERT Pretrain- ing Approach. In:arXiv:1907.11692 [cs.CL](2019) 12 Zishuo Bao 1,†, Yibo Liu2,†, Changyutao Qiu3,†

2019 arXiv
[15]

In:arXiv:1807.01774 [cs.LG](2018)

Falkner, S., Klein, A., Hutter, F.: BOHB: Robust and Efficient Hyperparameter Optimization at Scale. In:arXiv:1807.01774 [cs.LG](2018)

2018 arXiv
[16]

In:IEEE Transactions on Automation Science and Engineering, vol

Shahriari, B., Swersky, K., Wang, Z., Adams, R.P., de Freitas, N.: Taking the Hu- man Out of the Loop: A Review of Bayesian Optimization. In:IEEE Transactions on Automation Science and Engineering, vol. 13, no. 3, pp. 1204–1223 (2016)

2016
[17]

In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp

Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., Bowman, S.R.: GLUE: A Multi- Task Benchmark and Analysis Platform for Natural Language Understanding. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 353–355 (2018)

2018
[18]

In:Interna- tional Conference on Learning Representations (ICLR)(2019)

Loshchilov, I., Hutter, F.: Decoupled Weight Decay Regularization. In:Interna- tional Conference on Learning Representations (ICLR)(2019)

2019
[19]

In:OpenAI Blog, 2018

Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I.: Improving language understanding by generative pre-training. In:OpenAI Blog, 2018

2018
[20]

and Ruder, S.: Universal Language Model Fine-tuning for Text Clas- sification

Howard, J. and Ruder, S.: Universal Language Model Fine-tuning for Text Clas- sification. In:Proceedings of the 56th Annual Meeting of the Association for Com- putational Linguistics (ACL), pp. 328-339, 2018

2018
[21]

P.: Practical Bayesian Optimization of Machine Learning Algorithms

Snoek, J., Larochelle, H., and Adams, R. P.: Practical Bayesian Optimization of Machine Learning Algorithms. In:Advances in Neural Information Processing Sys- tems (NeurIPS), pp. 2951-2959, 2012

2012
[22]

1521–1530 (2016)

González, J., Dai, Z., Hennig, P., Lawrence, N.D.: Predictive Entropy Search for Multi-objective Bayesian Optimization.Proceedings of the International Confer- ence on Machine Learning (ICML), pp. 1521–1530 (2016)

2016

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Jang, C., Lee, H., Kim, J., Lee, J.: Model Fusion through Bayesian Optimization in Language Model Fine-Tuning.arXiv preprint arXiv:2411.06710v2(2024)

work page Pith review arXiv 2024

[2] [2]

Liu, T., Astorga, N., Seedat, N., van der Schaar, M.: Large Language Models to Enhance Bayesian Optimization.arXiv preprint arXiv:2402.03921v2(2024)

arXiv 2024

[3] [3]

Zhang, Y., Khanduri, P., Tsaknakis, I., Yao, Y., Hong, M., Liu, S.: An Introduction to Bilevel Optimization: Foundations and Applications in Signal Processing and Machine Learning.IEEE Transactions on Signal Processing(2023)

work page 2023

[4] [4]

Shen, H., Chen, P.-Y., Das, P., Chen, T.: SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection.arXiv preprint arXiv:2410.07471v2(2024)

arXiv 2024

[5] [5]

Jiang, D., Ren, X., Lin, B.Y.: LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion.arXiv preprint arXiv:2306.02561v3 (2023)

arXiv 2023

[6] [6]

Hernández-Lobato, D., Hernández-Lobato, J.M., Shah, A., Adams, R.P.: Predic- tive Entropy Search for Multi-objective Bayesian Optimization.arXiv preprint arXiv:1511.05467v3(2016)

work page Pith review arXiv 2016

[7] [7]

Daulton, S., Eriksson, D., Balandat, M., Bakshy, E.: Multi-Objective Bayesian Optimization over High-Dimensional Search Spaces.arXiv preprint arXiv:2109.10964v4(2022)

arXiv 2022

[8] [8]

In:Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp

Devlin,J.,Chang,M.W.,Lee,K.,Toutanova,K.:BERT:Pre-trainingofDeepBidi- rectional Transformers for Language Understanding. In:Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4171–4186 (2019)

work page 2019