Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Dynamic Fisher-weighted Model Merging via Bayesian Optimization

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read DF-Merge unifies model-wise scaling and parameter-wise importance weighting in one merging formula, tunes per-task coefficients by Bayesian optimization, and cuts the gap to multi-task fine-tuning to about three points on T5.

desk verdict A clean unification and a sensible new merge method, but the headline empirical comparison is confounded by an unequal validation-search budget. read the letter →

arxiv 2504.18992 v1 pith:YR2TE2UD submitted 2025-04-26 cs.CL

Sanwoo Lee , Jiahao Liu , Qifan Wang , Jingang Wang , Xunliang Cai , Yunfang Wu This is my paper · ORCID
classification cs.CL
keywords modelmergingtaskarithmeticFisherinformationBayesianoptimizationmulti-tasklearningparameterinterferenceT5validationefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that the two dominant families of model merging, scaling whole task vectors and weighting individual parameters by importance, are special cases of one linear merging formula, and that filling in the free coefficients with Bayesian optimization yields a better multi-task model than either family on its own. The proposed method, DF-Merge, scales each fine-tuned model along its task vector, estimates Fisher information at the scaled position, and merges by Fisher-weighted averaging, with Gaussian-process Bayesian optimization choosing the coefficients from validation-set accuracy. The authors report that this beats strong baselines on T5-base and T5-large across six NLP tasks and closes most of the gap to multi-task fine-tuning. The practical interest is that merging builds a multi-task model without training data or joint training, so a small accuracy gap versus multi-task learning is the price of a dramatic saving in data and compute.

What carries the argument

The central object is the generalized merging operator $f = (\sum_i C_{\theta_i})^{-1} (\sum_i C_{\theta_i} \lambda_i \tau_i) + \theta_{\text{pre}}$, where $C_{\theta_i}$ is a covariance matrix and $\tau_i = \theta_i - \theta_{\text{pre}}$ is the task vector. Setting $C = I$ recovers Task Arithmetic, model-wise scaling, while setting $C = \operatorname{diag}(\widehat{F}_{\theta_i})$ and $\lambda_i = 1/M$ recovers Fisher Merging, parameter-wise importance weighting. DF-Merge takes $C_{\theta_i} = \operatorname{diag}(\widehat{F}_{\theta_i(\lambda_i)})$, meaning Fisher information is computed at the scaled model $\theta_{\text{pre}} + \lambda_i \tau_i$, and lets Bayesian optimization choose the $\lambda_i$ to maximize average validation accuracy. The merge function converts the two prior strategies into one objective, Fisher information imports the low-loss-basin intuition, and Bayesian optimization performs the coefficient search.

What would settle it

Run DF-Merge with the same budget as the grid-search baselines, about 11 merge-then-evaluate rounds, on the same six T5-base tasks and check whether it still beats Task Arithmetic by the reported 4.48 points; if the margin collapses to near zero, the dynamic Fisher weighting is not what carries the result.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that model-wise scaling and parameter-wise importance weighting are not competing merge recipes but two restricted cases of a single formula, and that the unrestricted version, dynamic Fisher-weighted merging, dominates both. DF-Merge assigns each fine-tuned model a coefficient $\lambda_i$, computes diagonal Fisher information at the interpolated model $\theta_{\text{pre}} + \lambda_i \tau_i$, and uses Gaussian-process Bayesian optimization, with Expected Improvement or Upper Confidence Bound acquisition, to pick coefficients that maximize average held-out accuracy. On T5-base and T5-large across PAWS, QASC, QuaRTz, Story Cloze, WikiQA and Winogrande, it outperforms all baselines in average accuracy, best baseline 73.66 versus 78.14 for T5-base and 81.86 versus 83.59 for T5-large, and narrows the gap with multi-task fine-tuning to 3.55 and 3.15 points. The paper further claims that near-optimal performance is reached within a few iterations, 9 optimization rounds after 10 random points, and with as little as 5% of each validation set, and that ablations show both Fisher information and Bayesian optimization are necessary components.

Load-bearing premise

The claim of superiority assumes the comparison spends the same validation-evaluation budget: DF-Merge receives 60 coefficient evaluations, 10 random points plus 50 optimization rounds, while Task Arithmetic and TIES-Merging get a grid search of about 11 steps, so much of the gain could come from the larger budget rather than from the dynamic Fisher objective.

Editorial extensions

If this is right

  • If DF-Merge generalizes as reported, a user with several fine-tuned models sharing an initialization can build a multi-task model from parameter vectors and small validation sets alone, needing no training data and no joint training.
  • The resulting multi-task model can be optimized directly for non-differentiable metrics such as accuracy, since Bayesian optimization only requires the scalar validation score.
  • Early stopping is viable: after 10 random initial points, 9 optimization iterations already reach within 0.59 percentage points of the best 60-iteration result on T5-base.
  • Only a small validation budget is needed: 5% of each validation set suffices to approach full-validation performance and to outpace Task Arithmetic.
  • The gap to multi-task fine-tuning can shrink to 3.55 points for T5-base and 3.15 points for T5-large, making merging a plausible training-free substitute when a slight accuracy drop is acceptable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A controlled comparison that equalizes the validation-evaluation budget, 60 merge-then-evaluate rounds for DF-Merge versus about 11 grid-search steps for the baselines, is not reported; such a comparison could materially reduce the measured margin and is the natural next test of the paper's central claim.
  • The paper's own diagonal-Fisher assumption treats parameters as independent; extending the merge function to block-diagonal Fisher approximations, a direction the paper mentions, could further close the residual gap to multi-task fine-tuning.
  • Because Fisher information is computed from the model's predictive distribution rather than from labels, DF-Merge could plausibly be adapted to label-free merging where fine-tuned models pseudo-label test inputs, an extension the paper notes but does not implement.
  • The coefficient search could in principle transfer to models with different initializations or architectures if an alignment step were inserted, but the paper deliberately restricts itself to the shared-initialization setting.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes DF-Merge, a model-merging method that unifies model-wise scaling (task arithmetic) and parameter-wise importance weighting (Fisher merging) into a single parameterized objective. A set of per-task coefficients is optimized by Bayesian optimization on held-out validation accuracy, while the Fisher information used for weighting is re-estimated at the coefficient-scaled models. Experiments on T5-base and T5-large across six classification tasks report gains over averaging, Fisher merging, task arithmetic, DARE, and TIES-merging, narrowing the gap to multi-task fine-tuning. The paper also includes ablations, efficiency analyses, and landscape visualizations to support the proposed method.

Significance. If the empirical claims hold, the unified view in Section 2 is a clean and useful conceptual contribution, and the use of Bayesian optimization to select merging coefficients is a practical direction. The algebraic derivations in Section 2 correctly recover averaging, general task arithmetic, and Fisher merging as special cases of Eq. (7), and the paper provides a geometric interpretation that is illuminating. The code is promised to be public, which is helpful for reproducibility. However, the central empirical claim is currently undermined by a validation-evaluation budget confound: DF-Merge is allowed far more evaluations of the validation objective than the grid-search baselines, and the ablations in Table 3 suggest that most of the gain comes from the Bayesian-optimization search itself rather than from the dynamic Fisher weighting. The efficiency and near-optimality analyses are also partly self-referential because they report convergence on the very validation objective being optimized.

major comments (3)
  1. [§4.1 / Table 2] The comparison in Table 2 does not control for the number of validation evaluations used to select coefficients. DF-Merge is run with 10 random initial points plus 50 Bayesian optimization iterations (60 evaluations), while Task Arithmetic and TIES-Merging use a grid search with step 0.1 over intervals of length 1.0 (about 11 evaluations) and DARE adds a 5-point grid over its drop rate. If the extra evaluations are what locate a better coefficient, the reported gains in Table 2 may be a search-budget artifact rather than a property of the dynamic Fisher objective. This is a load-bearing issue because Table 3 shows that the largest component of the gain comes from the Bayesian optimization search (removing it costs 4.90–13.71 points), whereas the dynamic Fisher weighting contributes only 0.85–1.34 points. The authors should either run the baselines with a comparable evaluation budget or report DF-Merge at the same budget (e.g., after the first 11 evaluations) to support the superiority claim.
  2. [§5.2 / Figure 4] The claim that DF-Merge achieves near-optimal performance in a few iterations is self-referential because the trajectory in Figure 4 is plotted on the validation accuracy objective that Bayesian optimization is explicitly maximizing. The comparison is against the best point found in the same 60-iteration run, not against the global optimum or against test-set performance. To make the efficiency claim meaningful, the authors should report test accuracy at early stopping points (e.g., after 10, 15, or 20 evaluations) and compare with the test performance of the baselines under the same evaluation budget. The current analysis shows that the optimizer converges on its own objective, which is expected behavior, not near-optimality of the merged model on held-out data.
  3. [§3 / Eq. (8)] The phrase "dynamic Fisher-weighted" may overstate what Eq. (8) actually does. The Fisher information is re-estimated at the scaled model θ_pre + λ_i τ_i, but the predictive distribution used to compute it is still that of each individual task model, not a distribution conditioned on the merging coefficients in a way that couples tasks. The mechanism is plausible, but the paper does not provide evidence that this re-estimation, rather than the extra flexibility of the coefficients, is responsible for the improvements over GTA. The authors should clarify the exact role of the coefficient-dependent Fisher estimate and ideally ablate it against a version that uses a fixed Fisher estimate at, say, λ_i = 0.5.
minor comments (5)
  1. [Abstract / Section 3] The phrase "an unified" should be "a unified" in several places, and "an unified view" in Section 3 should also be corrected.
  2. [Figure 4 / Appendix B] The x-axis of the optimization trajectory figures is labeled "number of iterations" with values up to 60, but the text says the first 10 points are random initial evaluations and that 9 iterations suffice after that. Please clarify whether the axis counts total evaluations or BO iterations, and consistently label the random-initialization region.
  3. [Section 3 / Eq. (7)] The symbol f is used both for the merge function and for the black-box objective f_b; this is confusing. Please use distinct notation, for example g for the merge function and f for the objective.
  4. [Table 2] The significance asterisks are defined as "both DF-Merge (EI) and DF-Merge (UCB) significantly outperform the baseline," but the text says improvements are significant for "almost all baselines." Please state the significance test used and whether any multiple-comparison correction was applied, and report which individual comparisons are significant.
  5. [Section 5.3 / Figure 6] The color map in Figure 6 is not labeled with a colorbar; please add one so that the reported accuracy values can be read from the landscape plots.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: test metrics are held out; the unified formula is a mathematical construction, not a fitted prediction.

full rationale

I examined the derivation chain. The unified merge function in Eq. 7 is an explicit construction: setting C=I and lambda_i=1/M recovers Averaging, C=I gives General Task Arithmetic, and C=diag(F_hat) with lambda_i=1/M gives Fisher Merging. This is a mathematical identity, not a result whose evidence is its own assumption. DF-Merge (Eq. 8) is a new combination (Bayesian-optimization-selected lambda_i plus Fisher weights evaluated at scaled models); the coefficients are selected on held-out validation sets and Table 2 reports accuracy on held-out test sets. The claim of "near-optimal in a few iterations" is a statement about the optimization trajectory of the validation objective, not a disguised prediction of test performance. There are no load-bearing self-citations and no imported uniqueness theorem. The ablation study removes components and measures the drop, which is an independent check. The reader's concern about unequal validation-evaluation budget (60 BO evaluations vs ~11 grid-search points for TA/TIES) is a valid experimental-fairness issue, but it is not circularity: the test numbers are not forced by construction or by the fitted coefficients in a way that would make the comparison tautological. Therefore the circularity pass returns 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims rest on several assumptions and tuned hyperparameters. The coefficients lambda_i are fitted to validation sets, inheriting a tuning budget advantage over baselines. The diagonal Fisher approximation and 30-sample empirical Fisher are acknowledged as strong simplifications. No new physical or conceptual entities are introduced.

free parameters (4)
  • merging coefficients lambda_i = optimized per task via Bayesian optimization, values not tabulated (see Fig. 4 trajectory)
    The core tunable parameters; chosen to maximize validation accuracy.
  • BO initial points and iterations = 10 initial random points + 50 iterations
    Search budget; more budget than baseline grid search, threatening fair comparison.
  • UCB exploration constant beta = not specified in paper
    Controls exploration-exploitation; no value given, so reproducibility gap.
  • number of Fisher samples = 30 unlabeled validation samples
    Ad hoc; Fisher is estimated from 30 samples.
assumptions (4)
  • domain assumption Fine-tuned models from the same pretrained initialization are linearly mode-connected (low-loss basin)
    Motivates linear interpolation of task vectors; cited to prior work, not tested here.
  • domain assumption Diagonal Fisher information accurately approximates parameter importance for scaled models
    Used in Eq. 8; the paper acknowledges this as a strong assumption in Limitations.
  • domain assumption Empirical Fisher with predictive distribution (y~p_theta(y|x)) and 30 samples estimates the true Fisher
    Eq. 3 and Implementation Details; no evidence given for sample size adequacy.
  • domain assumption Validation set accuracy is a reliable proxy for test performance when maximizing the merge objective
    BO optimizes validation accuracy; the method assumes this transfers to test.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dynamic Fisher-weighted Model Merging via Bayesian Optimization." pith.science (2026). https://pith.science/paper/YR2TE2UD

@misc{pith2026250418992,
  author       = {Pith},
  title        = {Pith review of: Dynamic Fisher-weighted Model Merging via Bayesian Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YR2TE2UD}},
  note         = {Machine review of arXiv:2504.18992}
}
read the original abstract

The fine-tuning of pre-trained language models has resulted in the widespread availability of task-specific models. Model merging offers an efficient way to create multi-task models by combining these fine-tuned models at the parameter level, without the need for training data or joint training on multiple datasets. Existing merging approaches typically involve scaling the parameters model-wise or integrating parameter importance parameter-wise. Both approaches exhibit their own weaknesses, leading to a notable performance gap compared to multi-task fine-tuning. In this paper, we unify these seemingly distinct strategies into a more general merging framework, and introduce Dynamic Fisher-weighted Merging (DF-Merge). Specifically, candidate models are associated with a set of coefficients that linearly scale their fine-tuned parameters. Bayesian optimization is applied to dynamically adjust these coefficients, aiming to maximize overall performance on validation sets. Each iteration of this process integrates parameter importance based on the Fisher information conditioned by the coefficients. Experimental results show that DF-Merge outperforms strong baselines across models of different sizes and a variety of tasks. Our analysis shows that the effectiveness of DF-Merge arises from the unified view of merging and that near-optimal performance is achievable in a few iterations, even with minimal validation data.

Figures

Figures reproduced from arXiv: 2504.18992 by the authors.

Figure 1
Figure 1. Comparison of DF-Merge with primary ap￾proaches in model merging. left: leverages parameter importance evaluated at the fine-tuned models. middle: uniformly scales fine-tuned models to alleviate param￾eter interference. right: DF-Merge optimizes distinct scaling coefficients and incorporates parameter impor￾tance evaluated at the scaled models. achieving state-of-the-art performance across vari￾ous applications. The… view at source ↗
Figure 2
Figure 2. Accuracy (y-axis) of linear inter/extrapolation [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An illustration of DF-Merge. (1) The black-box function f(·) takes coefficients as inputs to (a) scale the task vectors, and (c) merges models (yellow star) after (b) accounting for the parameter importance using FI, where a contour depicts the local loss landscape of a specific task. (2) The validation set accuracy f(·) is used by Bayesian optimization to suggest the best guess on the coefficients for the next iter… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Bayesian optimization trajectory of DF-Merge (T5-base, UCB). Coefficients ( [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Average test set accuracy (%) of DF-Merge [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: A landscape of average validation accuracy [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Bayesian optimization trajectory of DF-Merge (T5-base, EI). [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Bayesian optimization trajectory of DF-Merge (T5-large, EI). [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Bayesian optimization trajectory of DF-Merge (T5-large, UCB). [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. AP-BMM: Approximating Capability-Cost Pareto Sets of LLMs via Asynchronous Prior-Guided Bayesian Model Merging

    cs.LG 2025-12 reject novelty 6.0 of 10

    The abstract claims asynchronous, GPU-efficient Bayesian model merging, while the full text presents a synchronous method with no wall-clock or asynchrony results.

Reference graph

Works this paper leans on

46 extracted references · 17 canonical work pages · cited by 1 Pith paper

  1. [1]

    Samuel Ainsworth, Jonathan Hayase, and Siddhartha Srinivasa. 2023. https://openreview.net/forum?id=CQsmMYmlP5T Git re-basin: Merging models modulo permutation symmetries . In The Eleventh International Conference on Learning Representations

    2023

  2. [2]

    Stephen Bach, Victor Sanh, Zheng Xin Yong, Albert Webson, Colin Raffel, Nihal V Nayak, Abheesht Sharma, Taewoon Kim, M Saiful Bari, Thibault F \'e vry, et al. 2022. Promptsource: An integrated development environment and repository for natural language prompts. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Sys...

    work page 2022

  3. [3]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gr...

    2020

  4. [4]

    Nico Daheim, Thomas M \"o llenhoff, Edoardo Ponti, Iryna Gurevych, and Mohammad Emtiyaz Khan. 2024. https://openreview.net/forum?id=D7KJmfEDQP Model merging by uncertainty-based gradient matching . In The Twelfth International Conference on Learning Representations

    work page 2024

  5. [5]

    MohammadReza Davari and Eugene Belilovsky. 2024. https://arxiv.org/abs/2312.06795 Model breadcrumbs: Scaling multi-task model merging with sparse masks . Preprint, arXiv:2312.06795

    arXiv 2024

  6. [6]

    Pala Tej Deep, Rishabh Bhardwaj, and Soujanya Poria. 2024. https://arxiv.org/abs/2406.11617 Della-merging: Reducing interference in model merging through magnitude-based sampling . Preprint, arXiv:2406.11617

    arXiv 2024

  7. [7]

    Yang Deng, Yuexiang Xie, Yaliang Li, Min Yang, Nan Du, Wei Fan, Kai Lei, and Ying Shen. 2019. Multi-task learning with multi-view attention for answer selection and knowledge base question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6318--6325

    work page 2019

  8. [8]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long a...

    doi:10.18653/v1/n19-1423 2019

Show all 46 references
  1. [9]

    Felix Draxler, Kambis Veschgini, Manfred Salmhofer, and Fred Hamprecht. 2018. Essentially no barriers in neural network energy landscape. In International conference on machine learning, pages 1309--1318. PMLR

    2018

  2. [10]

    Rahim Entezari, Hanie Sedghi, Olga Saukh, and Behnam Neyshabur. 2022. https://openreview.net/forum?id=dNigytemkL The role of permutation invariance in linear mode connectivity of neural networks . In International Conference on Learning Representations

    2022

  3. [11]

    Peter I Frazier. 2018. A tutorial on bayesian optimization. arXiv preprint arXiv:1807.02811

    2018 arXiv

  4. [12]

    Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry P Vetrov, and Andrew G Wilson. 2018. Loss surfaces, mode connectivity, and fast ensembling of dnns. Advances in neural information processing systems, 31

    2018

  5. [13]

    Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2023. https://openreview.net/forum?id=6t0Kwf8-jrj Editing models with task arithmetic . In The Eleventh International Conference on Learning Representations

    2023

  6. [14]

    Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro, and Pengxiang Cheng. 2023. https://openreview.net/forum?id=FCnohuR6AnM Dataless knowledge fusion by merging weights of language models . In The Eleventh International Conference on Learning Representations

    2023

  7. [15]

    Tushar Khot, Peter Clark, Michal Guerquin, Peter Jansen, and Ashish Sabharwal. 2020. Qasc: A dataset for question answering via sentence composition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 8082--8090

    2020

  8. [16]

    Deyuan Liu, Zecheng Wang, Bingning Wang, Weipeng Chen, Chunshan Li, Zhiying Tu, Dianhui Chu, Bo Li, and Dianbo Sui. 2024. Checkpoint merging via bayesian optimization in llm pretraining. arXiv preprint arXiv:2403.19390

    2024 arXiv

  9. [17]

    Ilya Loshchilov and Frank Hutter. 2019. https://openreview.net/forum?id=Bkg6RiCqY7 Decoupled weight decay regularization . In International Conference on Learning Representations

    2019

  10. [18]

    Michael S Matena and Colin A Raffel. 2022. Merging models with fisher-weighted averaging. Advances in Neural Information Processing Systems, 35:17703--17716

    2022

  11. [19]

    Bonan Min, Hayley Ross, Elior Sulem, Amir Pouran Ben Veyseh, Thien Huu Nguyen, Oscar Sainz, Eneko Agirre, Ilana Heintz, and Dan Roth. 2023. Recent advances in natural language processing via large pre-trained language models: A survey. ACM Computing Surveys, 56(2):1--40

    2023

  12. [20]

    Seyed Iman Mirzadeh, Mehrdad Farajtabar, Dilan Gorur, Razvan Pascanu, and Hassan Ghasemzadeh. 2021. Linear mode connectivity in multitask and continual learning. In 9th International Conference on Learning Representations, ICLR 2021

    2021

  13. [21]

    Behnam Neyshabur, Hanie Sedghi, and Chiyuan Zhang. 2020. What is being transferred in transfer learning? Advances in neural information processing systems, 33:512--523

    2020

  14. [22]

    Fernando Nogueira. 2014--. https://github.com/bayesian-optimization/BayesianOptimization Bayesian Optimization : Open source constrained global optimization tool for Python

    2014

  15. [23]

    Guillermo Ortiz-Jimenez, Alessandro Favero, and Pascal Frossard. 2023. https://openreview.net/forum?id=0A9f2jZDGW Task arithmetic in the tangent space: Improved editing of pre-trained models . In Thirty-seventh Conference on Neural Information Processing Systems

    2023

  16. [24]

    Mohammad Mahdi Abdollah Pour, Ali Pesaranghader, Eldan Cohen, and Scott Sanner. 2024. Gaussian process optimization for adaptable multi-objective text generation using linearly-weighted language models. In Findings of the Association for Computational Linguistics: NAACL 2024, ...

    2024

  17. [25]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1--67

    2020

  18. [26]

    Sebastian Ruder and Barbara Plank. 2017. Learning to select data for transfer learning with bayesian optimization. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 372--382

    2017

  19. [27]

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99--106

    2021

  20. [28]

    Rishi Sharma, James Allen, Omid Bakhshandeh, and Nasrin Mostafazadeh. 2018. https://doi.org/10.18653/v1/P18-2119 Tackling the story ending biases in the story cloze test . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Sho...

    2018 doi

  21. [29]

    Anders S gaard and Yoav Goldberg. 2016. https://doi.org/10.18653/v1/P16-2038 Deep multi-task learning with low level tasks supervised at lower layers . In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 23...

    2016 doi

  22. [30]

    Niranjan Srinivas, Andreas Krause, Sham Kakade, and Matthias Seeger. 2010. Gaussian process optimization in the bandit setting: no regret and experimental design. In Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML'10, page...

    2010

  23. [31]

    George Stoica, Daniel Bolya, Jakob Brandt Bjorner, Pratik Ramesh, Taylor Hearn, and Judy Hoffman. 2024. https://openreview.net/forum?id=LEYUkvdUhq Zipit! merging models from different tasks without training . In The Twelfth International Conference on Learning Representations

    2024

  24. [32]

    Oyvind Tafjord, Matt Gardner, Kevin Lin, and Peter Clark. 2019. https://doi.org/10.18653/v1/D19-1608 Q ua RT z: An open-domain dataset of qualitative relationship questions . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th ...

    2019 doi

  25. [33]

    Derek Tam, Mohit Bansal, and Colin Raffel. 2024. Merging by matching models in task parameter subspaces. Transactions on Machine Learning Research

    2024

  26. [34]

    Anke Tang, Li Shen, Yong Luo, Yibing Zhan, Han Hu, Bo Du, Yixin Chen, and Dacheng Tao. 2024. https://openreview.net/forum?id=iynRvVVAmH Parameter-efficient multi-task model fusion with partial linearization . In The Twelfth International Conference on Learning Representations

    2024

  27. [35]

    Christopher KI Williams and Carl Edward Rasmussen. 2006. Gaussian processes for machine learning, volume 2. MIT press Cambridge, MA

    2006

  28. [36]

    T Wolf. 2019. Huggingface's transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771

    2019 arXiv

  29. [37]

    Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, et al. 2022. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasin...

    2022

  30. [38]

    Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. 2024. Ties-merging: Resolving interference when merging models. Advances in Neural Information Processing Systems, 36

    2024

  31. [39]

    Enneng Yang, Zhenyi Wang, Li Shen, Shiwei Liu, Guibing Guo, Xingwei Wang, and Dacheng Tao. 2024. https://openreview.net/forum?id=nZP6NgD3QY Adamerging: Adaptive model merging for multi-task learning . In The Twelfth International Conference on Learning Representations

    2024

  32. [40]

    Yi Yang, Wen-tau Yih, and Christopher Meek. 2015. https://doi.org/10.18653/v1/D15-1237 W iki QA : A challenge dataset for open-domain question answering . In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2013--2018, Lisbon, Portu...

    2015 doi

  33. [41]

    Dani Yogatama, Lingpeng Kong, and Noah A Smith. 2015. Bayesian optimization of text representations. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2100--2105

    2015

  34. [42]

    Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2024. Language models are super mario: Absorbing abilities from homologous models as a free lunch. In Forty-first International Conference on Machine Learning

    2024

  35. [43]

    Yuan Zhang, Jason Baldridge, and Luheng He. 2019. https://doi.org/10.18653/v1/N19-1131 PAWS : Paraphrase adversaries from word scrambling . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language Tec...

    2019 doi

  36. [44]

    Zhanpeng Zhou, Zijun Chen, Yilan Chen, Bo Zhang, and Junchi Yan. 2024. https://openreview.net/forum?id=qg6AlnpEQH On the emergence of cross-task linearity in pretraining-finetuning paradigm . In Forty-first International Conference on Machine Learning

    2024

  37. [45]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  38. [46]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.