Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

If You Can't Use Them, Recycle Them: Optimizing Merging at Scale Mitigates Performance Tradeoffs

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that discarded 100B-scale checkpoints, which individually trade off capabilities, can be recycled by optimizing a linear merge into a Pareto-optimal model without hurting held-out performance.

desk verdict A useful empirical study of evolutionary merging at 104B scale, but the headline margins rest on single stochastic runs and need variance estimates before the numbers are trusted. read the letter →

arxiv 2412.04144 v3 pith:V7SFAZSL submitted 2024-12-05 cs.CL cs.AI

Muhammad Khalifa , Yi-Chern Tan , Arash Ahmadian , Tom Hosking , Honglak Lee , Lu Wang , Ahmet Üstün , Tom Sherborne
show 1 more author
Matthias Gallé
This is my paper · ORCID
classification cs.CLcs.AI
keywords modelmergingsoupsPareto-optimaltasktradeoffscheckpointrecyclingevolutionarysearchCMA-ESlargelanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Model development produces many intermediate checkpoints that trade off one capability against another, and the usual response is to keep the best and discard the rest. This paper asks whether those leftovers can be recycled: it defines the merge as a weighted average of up to sixteen 104B checkpoints and uses an evolutionary search to tune the weights. On two- and three-task tradeoff pairs (code vs. instruction-following, code vs. multistep reasoning, knowledge vs. instruction-following, and the triple code/math/instruction-following), the optimized merge beats uniform averaging, merge-best, and the best single model on the tasks being balanced, and it does not degrade held-out benchmarks. The paper reports that good merges spread weight across almost all checkpoints, including ones that score worst on the target tasks, and that the whole search costs roughly 10% of one training stage's compute.

What carries the argument

The load-bearing object is linear weight interpolation: a convex combination of the raw parameters of the 16 checkpoints, $\theta_{\mathrm{mrg}} = \sum_{i=1}^{16} \alpha_i \theta_i$ with $\alpha_i \ge 0$ and $\sum_i \alpha_i = 1$. This is the same operation as a "model soup," but here the weights are not fixed by hand: they are searched by CMA-ES, an evolutionary strategy that samples weight vectors from a multivariate normal distribution, evaluates the resulting merged model on the held-in tasks, and adapts its sampling distribution toward higher macro-average fitness. The machinery does two jobs at once: it converts a discrete set of discarded training runs into a continuous family of candidate models, and it turns Pareto-optimality into a single scalar objective (the unweighted average of task scores) that a gradient-free optimizer can drive.

What would settle it

Take the original 16 checkpoints, apply to each an independent random permutation of its hidden units (together with the corresponding weight-matrix permutations so each model still behaves identically), and rerun the same search on the permuted pool; if the optimized merge no longer beats uniform soup on the held-in average, the benefit depends on raw-parameter alignment rather than on the checkpoints' capabilities.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that linear merging is a viable training-free way to reduce capability tradeoffs at the 100B scale. Given the 16 checkpoints, the optimization searches the simplex of non-negative weights summing to one to maximize the unweighted average of held-in task scores. The resulting models lie on the Pareto frontier over the evaluated candidates and outperform uniform soup and task-best merging on held-in averages (e.g., 68.0 vs 66.0 on MBPP-IFEval, and 72.1 vs 70.7 on the three-task combination), while maintaining comparable or better held-out scores. The paper also shows that the optimal solutions are dense: nearly all checkpoints receive nonzero weight, and a checkpoint's standalone performance does not predict its contribution to a good merge.

Load-bearing premise

The method assumes the raw parameters of the 16 checkpoints can be averaged meaningfully, which requires a shared architecture and aligned parameter spaces despite different training stages, objectives, and data mixtures.

Editorial extensions

If this is right

  • Discarded intermediate checkpoints can be treated as a reusable resource: a frontier-model team can run the search recipe on whatever checkpoints it has and get a better balanced model without additional training.
  • Because good merges are dense, pruning the checkpoint pool by individual performance would remove inputs that contribute to the optimum; the pool should be kept broad, not filtered by score.
  • The method gives an upper-bound baseline for more complex merging techniques at 100B scale: if simple linear merging with searched weights is already Pareto-optimal over candidates, more elaborate methods must beat that to justify their extra cost.
  • The fact that held-out tasks do not degrade suggests the search is not overfitting to the exact evaluation prompts; the merged model generalises the balanced capability to other evaluations in the same family.
  • At roughly 10% of one SFT+PO stage's compute, this is a cheap way to probe the tradeoff frontier before committing to another training run.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next experiment is transferring a learned weight vector: use the optimized merge as the initialization for a short continued SFT/PO stage, then test whether the Pareto benefit survives further training, which the paper does not address.
  • The dense-weight finding hints that each checkpoint contributes a distinct axis in parameter space and that the optimal merge exploits the full span of these directions; this could be tested by measuring how well the merge direction aligns with task-performance gradients.
  • The recipe could be used outside checkpoint recycling, for example as a fast way to choose a final model during development by exploring the Pareto frontier across many candidate runs before selecting which checkpoint to ship.
  • The strong dependence on raw parameter alignment means the approach will work only within one model family; extending it to heterogeneous checkpoints with different tokenizers or architectures would require a different merge space, not just a better search.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies whether linearly merging 16 large (104B) checkpoints harvested from different training runs can reduce performance tradeoffs across capabilities such as code generation, instruction following, math, and multi-step reasoning. The authors formulate the merge as a convex combination of model parameters, θ_mrg = Σ α_i θ_i, and use CMA-ES to optimize the weights α_i against a macro-average fitness over held-in tasks. They report that search-optimized merges outperform uniform soup and merge-best baselines on held-in tasks in two-task and three-task settings, maintain or improve performance on held-out tasks, and that the best merges use almost all checkpoints with nonzero weights. The paper also provides a compute-cost estimate suggesting that the merge search costs about 10% of a single SFT+PO training stage.

Significance. If the reported results are robust, this is a useful and timely contribution: it scales model merging to ~100B parameters, moves beyond the expert-merging setup to heterogeneous generalist checkpoints, and provides both a practical training-free recipe for recycling suboptimal models and an interesting qualitative finding that low-performing checkpoints can contribute to good merges. The inclusion of held-out tasks and the analysis of the learned weight distributions are particular strengths. The main caveat is that the evidence rests on single stochastic optimization runs with no variance estimates, and the search protocol is internally inconsistent in ways that affect both reproducibility and the computational-cost claim.

major comments (4)
  1. [§4, Algorithm 1, App. D] The search protocol is internally inconsistent. Section 4 states that the population size is set to 4+3lnN (≈12 for N=16), and Algorithm 1 samples λ offspring per iteration and evaluates each; but the text then says "At each iteration, CMA-ES proposes a single weighting vector" and the compute-cost calculation in App. D uses only 50 evaluations per task (50 × MBPP cost + 50 × IFEval cost). If the standard CMA-ES in Algorithm 1 is used, the true number of fitness evaluations is roughly 50×λ, which would increase the reported computational cost by an order of magnitude and undermine the "10% of training compute" claim. If only one candidate per iteration is evaluated, the algorithm is not the CMA-ES described in Algorithm 1. Please clarify the exact evaluation protocol and align Section 4, Algorithm 1, Fig. 9, and App. D.
  2. [§5.1, Tables 1 and 3] Each reported search-optimized merge is the result of a single CMA-ES run with no repeated seeds, no variance estimates, and no explicit statement of whether the reported numbers are the best-ever solution, the final mean, or a post-hoc selected trajectory. The headline margins are modest (e.g., 68.0 vs. 66.0 for MBPP-IFEval in Table 1; 72.1 vs. 70.7 for MBPP-IFEval-GSM8K in Table 3), and the fitness is evaluated on one greedy decoding pass. Without repeated runs, the claim that search-optimized merging "always yields" a Pareto-frontier model and reliably outperforms baselines is not supported. Please provide multiple seeds (or at least a clear selection rule and a sensitivity analysis), report mean/std or confidence intervals, and if possible compare the optimization signal to random search or a small number of independent restarts.
  3. [§5.3, Figs. 7 and 8] The checkpoint-count ablation (N=2,4,8,16) and the "most models contribute" analysis are based on what appear to be single runs per configuration, with no repeated seeds or variance estimates. The centroids in Figs. 7-8 may therefore shift substantially under different random seeds, and the qualitative conclusion that dense merges are optimal is drawn from one or a few top-5 solution sets. Additionally, the captions of Fig. 7 and Fig. 8 both say the experiments optimize MBPP-MUSR, while the text indicates that Fig. 8 should show MBPP-IFEval; please correct this and clarify which configuration each figure reports.
  4. [§3.2] The method assumes that a direct weighted average of 16 checkpoints from different SFT/PO runs, with different objectives, data mixtures, and possibly different base models, is a well-defined operation in parameter space. This requires that the checkpoints are commensurable and that interpolation is meaningful, but the paper provides no evidence on this point (e.g., whether the models share the same architecture and initialization, or whether permutation symmetries were addressed). If the checkpoints are not interpolatable, the optimized weights have no reliable effect. Please add a discussion or a simple control (e.g., showing that random weighted averages of the same checkpoints produce sensible models, and that the optimized solutions are not artifacts of ill-conditioned interpolation).
minor comments (5)
  1. [§4 and Fig. 2 caption] The text and caption say "Models 1-8 are the result of supervised finetuning runs, while 8-16 from preference optimization," but the enumeration in Table 2 shows SFT models 1-8 and PO models 9-16; the second range should be 9-16.
  2. [Abstract and §5.1] The phrase "Pareto-optimal" is used without qualification. Since the search explores only a finite set of CMA-ES proposals, the claim should be stated as Pareto-optimality among the evaluated candidates/checkpoints, not as a global property of the entire weight simplex.
  3. [Footnote 3] The paper states that the 2K MMLU-Pro subset correlates highly with the full dataset but reports no correlation coefficient or error estimate; please provide the quantitative evidence or a reference for this claim.
  4. [Fig. 9] If the CMA-ES population size is indeed λ≈12 as stated in Section 4, the fitness-vs-iteration plot should show multiple points per iteration; the current plot appears to show one point per iteration, which is inconsistent with the described algorithm and should be aligned with the clarified search protocol.
  5. [App. C] The Spearman correlations in Fig. 4 are computed over only 16 checkpoints, and no significance levels or confidence intervals are reported; adding these would strengthen the tradeoff analysis.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: held-in gains are optimized objective values, and held-out tasks provide an external check.

full rationale

The paper's central claim is an empirical optimization result: CMA-ES maximizes the fitness R(θ') = (1/T) Σ Pt(θ') (Section 3.2) over non-negative simplex weights for θ_mrg = Σ α_i θ_i. The reported held-in gains in Tables 1 and 3 are the optimized values of that same fitness objective, so they are optimization outcomes, not out-of-sample predictions; the paper does not disguise them as predictive findings. The held-out tasks (MT-Bench and LBPP) are not included in the fitness function and are reported separately as an external check, and the paper explicitly frames them as validation. The Pareto-optimality claim rests on the mathematically valid footnote that a model achieving the best macro-average cannot be dominated by another model, which is a consequence of the chosen scalarization rather than a hidden fit or circular definition. No parameter is fitted to a subset and then reported as a prediction of a closely related quantity, no uniqueness theorem is imported from the authors' prior work, and the only overlapping-author citation (Matton et al. for the LBPP benchmark) is a dataset reference rather than a load-bearing theorem. Concerns about a single CMA-ES trajectory or missing repeated-seed variance estimates are robustness issues, not circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new entities or physical mechanisms. Its central claim rests on the ability to linearly interpolate heterogeneous checkpoints, a scalar fitness definition, the adequacy of a small held-out set, and the convergence of a short CMA-ES run. The merge weights themselves are the primary free parameters, fitted to held-in task evaluations.

free parameters (4)
  • Merge weights α_i (i=1..16) = not reported in full; heatmaps of top-5 solutions in Fig. 6
    The 16 weights are tuned by CMA-ES to maximize macro-average fitness on held-in tasks. These are the fitted parameters whose values drive the claimed result.
  • CMA-ES initial step size σ0 = 1.0
    Chosen manually in Section 4; affects search behavior and may influence the quality of found merges.
  • CMA-ES population size and number of iterations = 4 + 3 ln N ≈ 12; 50 iterations
    Defaults from Optuna and prior work; the paper uses 50 evaluations per task in cost estimates.
  • MMLUPro evaluation subset = 2,000 examples
    The paper uses a 2K subset and asserts it correlates highly with the full benchmark, but does not show the correlation evidence (Section 4, footnote 3).
assumptions (5)
  • domain assumption Linear parameter interpolation between checkpoints produces a valid merged model.
    Section 3.2 restricts to weight interpolation, assuming model soups is a strong baseline and meaningful for heterogeneous 104B checkpoints.
  • domain assumption Unweighted macro-average of task scores is a suitable measure of tradeoff severity.
    Section 3.2 defines R(θ') as the macro-average; single-objective optimization requires this scalarization.
  • domain assumption The two held-out tasks (MT-Bench, LBPP) are sufficient to detect overfitting to held-in tasks.
    Section 5.1 checks only these two tasks to conclude no performance compromise.
  • domain assumption CMA-ES with 50 evaluations finds near-optimal merge weights.
    Section 4 sets 50 iterations; no convergence analysis or repeated runs are provided.
  • domain assumption The 2K MMLUPro subset is representative of the full benchmark.
    Section 4 states correlation with the full dataset but gives no quantitative backing.

how reviews work

0 comments
Cite this review

Pith. "Pith review of If You Can't Use Them, Recycle Them: Optimizing Merging at Scale Mitigates Performance Tradeoffs." pith.science (2026). https://pith.science/paper/V7SFAZSL

@misc{pith2026241204144,
  author       = {Pith},
  title        = {Pith review of: If You Can't Use Them, Recycle Them: Optimizing Merging at Scale Mitigates Performance Tradeoffs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V7SFAZSL}},
  note         = {Machine review of arXiv:2412.04144}
}
read the original abstract

Model merging has shown great promise at combining expert models, but the benefit of merging is unclear when merging "generalist" models trained on many tasks. We explore merging in the context of large (~100B) models, by recycling checkpoints that exhibit tradeoffs among different tasks. Such checkpoints are often created in the process of developing a frontier model, and the suboptimal ones are usually discarded. Given a pool of model checkpoints obtained from different training runs (e.g., different stages, objectives, hyperparameters, and data mixtures), which naturally show tradeoffs across different language capabilities (e.g., instruction following vs. code generation), we investigate whether merging can recycle such suboptimal models into a Pareto-optimal one. Our optimization algorithm tunes the weight of each checkpoint in a linear combination, resulting in such an optimal model that outperforms both individual models and merge-based baselines. Further analysis shows that good merges tend to include almost all checkpoints with non-zero weights, indicating that even seemingly bad initial checkpoints can contribute to good final merges.

Figures

Figures reproduced from arXiv: 2412.04144 by the authors.

Figure 1
Figure 1. An overview of our setup. Given models obtained from different LLM training runs, we optimize linear merging weightings (α1, α2, α3) via iterative search to obtain a model with minimal task tradeoffs. Each represents a single model, with a to designate its performance on the two tasks. The purple color indicates a Pareto-optimal model, achieving a good balance between the two tasks without being dominated by other m… view at source ↗
Figure 2
Figure 2. Performance of individual models over the seven tasks covering different capabilities. Models 1-8 are the result of supervised finetuning runs, while 8-16 from preference optimization. Held-out tasks (MT-Bench and LBPP) are used to evaluate the resulting merges to make sure the merge optimization process does not overfit to the held-in tasks that we aim to minimize tradeoffs over. MT-Bench rating is scaled by a fact… view at source ↗
Figure 3
Figure 3. Performance tradeoffs with different merging approaches over different pairwise combinations. Shaded areas represent 95% confidence interval of the best-fit line computed over individual checkpoint scores (shown in green). Held-In Held-Out Avg. All Tasks MBPP IFEval Avg. MT-Bench LBPP Highest fitness model 63.0 65.7 60.1 7.42 30.4 41.6 Best on MBPP 64.0 56.5 60.3 7.68 32.9 40.3 Best on IFEval 56.8 72.0 64.4 8.50 28.… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Spearman’s rank correlation between task pairs. It is easy to see how some tasks exhibit strong performance tradeoffs, such as MBPP-IFEval and MMLU-Pro/MUSR. following, and math reasoning, by using MBPP-IFEval￾GSM8K as held-in tasks. We target this combination since IF…
Figure 6
Figure 6. Figure 6: Best solutions found via CMA-ES search when optimizing tradeoffs over the pairs MBPP-MUSR (left) MBPP-IFEval (mid) and MBPP-IFEval-GSM8K. We order the weightings over the x-axis based on the fitness of the individual model they correspond to. We observe that top-soluti…
Figure 7
Figure 7. Figure 7: Merges found via CMA-ES when optimizing MBPP￾MUSR tradeoffs over 2, 4, 8, and 16 checkpoints. We also show the centroid of each set of experiments (in large markers). Optimizing over more checkpoints (8 and 16) tends to yield less tradeoffs compared to fewer checkpoint…
Figure 8
Figure 8. Figure 8: Merges found via CMA-ES when optimizing MBPP￾MUSR tradeoffs over 2, 4, 8, and 16 checkpoints. We also show the centroid of each set of experiments. We find that optimizing over more checkpoints (8 and 16) outperforms optimization over fewer checkpoints (2,4), showing h…
Figure 9
Figure 9. Figure 9: Fitness vs. CMA-ES iterations when optimizing tradeoffs over task pairs (see Sect. 5.1). CMA-ES explores the search space to find merge weightings with high fitness or low task tradeoffs. MBPP: 2 × 100 × 109 × 500 = 1.01 × 1014 , IFEval: 2 × 100 × 109 × 541 = 1.09 × 10…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. LaMDAgent: An Autonomous Framework for Post-Training Pipeline Optimization via LLM Agents

    cs.CL 2025-05 conditional novelty 5.0 of 10

    An LLM-agent-driven search over fine-tuning and model-merging steps discovers post-training pipelines that beat fixed baselines by up to 9 points on tool use and 3.7 points on math, with caveats about held-out evaluation.

Reference graph

Works this paper leans on

50 extracted references · 12 canonical work pages · cited by 1 Pith paper

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  3. [3]

    Optuna: A next-generation hyperparameter optimization framework

    Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp.\ 2623--2631, 2019

    work page 2019

  4. [4]

    Evolutionary optimization of model merging recipes

    Akiba, T., Shing, M., Tang, Y., Sun, Q., and Ha, D. Evolutionary optimization of model merging recipes. arXiv preprint arXiv:2403.13187, 2024

    arXiv 2024

  5. [5]

    and Ludwig, S

    Alibrahim, H. and Ludwig, S. A. Hyperparameter optimization: Comparing genetic algorithm against grid search and bayesian optimization. In 2021 IEEE Congress on Evolutionary Computation (CEC), pp.\ 1551--1559. IEEE, 2021

    work page 2021

  6. [6]

    Qwen technical report

    Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023

    arXiv 2023

  7. [7]

    Training a helpful and harmless assistant with reinforcement learning from human feedback

    Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022

    arXiv 2022

  8. [8]

    and Bengio, Y

    Bergstra, J. and Bengio, Y. Random search for hyper-parameter optimization. Journal of machine learning research, 13 0 (2), 2012

    work page 2012

Show all 50 references
  1. [9]

    Where to start? analyzing the potential value of intermediate models

    Choshen, L., Venezian, E., Don-Yehia, S., Slonim, N., and Katz, Y. Where to start? analyzing the potential value of intermediate models. arXiv preprint arXiv:2211.00107, 2022

    2022 arXiv

  2. [10]

    W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al

    Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25 0 (70): 0 1--53, 2024

    2024

  3. [11]

    Training verifiers to solve math word problems

    Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    2021 arXiv

  4. [12]

    M., Gurevych, I., and Khan, M

    Daheim, N., M \"o llenhoff, T., Ponti, E. M., Gurevych, I., and Khan, M. E. Model merging by uncertainty-based gradient matching. arXiv preprint arXiv:2310.12808, 2023

    2023 arXiv

  5. [13]

    The art of balancing: Revolutionizing mixture of experts for maintaining world knowledge in language model alignment

    Dou, S., Zhou, E., Liu, Y., Gao, S., Zhao, J., Shen, W., Zhou, Y., Xi, Z., Wang, X., Fan, X., et al. The art of balancing: Revolutionizing mixture of experts for maintaining world knowledge in language model alignment. arXiv preprint arXiv:2312.09979, 2023

    2023 arXiv

  6. [14]

    The llama 3 herd of models

    Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  7. [15]

    Specializing smaller language models towards multi-step reasoning

    Fu, Y., Peng, H., Ou, L., Sabharwal, A., and Khot, T. Specializing smaller language models towards multi-step reasoning. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), Proceedings of the 40th International Conference on Machine Lear...

    2023

  8. [16]

    Knowledge is a region in weight space for fine-tuned language models

    Gueta, A., Venezian, E., Raffel, C., Slonim, N., Katz, Y., and Choshen, L. Knowledge is a region in weight space for fine-tuned language models. In Bouamor, H., Pino, J., and Bali, K. (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp.\ 1350--137...

    2023 doi

  9. [17]

    Hammoud, H. A. A. K., Michieli, U., Pizzati, F., Torr, P., Bibi, A., Ghanem, B., and Ozay, M. Model merging and safety alignment: One bad model spoils the bunch. arXiv preprint arXiv:2406.14563, 2024

    2024 arXiv

  10. [18]

    and Ostermeier, A

    Hansen, N. and Ostermeier, A. Completely derandomized self-adaptation in evolution strategies. Evolutionary computation, 9 0 (2): 0 159--195, 2001

    2001

  11. [19]

    T., Wortsman, M., Gururangan, S., Schmidt, L., Hajishirzi, H., and Farhadi, A

    Ilharco, G., Ribeiro, M. T., Wortsman, M., Gururangan, S., Schmidt, L., Hajishirzi, H., and Farhadi, A. Editing models with task arithmetic. arXiv preprint arXiv:2212.04089, 2022 a

    2022 arXiv

  12. [20]

    Y., Song, S., Hajishirzi, H., Kornblith, S., Farhadi, A., and Schmidt, L

    Ilharco, G., Wortsman, M., Gadre, S. Y., Song, S., Hajishirzi, H., Kornblith, S., Farhadi, A., and Schmidt, L. Patching open-vocabulary models by interpolating weights. Advances in Neural Information Processing Systems, 35: 0 29262--29277, 2022 b

    2022

  13. [21]

    T., Wortsman, M., Schmidt, L., Hajishirzi, H., and Farhadi, A

    Ilharco, G., Ribeiro, M. T., Wortsman, M., Schmidt, L., Hajishirzi, H., and Farhadi, A. Editing models with task arithmetic. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.net, 2023. URL https://openr...

    2023

  14. [22]

    Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D

    Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. d. l., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023

    2023 arXiv

  15. [23]

    Dataless knowledge fusion by merging weights of language models

    Jin, X., Ren, X., Preotiuc-Pietro, D., and Cheng, P. Dataless knowledge fusion by merging weights of language models. arXiv preprint arXiv:2212.09849, 2022

    2022 arXiv

  16. [24]

    B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D

    Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020

    2001 arXiv

  17. [25]

    A., and Zettlemoyer, L

    Li, M., Gururangan, S., Dettmers, T., Lewis, M., Althoff, T., Smith, N. A., and Zettlemoyer, L. Branch-train-merge: Embarrassingly parallel training of expert language models. arXiv preprint arXiv:2208.03306, 2022

    2022 arXiv

  18. [26]

    Pareto multi-task learning

    Lin, X., Zhen, H.-L., Li, Z., Zhang, Q.-F., and Kwong, S. Pareto multi-task learning. Advances in neural information processing systems, 32, 2019

    2019

  19. [28]

    Matena, M. S. and Raffel, C. A. Merging models with fisher-weighted averaging. Advances in Neural Information Processing Systems, 35: 0 17703--17716, 2022

    2022

  20. [29]

    On leakage of code generation evaluation datasets

    Matton, A., Sherborne, T., Aumiller, D., Tommasone, E., Alizadeh, M., He, J., Ma, R., Voisin, M., Gilsenan-McMahon, E., and Gall \'e , M. On leakage of code generation evaluation datasets. arXiv preprint arXiv:2407.07565, 2024

    2024 arXiv

  21. [30]

    H., Sherborne, T., Strubell, E., Dodge, J., and Dasigi, P

    Na, C., Magnusson, I., Jha, A. H., Sherborne, T., Strubell, E., Dodge, J., and Dasigi, P. Scalable data ablation approximations for language models through modular training and merging. In The 2024 Conference on Empirical Methods in Natural Language Processing, 2024

    2024

  22. [31]

    Training language models to follow instructions with human feedback

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 0 27730--27744, 2022

    2022

  23. [32]

    Rewarded soups: towards pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards

    Rame, A., Couairon, G., Dancette, C., Gaya, J.-B., Shukor, M., Soulier, L., and Cord, M. Rewarded soups: towards pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards. Advances in Neural Information Processing Systems, 36, 2024

    2024

  24. [33]

    G., Girgin, S., Douillard, A., and Bachem, O

    Ram \'e , A., Ferret, J., Vieillard, N., Dadashi, R., Hussenot, L., Cedoz, P.-L., Sessa, P. G., Girgin, S., Douillard, A., and Bachem, O. Warp: On the benefits of weight averaged rewarded policies. arXiv preprint arXiv:2406.16768, 2024

    2024 arXiv

  25. [34]

    Snoek, J., Larochelle, H., and Adams, R. P. Practical bayesian optimization of machine learning algorithms. Advances in neural information processing systems, 25, 2012

    2012

  26. [35]

    Musr: Testing the limits of chain-of-thought with multistep soft reasoning

    Sprague, Z., Ye, X., Bostrom, K., Chaudhuri, S., and Durrett, G. Musr: Testing the limits of chain-of-thought with multistep soft reasoning. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net, 2024. ...

    2024

  27. [36]

    M., Hauth, A., Millican, K., et al

    Team, G., Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  28. [37]

    S., Love, J., et al

    Team, G., Mesnard, T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., Sifre, L., Rivi \`e re, M., Kale, M. S., Love, J., et al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295, 2024

    2024 arXiv

  29. [38]

    Weight averaging for neural networks and local resampling schemes

    Utans, J. Weight averaging for neural networks and local resampling schemes. In Proc. AAAI-96 Workshop on Integrating Multiple Learned Models. AAAI Press, pp.\ 133--138. Citeseer, 1996

    1996

  30. [39]

    R., Chowdhury, S

    Vijjini, A. R., Chowdhury, S. B. R., and Chaturvedi, S. Exploring safety-utility trade-offs in personalized language models. arXiv preprint arXiv:2406.11107, 2024

    2024 arXiv

  31. [40]

    Wang, Y., Wang, X., Beutel, A., Prost, F., Chen, J., and Chi, E. H. Understanding and improving fairness-accuracy trade-offs in multi-task learning. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp.\ 1748--1757, 2021

    2021

  32. [41]

    Mmlu-pro: A more robust and challenging multi-task language understanding benchmark

    Wang, Y., Ma, X., Zhang, G., Ni, Y., Chandra, A., Guo, S., Ren, W., Arulraj, A., He, X., Jiang, Z., et al. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. arXiv preprint arXiv:2406.01574, 2024

    2024 arXiv

  33. [42]

    Y., Roelofs, R., Gontijo-Lopes, R., Morcos, A

    Wortsman, M., Ilharco, G., Gadre, S. Y., Roelofs, R., Gontijo-Lopes, R., Morcos, A. S., Namkoong, H., Farhadi, A., Carmon, Y., Kornblith, S., et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In Internation...

    2022

  34. [43]

    A., and Bansal, M

    Yadav, P., Tam, D., Choshen, L., Raffel, C. A., and Bansal, M. Ties-merging: Resolving interference when merging models. Advances in Neural Information Processing Systems, 36, 2024 a

    2024

  35. [44]

    What matters for model merging at scale? arXiv preprint arXiv:2410.03617, 2024 b

    Yadav, P., Vu, T., Lai, J., Chronopoulou, A., Faruqui, M., Bansal, M., and Munkhdalai, T. What matters for model merging at scale? arXiv preprint arXiv:2410.03617, 2024 b

    2024 arXiv

  36. [45]

    R., Rose, D

    Young, S. R., Rose, D. C., Karnowski, T. P., Lim, S.-H., and Patton, R. M. Optimizing deep learning hyper-parameters through an evolutionary algorithm. In Proceedings of the workshop on machine learning in high-performance computing environments, pp.\ 1--5, 2015

    2015

  37. [46]

    Language models are super mario: Absorbing abilities from homologous models as a free lunch

    Yu, L., Yu, B., Yu, H., Huang, F., and Li, Y. Language models are super mario: Absorbing abilities from homologous models as a free lunch. In Forty-first International Conference on Machine Learning, 2024

    2024

  38. [47]

    Judging llm-as-a-judge with mt-bench and chatbot arena

    Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E., et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36: 0 46595--46623, 2023

    2023

  39. [48]

    Instruction-following evaluation for large language models

    Zhou, J., Lu, T., Mishra, S., Brahma, S., Basu, S., Luan, Y., Zhou, D., and Hou, L. Instruction-following evaluation for large language models. arXiv preprint arXiv:2311.07911, 2023

    2023 arXiv

  40. [49]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  41. [50]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  42. [51]

    Knowledge is a Region in Weight Space for Fine-tuned Language Models

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

    2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.