Pith. sign in

REVIEW 4 major objections 4 minor 45 references

NAN: A Training-Free Solution to Coefficient Estimation in Model Merging

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Optimal merging weights track task-specific information, estimated by inverse parameter norm without training.

desk verdict A simple training-free norm heuristic for merging coefficients, wrapped in a derivation that doesn't actually produce the formula. read the letter →

arxiv 2505.16148 v1 pith:CARIZICC submitted 2025-05-22 cs.LG cs.CL

Chongjie Si , Kangtao Lv , Jingjing Jiang , Yadao Wang , Yongwei Wang , Xiaokang Yang , Wenbo Su , Bo Zheng
show 1 more author
Wei Shen
This is my paper
classification cs.LGcs.CL
keywords modelmergingtraining-freetaskvectorcoefficientestimationFrobeniusnormleastsquaresmulti-tasklearningparametervariance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that when merging independently fine-tuned models, the best mixing coefficient for each model is proportional to how much task-specific information that model carries, and that this quantity can be read off the model weights alone. It derives from a least-squares view of merging that the ideal weights are sample-size-weighted, then connects sample size to the inverse variance of the trained parameters. That motivates a training-free plug-in, NAN, which sets each model's coefficient to the inverse of its Frobenius norm, normalized across models. If the derivation and proxy hold, NAN would give a zero-cost way to improve any task-vector merging method, including in settings where training data and sample sizes are unavailable.

What carries the argument

The argument is carried by a chain of substitutions. First, the closed-form least-squares solution for two tasks writes the merged model as a weighted combination $W^* = \Omega_1 W_1^* + \Omega_2 W_2^*$, where the matrix weights are relative information contents $(A_1+A_2)^{-1} A_i$. Second, normalized features make $A_i \approx n_i I$, turning the weights into sample-size fractions. Third, the empirical law $n \propto 1/\mathrm{Var}(W)$ lets the paper substitute variance for unknown sample size, and zero-centered weights make $\mathrm{Var}(W) \propto \|W\|_F^2$. The final step replaces the squared norm by the unsquared Frobenius norm, yielding the inverse-norm NAN coefficients; a global factor $m/2$ rescales the merged weights when many models are combined.

What would settle it

Take a set of fine-tuned checkpoints with publicly known training sample sizes and check whether the inverse Frobenius norm ranking matches the sample-size ranking across tasks; if a model trained on considerably more data has a larger norm and therefore gets a smaller NAN weight, the proxy's core relation fails for that case. A second, cheaper check is the ablation of $\|W\|_F$ versus $\|W\|_F^2$: if the unsquared choice is load-bearing, swapping it should visibly change merged accuracy.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that the optimal merging coefficients should scale with the amount of task-specific information encoded in each model, and that the inverse parameter norm is a faithful, computable proxy for that information. Starting from two least-squares problems, the paper shows the jointly optimal solution is a weighted average of the individually optimal solutions, with matrix weights proportional to the unnormalized input covariances; under normalized, roughly isotropic features this becomes a sample-size-weighted average. Because sample sizes are unknown for open models, the paper invokes the empirical regularity that parameter variance decreases with training data volume, assumes zero-centered updates so variance is tracked by the squared Frobenius norm, and then deliberately uses the unsquared norm for numerical stability. The resulting NAN coefficient for model $i$ is $\alpha_i = (1/\|W_i\|_F) / \sum_j (1/\|W_j\|_F)$, applied either to raw weights or as a reweighting of an existing merging strategy's output.

Load-bearing premise

The method stands on the empirical relation that more training data makes a model's parameters lower-variance, and on the extra step of using the plain Frobenius norm instead of its square as the proxy for that variance.

Editorial extensions

If this is right

  • Plugging NAN into Task Arithmetic or Ties-Merging raises average accuracy across the eight vision tasks, the two-LLM language setting, and the two-VLM setting, with the largest gains on GSM8K and GeoQA.
  • NAN is training-free and requires only the checkpoints themselves, so it can be applied when data, Fisher information, or sample sizes are unavailable.
  • Because the coefficient rule is a one-line formula, it can be composed with any merging strategy that produces a weighted combination of task vectors or raw weights.
  • The same inverse-norm rule works across backbones (ViT-B/32, ViT-L/14, LLaMA2-13B, LLaVA-v1.5-13B) and across vision, language, and multimodal tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The derivation's sample-size step assumes normalized, isotropic features, so the cleanest test of the paper's logic is comparing NAN's inverse-norm weights against true sample-size weights on models where those sizes are published; a mismatch there would point to the proxy, not the least-squares core, as the fragile link.
  • Because the paper uses the unsquared norm while its variance argument naturally gives the squared norm, a direct ablation switching between $\|W\|_F$ and $\|W\|_F^2$ would isolate whether the stability motivation or the variance proxy is doing the empirical work.
  • The $m/2$ global rescaling is a practical band-aid for coefficient shrinkage with many models; one could test whether per-layer or per-block norm ratios behave differently from the global norm, which would decide whether a single scalar per model is the right granularity.
  • If the inverse-norm rule is genuinely capturing task-specific information, it should also predict which pairs of fine-tuned models will merge well before evaluation, making it useful as a cheap screening tool for model selection.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Norm-Aware Merging (NAN), a training-free coefficient estimator for model merging. Starting from a least-squares formulation for two tasks, the authors derive that under isotropic normalized features the optimal merged model is approximately a sample-size-weighted average of individually trained models (Eq. (9)). Because sample sizes are unknown in practice, they invoke an empirical relation n ∝ 1/Var(W), approximate Var(W) by the squared Frobenius norm, and then define coefficients α_i = (1/||W_i||_F) / Σ_j (1/||W_j||_F) (Eq. (10)). They plug these coefficients into Task Arithmetic and Ties-Merging and evaluate on eight image classification tasks, language tasks with LLaMA-2, and VLM tasks with LLaVA, reporting average improvements.

Significance. If the derivation were valid, NAN would be a valuable zero-cost plugin for task-vector merging. The paper has strengths: the method is simple, training-free, and tested across three domains (ViT-B/32, ViT-L/14, LLaMA-2-13B, LLaVA-13B), and the writing is concise. The experimental comparisons cover representative baselines. However, the central theoretical claim is not supported by the derivation: Eq. (10) is a heuristic re-scaling of the implied inverse-squared norm rule. The empirical improvements are also modest and occasionally negative. Significance is therefore limited; at best the paper offers a plausible heuristic with partial empirical support. No code is provided, which further limits reproducibility.

major comments (4)
  1. [§3.3, Eq. (10)] Eq. (10) does not follow from Eqs. (8)–(9). The derivation in §3.2 yields sample-size weights α_i = n_i / Σ_j n_j; combining this with the stated proxy n ∝ 1/Var(W) and Var(W) ∝ ||W||_F^2 gives α_i ∝ 1/||W_i||_F^2, not the 1/||W_i||_F used in Eq. (10). The intervening sentence about numerical stability changes the functional form rather than deriving it, so the central claim that NAN's coefficients are theoretically grounded is unsupported.
  2. [§3.3, paragraph beginning 'Recent findings suggest'] The empirical law n ∝ 1/Var(W) is load-bearing and is cited primarily to the authors' own Si et al. (2025) and to Du et al. (2025), with no demonstration that it holds for the ViT, LLaMA-2, or LLaVA fine-tuned checkpoints used in the experiments. Since parameter variance also depends on initialization scale, learning rate, regularization, and task difficulty, this proxy is not established for the models at hand; without it, the bridge from sample-size weights to norm-based weights collapses.
  3. [§3.1–§3.2] The linear least-squares solution is used as a proxy for fine-tuned deep-network weights, but no argument is given that the closed-form solution of Eq. (2) approximates the SGD-trained checkpoints being merged. The matrix-valued coefficients in Eq. (8) are then reduced to scalar sample-size weights using an isotropy assumption that is plausible for normalized features but not verified for the learned representations of the tested models.
  4. [§4, Tables 1–3] The claim that NAN 'consistently improves' the baselines is too strong. Several entries show degradation, e.g., TA+NAN on ViT-B/32 SVHN drops from 80.2 to 76.2, and Ties+NAN on ViT-L/14 is below plain Ties on SUN397, Cars, and RESISC45. Without standard deviations or significance tests, the evidence supports only 'sometimes improves, sometimes slightly hurts,' which weakens the empirical claim as stated.
minor comments (4)
  1. [Abstract] The sentence 'Extensive experiments on show that NAN consistently improves...' is incomplete; the missing specification should be supplied.
  2. [§4, Baselines] AdaMerging is attributed to Yadav et al. (2023), but the AdaMerging paper is Yang et al. (2023); additionally, AdaMerging++ and Layer-wise AdaMerging appear in Table 1 but are not described in the baselines paragraph.
  3. [§3.3] The global scaling factor m/2 is introduced with no derivation or ablation; since it scales all coefficients equally it cannot affect relative coefficients, so its role should be stated explicitly.
  4. [Eq. (10)] Clarify whether W_i denotes full model weights or task vectors; the text says 'model's weights W' but the method is applied both directly to raw weights and as a reweighting step in task-vector merging.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; the empirical evaluation is external, though the derivation of Eq. (10) contains a non-circular logical gap and relies in part on self-citations.

full rationale

The paper's central claim is that optimal merging weights track task-specific information, derived via least squares to Eq. (9) (sample-size-weighted averaging). Section 3.3 then replaces unknown sample sizes with an empirical proxy n ∝ 1/Var(W), citing Fort et al. (2019), Izmailov et al. (2018), Si et al. (2025), and Du et al. (2025). With the stated zero-mean assumption, Var(W) ∝ ||W||_F^2, so the proxy chain would imply α_i ∝ 1/||W_i||_F^2; the paper instead adopts 1/||W_i||_F, explicitly for numerical stability. This is an ungrounded functional-form change and a correctness weakness, but it is not circular: Eq. (10) is not an input to the derivation, and no fitted parameter is relabeled as a prediction. The method is parameter-free and evaluated on external benchmarks (eight image tasks, four language tasks, four VLM tasks), so improved merging performance is not equivalent by construction to the method's assumptions. The self-citations to Si et al. (2025) for the variance–sample-size law and zero-centered weights are shared with non-overlapping sources, making them supportive rather than the sole load-bearing justification. The stated limitation (shared-backbone scope) is a scope restriction, not a circularity. Overall, the logical gap between Eq. (9) and Eq. (10) belongs in correctness risk, not circularity; the self-citation component is minor, warranting a score of 2.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

No new entities are introduced. The method rests on two hand-chosen parameters (global scaling factor m/2 and norm exponent 1) and five assumptions, two of which (the variance-sample-size law and the linear-proxy assumption) are ad hoc to this paper.

free parameters (2)
  • global scaling factor m/2 = m/2 (m = number of models)
    Ad hoc multiplier applied to merged weights to compensate for small softmax-normalized coefficients; not derived from least-squares theory.
  • norm exponent = 1 (instead of 2)
    Theory suggests weights proportional to 1/Var(W) ∝ 1/||W||^2, but the paper uses 1/||W|| citing numerical stability; this is a hand-chosen modification.
assumptions (5)
  • standard math The joint least-squares solution is a matrix-weighted average of per-task solutions
    Standard linear algebra result (Equations 6 to 8).
  • domain assumption Input features are normalized and approximately isotropic, so A_i = X_i^T X_i ≈ n_i I_d
    Required to convert matrix weights to scalar sample-size weights (Equation 9). Unlikely to hold for real deep-network features.
  • domain assumption Fine-tuned model weights are approximately zero-centered, so Var(W) ∝ ||W||_F^2
    Used to justify using the Frobenius norm as a variance proxy; cited to Si et al. (2025) and Du et al. (2025).
  • ad hoc to paper Training sample size n_i is inversely proportional to weight variance (n ∝ 1/Var(W))
    Load-bearing empirical law cited to prior work, including a self-citation (Si et al., 2025). This is the bridge from the least-squares result to the norm-based formula.
  • ad hoc to paper Linear least-squares solutions are a valid proxy for fine-tuned neural network weights
    The derivation treats each fine-tuned model as the exact solution of an independent least-squares problem and the merged model as the exact solution of the joint problem; real fine-tuning is non-convex and regularized.

how reviews work

0 comments
Cite this review

Pith. "Pith review of NAN: A Training-Free Solution to Coefficient Estimation in Model Merging." pith.science (2026). https://pith.science/paper/CARIZICC

@misc{pith2026250516148,
  author       = {Pith},
  title        = {Pith review of: NAN: A Training-Free Solution to Coefficient Estimation in Model Merging},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CARIZICC}},
  note         = {Machine review of arXiv:2505.16148}
}
read the original abstract

Model merging offers a training-free alternative to multi-task learning by combining independently fine-tuned models into a unified one without access to raw data. However, existing approaches often rely on heuristics to determine the merging coefficients, limiting their scalability and generality. In this work, we revisit model merging through the lens of least-squares optimization and show that the optimal merging weights should scale with the amount of task-specific information encoded in each model. Based on this insight, we propose NAN, a simple yet effective method that estimates model merging coefficients via the inverse of parameter norm. NAN is training-free, plug-and-play, and applicable to a wide range of merging strategies. Extensive experiments on show that NAN consistently improves performance of baseline methods.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 12 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Rich Caruana. 1997. Multitask learning. Machine learning, 28:41--75

    1997

  4. [4]

    Jiaqi Chen, Jianheng Tang, Jinghui Qin, Xiaodan Liang, Lingbo Liu, Eric P Xing, and Liang Lin. 2021. Geoqa: A geometric question answering benchmark towards multimodal numerical reasoning. arXiv preprint arXiv:2105.14517

    arXiv 2021

  5. [5]

    Gong Cheng, Junwei Han, and Xiaoqiang Lu. 2017. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, 105(10):1865--1883

    work page 2017

  6. [6]

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. 2014. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3606--3613

    work page 2014

  7. [7]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    arXiv 2021

  8. [8]

    OpenCompass Contributors. 2023. Opencompass: A universal evaluation platform for foundation models. https://github.com/open-compass/opencompass

    2023

Show all 45 references
  1. [9]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human...

    2019

  2. [10]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, and 1 others. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv p...

    2020 arXiv

  3. [11]

    Zhekai Du, Yinjie Min, Jingjing Li, Ke Lu, Changliang Zou, Liuhua Peng, Tingjin Chu, and Mingming Gong. 2025. Loca: Location-aware cosine adaptation for parameter-efficient fine-tuning. arXiv preprint arXiv:2502.06820

    2025 arXiv

  4. [12]

    Stanislav Fort, Huiyi Hu, and Balaji Lakshminarayanan. 2019. Deep ensembles: A loss landscape perspective. arXiv preprint arXiv:1912.02757

    2019 arXiv

  5. [13]

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. 2019. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217--2226

    2019

  6. [14]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR)

    2021

  7. [15]

    Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Jiayi Lei, Yao Fu, Maosong Sun, and Junxian He. 2023. C-eval: A multi-level multi-discipline chinese evaluation suite for foundation models. In Advances in N...

    2023

  8. [16]

    Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2022. Editing models with task arithmetic. arXiv preprint arXiv:2212.04089

    2022 arXiv

  9. [17]

    Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. 2018. Averaging weights leads to wider optima and better generalization. arXiv preprint arXiv:1803.05407

    2018 arXiv

  10. [18]

    Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro, and Pengxiang Cheng. 2022. Dataless knowledge fusion by merging weights of language models. arXiv preprint arXiv:2212.09849

    2022 arXiv

  11. [19]

    Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. 2016. A diagram is worth a dozen images. In Computer Vision--ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11--14, 2016, Proceedings, Part IV 14, pages 2...

    2016

  12. [20]

    Edan Kinderman, Itay Hubara, Haggai Maron, and Daniel Soudry. 2024. Foldable supernets: Scalable merging of transformers with different initializations and tasks. arXiv preprint arXiv:2410.01483

    2024 arXiv

  13. [21]

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 2013. 3d object representations for fine-grained categorization. In Proceedings of the IEEE international conference on computer vision workshops, pages 554--561

    2013

  14. [22]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual instruction tuning. In NeurIPS

    2023

  15. [23]

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2023. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255

    2023 arXiv

  16. [24]

    Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. 2023. Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. arXiv preprint arXiv:2308.09583

    2023 arXiv

  17. [25]

    Michael S Matena and Colin A Raffel. 2022. Merging models with fisher-weighted averaging. Advances in Neural Information Processing Systems, 35:17703--17716

    2022

  18. [26]

    Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, and 1 others. 2011. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, volume 2011, page 4. Granada

    2011

  19. [27]

    Runqi Qiao, Qiuna Tan, Guanting Dong, Minhui Wu, Chong Sun, Xiaoshuai Song, Zhuoma GongQue, Shanglin Lei, Zhe Wei, Miaoxuan Zhang, and 1 others. 2024. We-math: Does your large multimodal model achieve human-like mathematical reasoning? arXiv preprint arXiv:2407.01284

    2024 arXiv

  20. [28]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1--67

    2020

  21. [29]

    Sebastian Ruder. 2016. An overview of gradient descent optimization algorithms. arXiv preprint arXiv:1609.04747

    2016 arXiv

  22. [30]

    Wenhao Shi, Zhiqiang Hu, Yi Bin, Junhua Liu, Yang Yang, See-Kiong Ng, Lidong Bing, and Roy Ka-Wei Lee. 2024. Math-llava: Bootstrapping mathematical reasoning for multimodal large language models. arXiv preprint arXiv:2406.17294

    2024 arXiv

  23. [31]

    Chongjie Si, Jingjing Jiang, and Wei Shen. 2025. Unveiling the mystery of weight in large foundation models: Gaussian distribution never fades. arXiv preprint arXiv:2501.10661

    2025 arXiv

  24. [32]

    Johannes Stallkamp, Marc Schlipsing, Jan Salmen, and Christian Igel. 2011. The german traffic sign recognition benchmark: a multi-class classification competition. In The 2011 international joint conference on neural networks, pages 1453--1460. IEEE

    2011

  25. [33]

    George Stoica, Daniel Bolya, Jakob Bjorner, Pratik Ramesh, Taylor Hearn, and Judy Hoffman. 2023. Zipit! merging models from different tasks without training. arXiv preprint arXiv:2305.03053

    2023 arXiv

  26. [34]

    Mirac Suzgun, Nathan Scales, Nathanael Sch \"a rli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, , and Jason Wei. 2022. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261

    2022 arXiv

  27. [35]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  28. [36]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R \'e mi Louf, Morgan Funtowicz, and 1 others. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 conference on empiri...

    2020

  29. [37]

    Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and 1 others. 2022. Model soups: averaging weights of multiple fine-tuned models improves accuracy without in...

    2022

  30. [38]

    Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. 2010. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE computer society conference on computer vision and pattern recognition, pages 3485--3492. IEEE

    2010

  31. [39]

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. 2024. Wizardlm: Empowering large pre-trained language models to follow complex instructions. In The Twelfth International Conference on Learning Representations

    2024

  32. [40]

    Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. 2023. Ties-merging: Resolving interference when merging models. Advances in Neural Information Processing Systems, 36:7093--7115

    2023

  33. [41]

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, and 1 others. 2024. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122

    2024 arXiv

  34. [42]

    Enneng Yang, Zhenyi Wang, Li Shen, Shiwei Liu, Guibing Guo, Xingwei Wang, and Dacheng Tao. 2023. Adamerging: Adaptive model merging for multi-task learning. arXiv preprint arXiv:2310.02575

    2023 arXiv

  35. [43]

    LeCun Yann. 1998. 'http://yann.lecun. com/exdb/mnist/' The mnist database of handwritten digits

    1998

  36. [44]

    Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2024 a . Extend model merging from fine-tuned to pre-trained large language models via weight disentanglement. arXiv preprint arXiv:2408.03092

    2024 arXiv

  37. [45]

    Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2024 b . Language models are super mario: Absorbing abilities from homologous models as a free lunch. In Forty-first International Conference on Machine Learning

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.