Training-free LLM Merging for Multi-task Learning

Hongzhi Yin; Shanshan Ye; Wanyu Wang; Xiangyu Zhao; Xian Wu; Yefeng Zheng; Yejing Wang; Yi Chang; Zichuan Fu

REVIEW 4 major objections 5 minor 49 references

Training-free LLM Merging for Multi-task Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Hi-Merging fuses two fine-tuned LLMs into one model that outperforms existing merging methods and, in most tested scenarios, beats a model fine-tuned on the combined datasets.

desk verdict Layer-wise contribution-guided conflict elimination is a real increment over DARE/TIES/DELLA, but the contribution scores are computed on undisclosed splits, so the headline comparison with multi-task fine-tuning needs a clean held-out evaluation before it can be trusted. read the letter →

arxiv 2506.12379 v1 pith:54JYBCNV submitted 2025-06-14 cs.CL cs.AIcs.LG

Zichuan Fu , Xian Wu , Yejing Wang , Wanyu Wang , Shanshan Ye , Hongzhi Yin , Yi Chang , Yefeng Zheng

show 1 more author

Xiangyu Zhao

This is my paper

classification cs.CLcs.AIcs.LG

keywords modelmergingmulti-tasklearningdeltavectorspruningandscalingcontributionanalysisparameterconflictmultilingualLLMstraining-free

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Hi-Merging claims that two specialized fine-tuned LLMs sharing a foundation model can be fused into one model with no additional training, by pruning and scaling their delta vectors at two levels. A model-wise pass keeps only the largest parameter updates and shrinks their magnitude, removing noise and over-sharp updates. A layer-wise pass then scores each transformer layer's contribution to each task and iteratively repairs the most conflicted layers by dropping, pruning, or scaling one model's delta. The paper reports that the merged model consistently outperforms existing merging baselines and, in most of the tested bilingual and multi-task scenarios, also beats a model fine-tuned on the combined datasets. If this holds, users could combine public task-specific models without retraining or accessing original fine-tuning data.

What carries the argument

The load-bearing object is the contribution score $c_l^{m_1,m_2} = \alpha_l^{m_1,m_2} + \beta_l^{m_1,m_2}$, where $\alpha$ measures the performance drop on task $t_{m_1}$ when layer $l$'s delta is removed from a merged model and $\beta$ measures the gain when that layer's delta is added to the foundation model. From it the paper defines a per-model, per-layer conflict $\gamma_l^m = c_{m,m} - c_{m,G}$; summing over the two models gives $\Gamma_l$, the ranking used to choose which layers to repair first. The repair operations are the same pruning (keep the largest $p$ fraction of delta entries) and scaling (multiply by $s$) used in the model-wise pass, applied iteratively layer by layer. The hierarchy is what carries the argument: the model-wise pass removes noise and overfitting before the layer-wise pass resolves knowledge-alignment conflicts.

What would settle it

Re-run Hi-Merging with contribution scores computed on an explicit held-out validation split, then with scores computed on the test split; if the reported superiority over baselines vanishes or reverses, the conflict ranking was selecting on the evaluation labels. As a second check, repair layers in an order opposite to the contribution ranking; if final accuracy moves a lot, the additivity assumption fails.

Watch

Extended reading notes

Core claim

The central claim is that parameter conflict in LLM merging is concentrated in a few layers and can be located and fixed without training. Hi-Merging writes each fine-tuned model as the foundation weights plus a delta vector $\delta_m$, first applying top-$p$ pruning and a scaling factor $s$ to each delta; the paper shows this can even improve a single fine-tuned model, e.g. $p=0.1, s=0.9$ on MedQA. It then forms a naive merged model and, for every layer $l$, measures a deletion impact $\alpha$ and an addition impact $\beta$ on each task, defining contribution $c_l = \alpha + \beta$ and conflict $\gamma_l^m = c_{m,m} - c_{m,G}$. Layers with the largest total conflict are processed iteratively: if both models are hurt, the larger contribution is kept; if one is hurt, that delta is pruned and scaled again; if both improve, the layer is left unchanged. The final model is $\theta_F + \delta_A + \delta_B$. Experiments on Chinese and English multiple-choice and open QA report that this hierarchical, contribution-guided procedure outperforms existing merging methods and often exceeds the multi-task fine-tuned model.

Load-bearing premise

The method assumes that a layer's importance to the final merged model is truthfully measured by adding or deleting that layer's delta and scoring on labeled task data, that these per-layer contributions add up, and that the labels used for scoring are not the same test labels later reported; the paper does not state which split is used.

Editorial extensions

If this is right

If the claim is right, two public fine-tuned models can be combined into one model without retraining or seeing the original fine-tuning data, as demonstrated for Chinese/English MCQA, QA, and open-source medical models.
The merged model can beat a model fine-tuned on the combined datasets, which the paper attributes to avoiding task interference during training and to the sparsity of LLM parameter space.
The method's success depends on the foundation model: results on Qwen2-7B-Instruct are strong, while on Llama-3-8B-Instruct merged models can fall short of individual fine-tuned models, which the paper attributes to noisier delta vectors.
Per the ablation, the layer-wise pass contributes more than the model-wise pass, and removing layer-wise scaling hurts more than removing layer-wise pruning.
Merging two same-domain medical models is mutually beneficial, but merging medical and math models causes slight drops on both domains, so gains shrink as fine-tuning domains diverge.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The contribution analysis requires labeled task data and one forward pass per layer, so 'training-free' means no gradient updates rather than no data or compute cost; the overhead grows with model depth and scoring-set size.
The paper never states whether the labels used for $\alpha$ and $\beta$ come from a held-out split or from the same test set later reported; if they come from the test set, the iterative ranking is selecting on the evaluation metric and the reported gains would be optimistic.
The additivity assumption, that per-layer contributions can be summed, can be checked by repairing layers in a different order; if final accuracy changes substantially, interactions dominate and the ranking is not stable.
The same scoring machinery could also choose a foundation model or merge more than two models by hierarchical pairwise combination, though the paper only demonstrates pairwise merging from one foundation model.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things you should know. First, the actual increment here is real: Hi-Merging builds on the DARE/TIES/DELLA toolkit but adds layer-wise, contribution-guided conflict elimination, with iterative handling of severe, partial, and mutually beneficial layers. That is not reducible to the cited parameter-wise methods. Second, the headline numerical claim is currently unverifiable, because the contribution scores in Eqs. (8)-(10) are computed with the same task metrics that the final tables report, and the paper never discloses which split supplies them.

What the paper does well: the problem is sensible, the hierarchy of model-wise then layer-wise pruning and scaling is well motivated, and the experiments are broad—bilingual MCQA, bilingual QA, cross-task cross-language merging, LoRA fine-tuned and off-the-shelf medical models, ablations, and a second base model. Code is promised. The limitations section is honest about the two-model-only scope. The citation pattern looks normal, with the relevant baselines included; no red flags there.

The soft spot is the evaluation protocol, and it is load-bearing. The contribution score c_l = alpha_l + beta_l sums a deletion counterfactual and an addition counterfactual, which is a heuristic with no shown additivity. The algorithm also computes c_l once and then iteratively prunes layers without recomputing, which presumes that additivity. More importantly, p and s are grid-searched over [0.1, 1.0], and the layer-wise values are set proportional to model-wise values, but the chosen values are never reported. If the same test split that produces the tables also drives the contribution analysis and the grid search, then the final numbers are the objective of a selection procedure, and the comparison against multi-task fine-tuning is circular. The reported differences are small—60.156 vs 59.531 on a 400-example MedQA set, for instance—and there are no error bars or significance tests.

One additional mismatch: on Llama-3-8B-Instruct, the appendix shows Hi-Merging below the multi-task model on both MCQA and QA (Tables 9 and 11). That does not kill the "most scenarios" claim, but it should be read carefully.

None of this means the method is wrong. A clean validation split could fix most of the concern, and the merge heuristic itself is worth investigating. But as submitted, the central claim is unsupported. It still deserves full peer review, precisely because the idea is plausible and the fix is feasible. Send it to reviewers, but insist on split disclosure, reported hyperparameter values, and variance estimates before publication. I would not cite it yet.

Referee Report

4 major / 5 minor

Summary. The paper proposes Hi-Merging, a training-free method for merging two fine-tuned large language models that share the same foundation model. The method first applies model-wise pruning and scaling to each model's delta vector, then performs iterative layer-wise pruning and scaling guided by a new 'contribution analysis' computed from deletion and addition counterfactuals. Experiments cover bilingual answering (MedQA/CMExam), bilingual QA (HealthCareMagic/cMedQA2), cross-lingual multi-task settings, and merging of open-source medical LLMs, with claims that Hi-Merging consistently outperforms existing merging baselines and often surpasses joint fine-tuning on combined data.

Significance. If the claims are sound, the paper addresses a practical problem: fusing existing task-specialized LLMs without access to private training data or additional training. The paper releases code, includes several realistic merging scenarios, and evaluates on medical QA datasets in two languages, including merging publicly available community models. These are concrete strengths. However, the central quantitative claim is currently supported by an evaluation protocol that does not clearly separate the data used for contribution analysis and hyperparameter selection from the data used for final reporting, and the reported differences from baselines are often small in absolute terms. The contribution-analysis step is also not theoretically justified. The paper is therefore promising but needs substantial revision before its headline comparisons can be accepted.

major comments (4)

[Section 3.2.1, Eqs. (8)-(10); Section 4.1.3; Tables 2-4] The paper never states which data split supplies P_t in the contribution scores of Eqs. (8)-(9). Table 1 explicitly distinguishes train, validation, and test, but the method description, the grid search over p and s in Section 4.1.3, and the reported tables all use the same metric names on the same datasets. If the contribution analysis, the selection of p/s, and the layer-wise conflict elimination are performed on the test split whose numbers are later reported in Tables 2-4, then the reported gains are partly selection artifacts rather than independent test results. The authors must specify the split used for each selection step and report final test scores obtained after fixing all hyperparameters.
[Section 3.2.1, Eq. (10)] The additive form c_l = alpha_l + beta_l is not justified. alpha_l measures the effect of removing layer l from the merged model, while beta_l measures the effect of adding layer l to the frozen base model. For a smooth performance function, these two counterfactuals are approximately opposite first-order terms, so c_l is approximately zero to first order; the nonzero signal that drives the algorithm is therefore second-order or noise. The paper gives no additivity argument, and the layer-wise scores are computed once and not recomputed after each layer is modified in the iterative procedure of Section 3.2.2. This undermines the claim that Eq. (10) truthfully measures each layer's contribution to the final merged model.
[Section 4.2, Table 2; Section 4.3, Table 3; Section 4.4, Table 4] The performance differences between Hi-Merging and strong baselines are often very small, and no variance or significance information is reported. In Table 2, for example, Hi-Merging's MedQA accuracy is 60.1562 versus 59.5312 for Task Arithmetic on a 400-example test set, and the average-rank comparison across methods is sensitive to small metric differences. Without multiple seeds or an appropriate significance test, the claim of 'consistently outperforming' existing merging techniques is not supported. The authors should report repeated-run variability or, at minimum, justify that the observed differences exceed expected noise.
[Section 4.1.3] The chosen values of the pruning threshold p and scaling factor s are never reported for any experiment. The text states only that p and s range from 0.1 to 1.0 and that the layer-wise values are half of the model-wise values. Since these hyperparameters are selected on the basis of the same task metrics later used for evaluation, the absence of reported values makes both reproducibility and the independence of the reported results impossible to assess. The paper should disclose the selected p and s for each table and clearly indicate which split was used for their selection.

minor comments (5)

[Abstract] The sentence 'We introduces Hierarchical Iterative Merging' contains a subject-verb agreement error; it should be 'We introduce'.
[Figure 1 caption] The caption lists three paradigms but labels the third as '(left)' in two places; the third panel should presumably be '(right)'. The figure also appears to be referenced before it is introduced in the text.
[Section 4.6] The text says 'leading to the improved performance demonstrated in previous experiments in Table 8,' but Table 8 appears in the appendix and is a QA experiment; the intended cross-reference is likely Table 2 or the main experimental section. This should be corrected.
[Section 3.2.1, Eqs. (8)-(9)] The notation θ_m2 - \hatδ^l_m2 is ambiguous: it is not clear whether \hatδ^l_m2 is a layer-indexed component of the full delta vector or a full delta vector with only layer l modified, and the same ambiguity affects the addition term in Eq. (9). Define the layer-restricted delta vector explicitly.
[Section 4.1.3 and Appendix A.2] The Appendix results for Llama-3-8B-Instruct (Tables 9 and 11) show that Hi-Merging does not always exceed the fine-tuned individual models, and the abstract's wording 'surpasses the performance of models fine-tuned on combined datasets in most scenarios' is appropriately qualified. The main text should explicitly carry this caveat into the discussion of RQ4 and the generality of the method.

Circularity Check

2 steps flagged · score 6.0 of 10

Contribution analysis and the p/s grid search are fitted on the same accuracy/BLEU/ROUGE metrics that Tables 2-4 report, so the headline gains are selection objectives rather than independent predictions.

fitted input called prediction [Section 3.2.1-3.2.2, Eqs. (8)-(13)]
"Ptm1(·) represents the performance metric on the task tm1. For example, BLEU-4 (Papineni et al., 2002) score for the QA task. ... αl m1,m2 = Ptm1(θm2 − ˆδ l m2) − Ptm1(θm2), βl m1,m2 = Ptm1(ˆθF + δl m2) − Ptm1(θF). ... We sum up two impacts as the overall contribution: cl m1,m2 = αl m1,m2 + βl m1,m2."

The layer conflict score γl in Eq. (11) is built from these c-values, and the iterative procedure prunes, scales, or zeroes the most conflicting layer deltas; the final weights in Eq. (13) are therefore chosen to improve Pt on exactly the tasks reported in Tables 2-4. The paper never states that Pt in Eqs. (8)-(9) is restricted to a held-out split, even though Table 1 distinguishes train, validation, and test. As written, the reported accuracy/BLEU/ROUGE values are the selection objective of the contribution-guided pruning, not independent out-of-sample predictions. The claimed gains over multi-task fine-tuning are thus fitted quantities for this method.
fitted input called prediction [Section 3.1 and Section 4.1.3, Figure 2]
"Figure 2: The accuracy of the fine-tuned Qwen2-7B-Instruct on the MedQA dataset after the model-wise pruning and scaling process with different combinations of the pruning threshold p and the scaling factor s. ... Both p and s in model-wise process range from 0.1 to 1.0 with a step of 0.1."

The model-wise pruning threshold p and scaling factor s are searched by measuring accuracy on MedQA, which is one of the reported evaluation metrics in Tables 2-4. The paper text says it 'empirically validate[s]' the operations by iterating p and s over [0.1,1] and identifies combinations such as p=0.1, s=0.9 as defeating the original model. The layer-wise p and s are then set to half of the model-wise values. Consequently, the hyperparameters defining the final Hi-Merging model are fitted to the same dataset metrics that are later reported as evidence of superiority.

full rationale

The core concern is not self-citation or imported uniqueness theorems; the paper is a self-contained merging recipe. However, the central comparison is contaminated by target-metric selection. Eq. (8)-(10) define layer contribution as a function of Pt, Eqs. (11)-(13) use that contribution to choose which layer deltas to keep, prune, or delete, and Tables 2-4 report Pt. This is a fitted-input-called-prediction pattern: the final model's reported performance is the objective of the selection procedure, not a result obtained after committing to fixed decisions independently of those metrics. The p/s grid search in Figure 2 is also conducted on MedQA accuracy, one of the reported numbers. The paper never ties Pt to a declared validation split, so the reader cannot tell whether the reported test metrics were used to select the method's operations. This does not make the final weights literally equal to the input by definition, so a full score-8/10 circularity is not warranted; but it makes the headline claim of surpassing multi-task fine-tuning a selection artifact-at-risk, warranting 6/10.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on several free hyperparameters not reported and on the unstated availability of labeled evaluation data for contribution analysis. No new physical or architectural entities are introduced.

free parameters (3)

Model-wise pruning threshold p_m per model (p_A, p_B) = Not reported; grid searched over {0.1, ..., 1.0}
Controls the fraction of delta vector kept (Eq 4-7). Figure 2 shows performance varies substantially with p, so the final choice affects all downstream comparisons.
Model-wise scaling factor s_m per model = Not reported; grid searched over {0.1, ..., 1.0}
Controls the magnitude of retained delta values (Eq 7). Tuned on task performance without disclosing selected values.
Layer-wise pruning threshold and scaling factor = Set to half of the model-wise values (Sec 4.1.3)
Used in partial conflict resolution (Eq 12). The halving rule is ad hoc and not justified.

assumptions (3)

ad hoc to paper Layer contributions are additive: the total contribution c_l = alpha_l + beta_l (Eq 10) predicts the layer's value in the merged model.
Eq (10) sums a deletion impact and an addition impact without proof that the two are independent or that they predict interaction in the merged model.
domain assumption Labeled performance data for every task is available and representative at merge time, and its use does not leak into the reported test evaluation.
Eq (8)-(9) require P_t on each task. The paper does not specify which split is used, yet the entire merit of the reported numbers depends on this.
domain assumption After pruning and scaling, delta vectors can be linearly added to the base model to produce a functional merged model.
Eq (13) sets theta_merge = theta_F + delta_A + delta_B, relying on the standard delta-merging linearity assumption.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Training-free LLM Merging for Multi-task Learning." pith.science (2026). https://pith.science/paper/54JYBCNV

@misc{pith2026250612379,
  author       = {Pith},
  title        = {Pith review of: Training-free LLM Merging for Multi-task Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/54JYBCNV}},
  note         = {Machine review of arXiv:2506.12379}
}

read the original abstract

Large Language Models (LLMs) have demonstrated exceptional capabilities across diverse natural language processing (NLP) tasks. The release of open-source LLMs like LLaMA and Qwen has triggered the development of numerous fine-tuned models tailored for various tasks and languages. In this paper, we explore an important question: is it possible to combine these specialized models to create a unified model with multi-task capabilities. We introduces Hierarchical Iterative Merging (Hi-Merging), a training-free method for unifying different specialized LLMs into a single model. Specifically, Hi-Merging employs model-wise and layer-wise pruning and scaling, guided by contribution analysis, to mitigate parameter conflicts. Extensive experiments on multiple-choice and question-answering tasks in both Chinese and English validate Hi-Merging's ability for multi-task learning. The results demonstrate that Hi-Merging consistently outperforms existing merging techniques and surpasses the performance of models fine-tuned on combined datasets in most scenarios. Code is available at: https://github.com/Applied-Machine-Learning-Lab/Hi-Merging.

Figures

Figures reproduced from arXiv: 2506.12379 by the authors.

**Figure 2.** The accuracy of the fine-tuned Qwen2-7B [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** The demonstration of different conflict elimi [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Performance of Hi-Merging on two opensource medical models, Echelon-AI/Med-Qwen2-7B and shtdbb/qwen2-7b-med, which are fine-tuned from the foundation model Qwen/Qwen2-7B-Instruct. models consistently outperform their individual fine-tuned counterparts, with many even …

**Figure 5.** Figure 5: The visualization of layer contributions and [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

**Figure 6.** Figure 6: Impact of training sample size on model merging conflicts. Blue and orange lines represent the average [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

49 extracted references · 18 canonical work pages

[1]

Takuya Akiba, Makoto Shing, Yujin Tang, Qi Sun, and David Ha. 2024. https://doi.org/10.48550/ARXIV.2403.13187 Evolutionary optimization of model merging recipes . CoRR, abs/2403.13187

arXiv doi:10.48550/arxiv.2403.13187 2024
[2]

Lucas Bandarkar, Benjamin Muller, Pritish Yuvraj, Rui Hou, et al. 2025. https://openreview.net/forum?id=vQhn4wrQ6j Layer swapping for zero-shot cross-lingual transfer in large language models . In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025 . OpenReview.net

work page 2025
[3]

Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, et al

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, et al. 2020. https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html Language models are few-shot learners . In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, De...

work page 2020
[4]

Junying Chen, Xidong Wang, Anningzhe Gao, Feng Jiang, Shunian Chen, Hongbo Zhang, Dingjie Song, Wenya Xie, Chuyi Kong, Jianquan Li, Xiang Wan, Haizhou Li, and Benyou Wang. 2023. https://arxiv.org/abs/2311.09774 Huatuogpt-ii, one-stage training for medical adaption of llms . Preprint, arXiv:2311.09774

arXiv 2023
[5]

Daixuan Cheng, Shaohan Huang, and Furu Wei. 2024. https://openreview.net/forum?id=y886UXPEZ0 Adapting large language models via reading comprehension . In The Twelfth International Conference on Learning Representations

work page 2024
[6]

Yiming Cui, Ziqing Yang, and Xin Yao. 2023. https://arxiv.org/abs/2304.08177 Efficient and effective text encoding for chinese llama and alpaca . arXiv preprint arXiv:2304.08177

arXiv 2023
[7]

Kia Dashtipour, Soujanya Poria, Amir Hussain, Erik Cambria, Ahmad Y. A. Hawalah, Alexander F. Gelbukh, and Qiang Zhou. 2016. https://doi.org/10.1007/S12559-016-9415-7 Multilingual sentiment analysis: State of the art and independent comparison of techniques . Cogn. Comput., 8(4):757--771

work page doi:10.1007/s12559-016-9415-7 2016
[8]

MohammadReza Davari and Eugene Belilovsky. 2024. https://doi.org/10.1007/978-3-031-73226-3\_16 Model breadcrumbs: Scaling multi-task model merging with sparse masks . In Computer Vision - ECCV 2024 - 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part LXXV , volume 15133 of Lecture Notes in Computer Science, pages 270--...

doi:10.1007/978-3-031-73226-3 2024

Show all 49 references

[9]

Pala Tej Deep, Rishabh Bhardwaj, and Soujanya Poria. 2024. https://doi.org/10.48550/ARXIV.2406.11617 Della-merging: Reducing interference in model merging through magnitude-based sampling . CoRR, abs/2406.11617

2024 doi arXiv
[10]

Jacob Devlin, Ming - Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/V1/N19-1423 BERT: pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North American Chapter of the Associat...

2019 doi
[11]

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

2024 arXiv
[12]

Elias Frantar and Dan Alistarh. 2023. https://proceedings.mlr.press/v202/frantar23a.html Sparsegpt: Massive language models can be accurately pruned in one-shot . In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA , volume 202 of...

2023
[13]

Zichuan Fu, Xiangyang Li, Chuhan Wu, Yichao Wang, Kuicai Dong, Xiangyu Zhao, Mengchen Zhao, Huifeng Guo, and Ruiming Tang. 2024. https://doi.org/10.1145/3698878 A unified framework for multi-domain ctr prediction via large language models . ACM Trans. Inf. Syst. Just Accepted

2024 doi
[14]

Mahak Gambhir and Vishal Gupta. 2017. https://doi.org/10.1007/S10462-016-9475-9 Recent automatic text summarization techniques: a survey . Artif. Intell. Rev., 47(1):1--66

2017 doi
[15]

Jinmin He, Kai Li, Yifan Zang, Haobo Fu, QIANG FU, Junliang Xing, and Jian Cheng. 2024. https://openreview.net/forum?id=3qUks3wrnH Efficient multi-task reinforcement learning with cross-task policy guidance . In The Thirty-eighth Annual Conference on Neural Information Process...

2024
[16]

Gabriel Ilharco, Marco T \' u lio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2023. https://openreview.net/forum?id=6t0Kwf8-jrj Editing models with task arithmetic . In The Eleventh International Conference on Learning Representations, ICL...

2023
[17]

Di Jin, Eileen Pan, Nassim Oufattole, Wei - Hung Weng, Hanyi Fang, and Peter Szolovits. 2020. https://arxiv.org/abs/2009.13081 What disease does this patient have? A large-scale open domain question answering dataset from medical exams . CoRR, abs/2009.13081

2020 arXiv
[18]

Xisen Jin, Xiang Ren, Daniel Preotiuc - Pietro, and Pengxiang Cheng. 2023. https://openreview.net/forum?id=FCnohuR6AnM Dataless knowledge fusion by merging weights of language models . In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwa...

2023
[19]

Yunxiang Li, Zihan Li, Kai Zhang, Ruilong Dan, Steve Jiang, and You Zhang. 2023. Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge. Cureus, 15(6)

2023
[20]

Chin-Yew Lin. 2004. https://aclanthology.org/W04-1013 ROUGE : A package for automatic evaluation of summaries . In Text Summarization Branches Out, pages 74--81, Barcelona, Spain. Association for Computational Linguistics

2004
[21]

Junling Liu, Peilin Zhou, Yining Hua, Dading Chong, Zhongyu Tian, Andrew Liu, Helin Wang, Chenyu You, Zhenhua Guo, Lei Zhu, et al. 2023. Benchmarking large language models on cmexam--a comprehensive chinese medical exam dataset. arXiv preprint arXiv:2306.03030

2023 arXiv
[22]

Qidong Liu, Xian Wu, Yejing Wang, Zijian Zhang, Feng Tian, Yefeng Zheng, and Xiangyu Zhao. 2024 a . http://papers.nips.cc/paper\_files/paper/2024/hash/2f0728449cb3150189d765fc87afc913-Abstract-Conference.html LLM-ESR: large language models enhancement for long-tailed sequentia...

2024
[23]

Qidong Liu, Xian Wu, Xiangyu Zhao, Yuanshao Zhu, Derong Xu, Feng Tian, and Yefeng Zheng. 2024 b . https://doi.org/10.1145/3626772.3657722 When MOE meets llms: Parameter efficient fine-tuning for multi-task medical applications . In Proceedings of the 47th International ACM SIG...

2024
[24]

Qidong Liu, Xian Wu, Xiangyu Zhao, Yuanshao Zhu, Zijian Zhang, Feng Tian, and Yefeng Zheng. 2024 c . https://doi.org/10.48550/ARXIV.2402.02803 Large language model distilling medication recommendation model . CoRR, abs/2402.02803

2024 doi arXiv
[25]

Guerreiro, Ricardo Rei, Duarte M

Pedro Henrique Martins, Patrick Fernandes, João Alves, Nuno M. Guerreiro, Ricardo Rei, Duarte M. Alves, José Pombal, Amin Farajian, Manuel Faysse, Mateusz Klimaszewski, Pierre Colombo, Barry Haddow, José G. C. de Souza, Alexandra Birch, and André F. T. Martins. 2024. https://a...

2024 arXiv
[26]

Michael Matena and Colin Raffel. 2022. http://papers.nips.cc/paper\_files/paper/2022/hash/70c26937fbf3d4600b69a129031b66ec-Abstract-Conference.html Merging models with fisher-weighted averaging . In Advances in Neural Information Processing Systems 35: Annual Conference on Neu...

2022
[27]

OpenAI. 2023. https://doi.org/10.48550/ARXIV.2303.08774 GPT-4 technical report . CoRR, abs/2303.08774

2023 doi arXiv
[28]

Kishore Papineni, Salim Roukos, Todd Ward, and Wei - Jing Zhu. 2002. https://doi.org/10.3115/1073083.1073135 Bleu: a method for automatic evaluation of machine translation . In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, ...

2002
[29]

Sidak Pal Singh and Martin Jaggi. 2020. https://proceedings.neurips.cc/paper_files/paper/2020/file/fb2697869f56484404c8ceee2985b01d-Paper.pdf Model fusion via optimal transport . In Advances in Neural Information Processing Systems, volume 33, pages 22045--22055. Curran Associ...

2020
[30]

Hongyan Tang, Junning Liu, Ming Zhao, and Xudong Gong. 2020. https://doi.org/10.1145/3383313.3412236 Progressive layered extraction (ple): A novel multi-task learning (mtl) model for personalized recommendations . In Proceedings of the 14th ACM Conference on Recommender System...

2020
[31]

Masatoshi Tsuchiya. 2018. https://aclanthology.org/L18-1239 Performance impact caused by hidden bias of training data for recognizing textual entailment . In Proceedings of the Eleventh International Conference on Language Resources and Evaluation ( LREC 2018) , Miyazaki, Japa...

2018
[32]

Haifeng Wang, Hua Wu, Zhongjun He, Liang Huang, and Kenneth Ward Church. 2022. https://doi.org/10.1016/j.eng.2021.03.023 Progress in machine translation . Engineering, 18:143--153

2022 doi
[33]

Yejing Wang, Zhaocheng Du, Xiangyu Zhao, Bo Chen, Huifeng Guo, Ruiming Tang, and Zhenhua Dong. 2023 a . https://doi.org/10.1145/3539618.3591767 Single-shot feature selection for multi-task recommendations . In Proceedings of the 46th International ACM SIGIR Conference on Resea...

2023
[34]

Yuhao Wang, Ha Tsz Lam, Yi Wong, Ziru Liu, Xiangyu Zhao, Yichao Wang, Bo Chen, Huifeng Guo, and Ruiming Tang. 2023 b . https://arxiv.org/abs/2302.03525 Multi-task deep recommender systems: A survey . Preprint, arXiv:2302.03525

2023 arXiv
[35]

Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, et al. 2022. https://proceedings.mlr.press/v162/wortsman22a.html Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time . In International Confer...

2022
[36]

Derong Xu, Ziheng Zhang, Zhenxi Lin, Xian Wu, Zhihong Zhu, Tong Xu, Xiangyu Zhao, Yefeng Zheng, and Enhong Chen. 2024 a . https://aclanthology.org/2024.lrec-main.1044 Multi-perspective improvement of knowledge graph completion with large language models . In Proceedings of the...

2024
[37]

Derong Xu, Ziheng Zhang, Zhihong Zhu, Zhenxi Lin, Qidong Liu, Xian Wu, Tong Xu, Xiangyu Zhao, Yefeng Zheng, and Enhong Chen. 2024 b . https://doi.org/10.18653/v1/2024.findings-emnlp.456 Mitigating hallucinations of large language models in medical information extraction via co...

2024 doi
[38]

Derong Xu, Ziheng Zhang, Zhihong Zhu, Zhenxi Lin, Qidong Liu, Xian Wu, Tong Xu, et al. 2024 c . https://doi.org/10.1145/3627673.3679673 Editing factual knowledge and explanatory ability of medical large language models . In Proceedings of the 33rd ACM International Conference ...

2024
[39]

Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al - Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. https://doi.org/10.18653/V1/2021.NAACL-MAIN.41 mt5: A massively multilingual pre-trained text-to-text transformer . In Proceedings of the 2021 Conferenc...

2021 doi
[40]

Raffel, and Mohit Bansal

Prateek Yadav, Derek Tam, Leshem Choshen, Colin A. Raffel, and Mohit Bansal. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/1644c9af28ab7916874f6fd6228a9bcf-Abstract-Conference.html Ties-merging: Resolving interference when merging models . In Advances in Neural Info...

2023
[41]

An Yang, Baosong Yang, Binyuan Hui, et al. 2024 a . https://doi.org/10.48550/ARXIV.2407.10671 Qwen2 technical report . CoRR, abs/2407.10671

2024 doi arXiv
[42]

Enneng Yang, Li Shen, Guibing Guo, Xingwei Wang, Xiaochun Cao, Jie Zhang, and Dacheng Tao. 2024 b . https://doi.org/10.48550/ARXIV.2408.07666 Model merging in llms, mllms, and beyond: Methods, theories, applications and opportunities . CoRR, abs/2408.07666

2024 doi arXiv
[43]

Enneng Yang, Zhenyi Wang, Li Shen, Shiwei Liu, Guibing Guo, Xingwei Wang, and Dacheng Tao. 2024 c . https://openreview.net/forum?id=nZP6NgD3QY Adamerging: Adaptive model merging for multi-task learning . In The Twelfth International Conference on Learning Representations, ICLR...

2024
[44]

Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2024. https://openreview.net/forum?id=fq0NaiU8Ex Language models are super mario: Absorbing abilities from homologous models as a free lunch . In Forty-first International Conference on Machine Learning, ICML 2024, Vienna...

2024
[45]

Sheng Zhang, Xin Zhang, Hui Wang, Lixiang Guo, and Shanshan Liu. 2018. https://doi.org/10.1109/ACCESS.2018.2883637 Multi-scale attentive interaction networks for chinese medical question answer selection . IEEE Access , 6:74061--74071

2018
[46]

Zijian Zhang, Shuchang Liu, Jiaao Yu, Qingpeng Cai, Xiangyu Zhao, Chunxu Zhang, Ziru Liu, Qidong Liu, Hongwei Zhao, Lantao Hu, Peng Jiang, and Kun Gai. 2024. https://doi.org/10.1145/3626772.3657686 M$ ^ 3 $oe: Multi-domain multi-task mixture-of experts recommendation framewo...

2024
[47]

Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, et al. 2024. https://arxiv.org/abs/2303.18223 A survey of large language models . Preprint, arXiv:2303.18223

2024 arXiv
[48]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
[49]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Takuya Akiba, Makoto Shing, Yujin Tang, Qi Sun, and David Ha. 2024. https://doi.org/10.48550/ARXIV.2403.13187 Evolutionary optimization of model merging recipes . CoRR, abs/2403.13187

arXiv doi:10.48550/arxiv.2403.13187 2024

[2] [2]

Lucas Bandarkar, Benjamin Muller, Pritish Yuvraj, Rui Hou, et al. 2025. https://openreview.net/forum?id=vQhn4wrQ6j Layer swapping for zero-shot cross-lingual transfer in large language models . In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025 . OpenReview.net

work page 2025

[3] [3]

Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, et al

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, et al. 2020. https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html Language models are few-shot learners . In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, De...

work page 2020

[4] [4]

Junying Chen, Xidong Wang, Anningzhe Gao, Feng Jiang, Shunian Chen, Hongbo Zhang, Dingjie Song, Wenya Xie, Chuyi Kong, Jianquan Li, Xiang Wan, Haizhou Li, and Benyou Wang. 2023. https://arxiv.org/abs/2311.09774 Huatuogpt-ii, one-stage training for medical adaption of llms . Preprint, arXiv:2311.09774

arXiv 2023

[5] [5]

Daixuan Cheng, Shaohan Huang, and Furu Wei. 2024. https://openreview.net/forum?id=y886UXPEZ0 Adapting large language models via reading comprehension . In The Twelfth International Conference on Learning Representations

work page 2024

[6] [6]

Yiming Cui, Ziqing Yang, and Xin Yao. 2023. https://arxiv.org/abs/2304.08177 Efficient and effective text encoding for chinese llama and alpaca . arXiv preprint arXiv:2304.08177

arXiv 2023

[7] [7]

Kia Dashtipour, Soujanya Poria, Amir Hussain, Erik Cambria, Ahmad Y. A. Hawalah, Alexander F. Gelbukh, and Qiang Zhou. 2016. https://doi.org/10.1007/S12559-016-9415-7 Multilingual sentiment analysis: State of the art and independent comparison of techniques . Cogn. Comput., 8(4):757--771

work page doi:10.1007/s12559-016-9415-7 2016

[8] [8]

MohammadReza Davari and Eugene Belilovsky. 2024. https://doi.org/10.1007/978-3-031-73226-3\_16 Model breadcrumbs: Scaling multi-task model merging with sparse masks . In Computer Vision - ECCV 2024 - 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part LXXV , volume 15133 of Lecture Notes in Computer Science, pages 270--...

doi:10.1007/978-3-031-73226-3 2024