REVIEW 13 cited by
TIES-Merging: Resolving Interference When Merging Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Transfer learning - i.e., further fine-tuning a pre-trained model on a downstream task - can confer significant advantages, including improved downstream performance, faster convergence, and better sample efficiency. These advantages have led to a proliferation of task-specific fine-tuned models, which typically can only perform a single task and do not benefit from one another. Recently, model merging techniques have emerged as a solution to combine multiple task-specific models into a single multitask model without performing additional training. However, existing merging methods often ignore the interference between parameters of different models, resulting in large performance drops when merging multiple models. In this paper, we demonstrate that prior merging techniques inadvertently lose valuable information due to two major sources of interference: (a) interference due to redundant parameter values and (b) disagreement on the sign of a given parameter's values across models. To address this, we propose our method, TRIM, ELECT SIGN & MERGE (TIES-Merging), which introduces three novel steps when merging models: (1) resetting parameters that only changed a small amount during fine-tuning, (2) resolving sign conflicts, and (3) merging only the parameters that are in alignment with the final agreed-upon sign. We find that TIES-Merging outperforms several existing methods in diverse settings covering a range of modalities, domains, number of tasks, model sizes, architectures, and fine-tuning settings. We further analyze the impact of different types of interference on model parameters, and highlight the importance of resolving sign interference. Our code is available at https://github.com/prateeky2806/ties-merging
Forward citations
Cited by 13 Pith papers
-
Emergent Misalignment Recruits a Pre-existing Persona Subspace
Fine-tuning on narrow bad data recruits a low-rank persona subspace already present in a frozen instruction-tuned model; holding that subspace out of activations prevents broad misalignment, and injecting it into the ...
-
Overcoming the Communication-Performance Tradeoff in LLM Pretraining
SparseLoCo combines error feedback with Top-k sparsification and 2-bit quantization to send 1-3% of the pseudo-gradient during LLM pre-training while matching or beating DiLoCo's dense updates.
-
modelDNA: Calibrated Lineage Verification and Merge Decomposition from Sampled Weight Fingerprints
Sampled weight fingerprints recover LLM parentage with AUROC 1.0 and zero false positives, and recover published mergekit mixture weights without full downloads.
-
Spectral Rewiring for Exploration, Purification, and Model Merging
Subspace-Aligned Rewiring projects RL weight updates onto the base model’s SVD basis, retaining a compact rewiring matrix that preserves reasoning and improves exploration and multi-domain merging.
-
Athena-Brain Technical Report: An Efficient Robot Brain for General Intelligence and Embodied Interaction
An 8B LLM post-trained with SFT, RL, embodied-expert training, and model merging reaches high in-domain embodied-task success with very short responses.
-
Guard Vector: Beyond English LLM Guardrails with Task-Vector Composition and Streaming-Aware Prefix SFT
A task vector from an English guard model transfers safety classification to Korean, Chinese, and Japanese models, and a prefix-SFT variant maintains accuracy under streaming with a single-token classifier.
-
DivMerge: A divergence-based model merging method for multi-tasking
DivMerge learns task-arithmetic merging weights by minimizing Jensen-Shannon divergence between each specialist model and the merged model, improving multi-task performance and scalability.
-
Competition and Attraction Improve Model Fusion
M2N2 evolves merging boundaries, uses resource competition for diversity and attraction-based pairing, achieving from-scratch evolution and specialized model fusion.
-
Rethinking Heterogeneous LLM Merging: A Weighted Model Averaging Perspective
After truncating or expanding checkpoints to a shared shape, small-ratio weight averaging slightly improves average benchmark scores over strong Qwen sources, but headline gains are inflated by per-task best-ratio selection.
-
First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers
Single task-vector perturbations around a multitask LoRA point are first-order linear to 1e-2, but pairwise update-order sensitivity is set by a per-pair Lie bracket and can appear inside that window.
-
DeltaMerge-LowRes: Composing Language and Task Deltas for Low-Resource Adaptation
A TIES-style merge across language-delta and task-delta axes (cross-axis TIES) beats additive and task-only composition on low-resource summarisation and QA.
-
Exploring Sparse Adapters for Scalable Merging of Parameter Efficient Experts
Sparse adapters trained with max connection sensitivity outperform LoRA and full fine-tuning both alone and after merging 20 task experts, but still lag multitask training on unseen tasks.
-
The Future of Continual Learning in the Era of Foundation Models: Three Key Directions
Continual learning should pivot from weight-update-based methods to continual compositionality and orchestration of foundation models and agents.
Discussion (0). Sign in to comment.