{"id":"ee6d69f2-13d5-4243-8f71-6ef2016ce411","arxiv_id":"2411.18729","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"AWD improves multi-task model merging by optimizing task vectors toward orthogonality through a shared redundant-vector subtraction.","lead":"This paper proposes Adaptive Weight Disentanglement (AWD), a post-hoc method that subtracts a learned shared vector from task-specific weight vectors to make them more orthogonal before merging. It reports accuracy gains over existing merging methods on vision and language benchmarks. A smart generalist should read it because model merging is a cheap way to combine fine-tuned AI models without retraining, and reducing task interference is the key open problem in that area.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"First-order Taylor truncation in §3.2 drops second-order terms of the same order as the retained term; orthogonality does not imply zero merging gap even in a simple quadratic loss.","rationale":"The central claim rests on the derivation that orthogonal task vectors make the merging gap Gi zero (Eq. 10). That derivation is invalid because the first-order Taylor expansion of the difference of two nearby losses is not dominated by the retained term: the cross second-order term is the same order in the perturbation, and at λi = 1 it cancels the retained term. The quadratic counterexample shows the exact gap is positive even with orthogonal task vectors, so the theoretical conclusion is false under standard smooth losses, independently of the gradient-substitution assumption (Eq. 8) that the reader flagged. This does not refute the empirical value of AWD, which shows consistent accuracy gains and robust behavior across architectures and task counts; those results stand as evidence that the method works. But the paper's headline contribution—'theoretically proving that task vectors should be orthogonal'—is not supported, and the proof needs either correction (e.g., keeping second-order terms and identifying a different sufficient condition) or re-framing as a heuristic motivated by empirical cosine-similarity correlations. The reader's conditional verdict remains appropriate, but the requested revision should explicitly address the second-order truncation, not only the gradient approximation.","tokens_in":22514,"tokens_out":13218,"duration_ms":116199,"concrete_test":"Analytically verify the quadratic counterexample: set Θ = 0, task vectors τ1 = e1 and τ2 = e2 (orthogonal), λ1 = λ2 = 1, and losses L1(θ) = ½‖θ − e1‖², L2(θ) = ½‖θ − e2‖². Compute G1 = L1(Θ + τ1 + τ2) − L1(Θ + τ1) exactly; if G1 = ½ > 0, the claim in Eq. 10 is false and the first-order truncation is invalid. For an empirical check, fine-tune a small MLP from a shared initialization on two near-orthogonal tasks, verify the task vectors are near-orthogonal, merge with Task Arithmetic, and measure the per-task loss gap; a nonzero gap shows orthogonality alone does not satisfy Task Consistency.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The theoretical derivation in §3.2 expands Gi = Li(Θ + Σλjτj) − Li(Θ + λiτi) to first order around Θ and drops all second-order terms (Eq. 5–7). This truncation is not valid: the cross term λiτi^⊤Hi(Σ_{j≠i}λjτj) is first-order in the other task vectors, exactly the same order as the retained term Σ_{j≠i}⟨∇Li(Θ), λjτj⟩. Near a task optimum, ∇Li(Θ) ≈ −Hiτi, so at λi = 1 the retained first-order term and the cross term cancel, leaving Gi ≈ ½(Σ_{j≠i}λjτj)^⊤Hi(Σ_{j≠i}λjτj), which is positive and does not vanish when task vectors are orthogonal. Concrete counterexample: take Θ = 0, K = 2, τ1 = e1 ⊥ τ2 = e2, λ1 = λ2 = 1, and Li(θ) = ½‖θ − ei‖². Then G1 = L1(e1 + e2) − L1(e1) = ½‖e2‖² > 0, contradicting Eq. 10's claim that cos(τ1, τ2) = 0 ⇒ G1 = 0. Thus orthogonality is not sufficient for Task Consistency (Eq. 2); the proof's conclusion is an artifact of the truncation, independent of the additional gradient-substitution assumption in Eq. 8.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Adaptive Weight Disentanglement (AWD), a post-hoc model-merging method that subtracts a single learned redundant vector δ from every task vector τ_i, producing disentangled task vectors that are more mutually orthogonal while preserving per-task performance. The paper first introduces a Task Consistency Property (the loss of the merged model on task i should equal the loss when only task vector i is added) and claims a proof that orthogonality of task vectors is sufficient for this property. AWD then minimizes the average absolute cosine similarity of the disentangled vectors plus an L2 norm penalty on δ. Experiments cover eight vision tasks (ViT-B/16, ViT-B/32, ViT-L/14) and eight GLUE tasks (RoBERTa-Base/Large), with AWD integrated into Task Arithmetic and AdaMerging, plus robustness, loss-landscape, and ablative analyses. Code is publicly available.","tokens_in":2096,"tokens_out":2028,"duration_ms":209795,"significance":"The empirical contribution is substantial and well executed: AWD consistently improves over Task Arithmetic, Ties-Merging, Consensus Merging, DARE, and AdaMerging across architectures and modalities, with reported gains of up to 2.8% (ViT-B/32) over Ties-Merging, and the computational cost is modest (2-8 minutes per task set, Table 5). The reversed-orthogonality control in Tables 6-7 of the supplementary is a genuine falsifiable test: preserving individual-task reconstruction while actively increasing cosine similarity collapses merged accuracy to roughly 32-56%, which strongly supports orthogonality as an empirically active mechanism rather than a tautological artifact. The paper also ships code and reports hyperparameter selection on validation sets. If the theoretical claim in Section 3.2 were correct, the paper would provide a principled sufficient condition for interference-free merging; as it stands, however, that proof is unsound (see major comments), so the paper's durable contribution is an effective and cheap heuristic with consistent empirical support, not an established theorem.","major_comments":[{"comment":"The first-order Taylor truncation is inconsistent: the quadratic expansion of Li(Θ + Σ_j λj τj) contains cross terms (λi τi)⊤ Hi (Σ_{j≠i} λj τj) that are of exactly the same order in the task-vector magnitudes as the retained term Σ_{j≠i} ⟨∇Li(Θ), λj τj⟩ when the gradient is approximated by ∇Li(Θ) ≈ −Hi τi near a task optimum. Dropping one and keeping the other is not justified by the smallness of τ. The claim in Eq. (10) is false even in a quadratic model satisfying all the paper's assumptions. Take Θ=0, K=2, L1(θ)=½||θ−e1||², L2(θ)=½||θ−e2||², τ1=e1, τ2=e2, λ1=λ2=1. Then cos(τ1,τ2)=0 and the gradient substitution in Eq. (8) holds with k1=−1 (since ∇L1(0)=−τ1), yet G1 = L1(e1+e2) − L1(e1) = ½ > 0, contradicting Eq. (10). The first-order expansion gives G1 ≈ 0 only through an exact cancellation of the retained term with the dropped cross term; the true second-order gap ½(e2)⊤ H1 (e2) is positive and does not vanish when the task vectors are orthogonal. Since the paper's λ search range is {0.3, ..., 1.0}, this regime is inside the method's operating range. The implication 'orthogonality ⇒ zero merging gap' must be removed or replaced by a correct second-order analysis, and the abstract's claim of a theoretical proof must be qualified accordingly.","section":"§3.2, Eqs. (5)-(10); Supplementary §6.2, Eqs. (26)-(37)"},{"comment":"The identification ∇Θ Li(Θ) ≈ ki τi with ki < 0 is load-bearing and unsupported. The task vector τi = Θ⋆_i − Θ is the total weight displacement along the fine-tuning trajectory; under gradient descent it equals a sum of gradients evaluated at many iterates, not the gradient at the single point Θ. A scalar proportionality between the gradient at Θ and the total displacement is a strong structural assumption, and it is the only bridge from the merging gap to cosine similarity in Eqs. (9)-(10). The paper should either validate it empirically (e.g., measure cos(∇Li(Θ), τi) on a subset of the eight vision tasks, which the pilot-experiment infrastructure permits) or present Eq. (8) explicitly as a heuristic. As written, the derivation converts an unverified approximation into a conclusion stated without caveat in the abstract.","section":"§3.2, Eq. (8)"},{"comment":"Given the two issues above, the statements 'theoretically proving that task vectors employed in model merging should be orthogonal' (Abstract) and 'we derive that the task vectors corresponding to different tasks should be mutually orthogonal' (Contribution 1) overstate what the manuscript establishes. What the paper actually demonstrates is (i) an empirical regularizer that lowers pairwise cosine similarity while preserving individual task performance, and (ii) a consistent, controlled correlation between reduced cosine similarity and improved merged accuracy (Fig. 6 and Tables 6-7 of the supplementary). The authors should reframe Section 3.2 as a motivating analysis with explicitly stated assumptions (for instance, a diagonal-Hessian or NTK-linearized model along the lines of [21]), or supply a genuine second-order derivation; the suggested sufficient condition in such an analysis is likely to be stronger than pairwise task-vector orthogonality (it should also involve the Hessians Hi). The empirical claims do not depend on the proof, so this revision is feasible within the paper's scope.","section":"Abstract and §1, Contributions"}],"minor_comments":[{"comment":"The sentence 'Property 1 and 23 are characteristics jointly manifested...' is garbled and should read 'Properties 1 and 2'; the cross-references between the main text and the supplementary (Eq. 23 as a restatement of Property 2, and the duplicated numbering of Eqs. 15-37) should be unified.","section":"§3.1-§3.2"},{"comment":"The hyperparameter N (solution steps) is implied to be 1000 only by the caption of Table 5, and the learning rate β is not reported anywhere; both should be stated in Section 7.3. Also, 'we introduce an norm constraint' should read 'an L2 norm constraint'.","section":"§3.3, §7.3, Table 5"},{"comment":"The paper should state explicitly whether the main results in Tables 1-3 use the full task-vector AWD or the linear-layer-only variant described in the pilot experiment; the text suggests the simplified variant is used for transformers, but the main tables and Section 3.3 are written as if the full vector is optimized.","section":"§4.2-§4.3"},{"comment":"The Replacement Gap bound should be stated with an absolute value: from Eq. (21), the meaningful statement is |Ri| ≤ |∇Li(Θ)| ||δ||, not Ri ≤ |∇Li(Θ)| ||δ||; as written the inequality is either signed-inconsistent or trivial.","section":"Supplementary §6.1, Eqs. (15)-(22)"},{"comment":"Entries [27] and [28] are the same paper (Tang et al., ICLR 2024) and should be merged; several equations in the main text render with garbled summation symbols (e.g., Eqs. (5)-(7)), which should be corrected in the camera-ready version.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The empirical study is strong, reproducible in spirit (code provided), and the reversed-orthogonality control is exactly the kind of falsifiable test that separates a real phenomenon from a tautology. The blocking issue is confined to Section 3.2: the proof of the orthogonality sufficient condition is invalid, and the counterexample in major comment 1 is a minimal within-model refutation. I do not think rejection is warranted because the method's value does not depend on the proof; however, publication as-is would put a false theorem on record in the abstract. A revision that either supplies a correct second-order statement or explicitly downgrades the theory to a motivating heuristic would make the paper publishable. I would also ask the authors to clarify which variant (full vs linear-layer-only) produced Tables 1-3, since that affects reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the empirical contribution is real and worth knowing; the proof that motivates it is not. AWD is a cheap post-hoc step that subtracts a learned shared vector from all task vectors, pushing them toward orthogonality, and it consistently improves merging accuracy on ViT-B/16/B/32/L/14 and RoBERTa. The gains over Ties-Merging are around 1.5–2.8 points, and the method also improves AdaMerging. That pattern is strong enough to make me want to try it.\n\nWhat's new: the specific formulation—one shared redundant δ, optimized by maximizing pairwise cosine orthogonality under a norm penalty—is, as far as I know, not in prior merging work. The task consistency property is a reasonable relaxation of the task arithmetic property, and the paper's empirical ablation (reversing the objective destroys performance, reducing cosine similarity tracks with accuracy) supports the *heuristic* that orthogonality helps.\n\nWhere it falls apart is Section 3.2. The derivation justifies orthogonality by first-order Taylor expansion and then replaces the gradient with a scaled task vector. The stress-test example is right: in a quadratic loss with orthogonal task vectors, the merging gap does not vanish. The cross term λ_i τ_i^T H_i Σ_{j≠i} λ_j τ_j is the same order as the retained term, and they cancel near a task optimum. So the statement 'cos(τ_i, τ_j)=0 ⇒ G_i=0' is false even in the simple case. The theory should be reframed as a heuristic or a sufficient condition under stronger assumptions. As written, the claim that task vectors 'should be orthogonal' is not proven.\n\nOther soft spots are minor: no error bars on the main tables, a duplicated citation [27]/[28], and the gradient-substitution constant k_i is unspecified. The experiments are extensive and the code is available, so the empirical claim is checkable.\n\nBottom line: this paper is for anyone working on model merging who wants a cheap, plug-in improvement. It deserves serious review, but the authors need to rewrite the theory section or moderate the claim. I would not block acceptance on the empirical side, but I would not accept the theory as is.","headline":"A useful merging post-processing trick with a stronger empirical than theoretical story.","tokens_in":23380,"tokens_out":2906,"would_cite":true,"duration_ms":39805,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Task vectors should be mutually orthogonal to minimize interference in model merging, the paper argues; its one-vector fix subtracts a learned redundant vector from every task vector to get there, while preserving per-task accuracy.","keywords":["model merging","task arithmetic","task interference","weight disentanglement","orthogonal task vectors","Task Consistency Property","redundant vector","multi-task learning"],"falsifier":"Estimate the true task gradients $\\nabla_{\\Theta} L_i(\\Theta)$ on validation data and compute $\\cos(\\nabla_{\\Theta} L_i(\\Theta), \\tau_i)$ for each task; values far from $\\pm 1$ would show the scalar-multiple substitution is not the operative mechanism. The decisive test is to pick or construct two tasks with exactly orthogonal task vectors ($\\cos(\\tau_1, \\tau_2) = 0$) whose loss gradients at $\\Theta$ are strongly correlated: the paper's equation then predicts a zero merging gap, so measurable accuracy loss on either task after merging would falsify the claim that orthogonality is sufficient.","tokens_in":22285,"feed_emoji":"🧩","tokens_out":19089,"duration_ms":140203,"temperature":0.7,"pith_summary":"The paper tackles a practical obstacle in weight-space model merging: when the fine-tuned weight differences, or task vectors, of several tasks are added to one pretrained model, the vectors interfere and the merged model underperforms the individual fine-tuned models. Its central claim is that this interference is minimized when task vectors are mutually orthogonal, a condition derived by expanding each task's loss to first order around the pretrained weights and showing the merging gap is a weighted sum of pairwise inner products of task vectors. From that geometric condition it builds a cheap post-hoc fix, Adaptive Weight Disentanglement (AWD): learn one shared redundant vector, subtract it from every task vector, and merge the cleaned vectors. The cleaned vectors show steeply reduced pairwise cosine similarity, keep individually reconstructed models at full accuracy, and improve merged accuracy over methods that rely on pruning or sign-resolution heuristics, on eight-task CLIP vision benchmarks and on GLUE with RoBERTa.","feed_headline":"Making task vectors orthogonal lifts merged-model accuracy","feed_subtitle":"A single learned vector, subtracted from every task vector, cuts interference and beats Ties-Merging and AdaMerging.","key_machinery":"The load-bearing identity is the first-order merging-gap approximation $G_i \\approx k_i \\sum_{j \\neq i} \\lambda_j \\langle \\tau_i, \\tau_j \\rangle$ (Eq. 9 in the paper), obtained by Taylor-expanding each task's loss around the pretrained weights $\\Theta$ and substituting the gradient with $\\nabla_{\\Theta} L_i(\\Theta) \\approx k_i \\tau_i$, $k_i < 0$. It converts 'no interference' into a geometric condition: pairwise orthogonal task vectors yield a zero merging gap, making orthogonality a sufficient condition for the Task Consistency Property. The mechanism that realizes this condition is a single trainable redundant vector $\\delta$, shared across all tasks and initialized at zero; subtracting it produces the disentangled vectors $\\hat{\\tau}_i = \\tau_i - \\delta$. The objective $L = L_O + \\alpha L_R$ balances two terms: $L_O$ penalizes the mean absolute cosine similarity between every pair of disentangled vectors, and $L_R = \\|\\delta\\|$ keeps the perturbation small, with the appendix bounding the per-task replacement gap by $\\|\\nabla_{\\Theta} L_i(\\Theta)\\| \\cdot \\|\\delta\\|$. Because only the weight vectors themselves enter the optimization of $\\delta$, the procedure costs about two minutes on ViT-B/32 and eight minutes on ViT-L/14, and the outputs drop directly into existing merging recipes.","core_discovery":"The paper's central assertion is that task vectors used in model merging should be orthogonal to minimize interference among tasks. It formalizes the goal as a Task Consistency Property — the merged model should match, on each task, the performance obtained when only that task's own vector is added — and derives that a first-order Taylor expansion makes the per-task merging gap approximately $G_i \\approx k_i \\sum_{j \\neq i} \\lambda_j \\langle \\tau_i, \\tau_j \\rangle$, after replacing the unobservable gradient $\\nabla_{\\Theta} L_i(\\Theta)$ with a negative scalar multiple of $\\tau_i$. Mutual orthogonality of the task vectors therefore drives every gap to zero, which the paper presents as a sufficient condition, not a necessary one. On the practical side, AWD learns a single redundant vector $\\delta$, initialized at zero, and exports disentangled vectors $\\hat{\\tau}_i = \\tau_i - \\delta$ by minimizing $L = L_O + \\alpha L_R$, where $L_O$ is the mean absolute pairwise cosine similarity of the $\\hat{\\tau}_i$ and $L_R = \\|\\delta\\|$ protects per-task performance; the appendix bounds the per-task replacement gap by $\\|\\nabla_{\\Theta} L_i(\\Theta)\\| \\cdot \\|\\delta\\|$. Plugged into Task Arithmetic or AdaMerging, the disentangled vectors raise average accuracy by up to 6.1 points over plain Task Arithmetic on the eight-task vision benchmark, and the paper's control experiment that deliberately increases cosine similarity collapses accuracy, which it reads as causal evidence for the orthogonality mechanism.","pith_inferences":["The derivation establishes a sufficient condition, not a characterization: non-orthogonal vectors can still interfere little if the unknown negative scalars $k_i$ happen to offset the inner products, so the paper leaves open how far orthogonality is also necessary.","The gradient-substitution step is directly checkable: on any task where fine-tuning data is available, measuring $\\cos(\\nabla_{\\Theta} L_i(\\Theta), \\tau_i)$ would show whether orthogonality is the operative mechanism or a proxy for a blunter effect such as shrinking the norm or variance of the summed vectors.","The single shared $\\delta$ presumes that all tasks share a common direction of redundancy; for heterogeneous task families a natural extension the paper does not explore is per-group or per-task redundant vectors, and a comparison would bound where the shared-vector assumption starts to cost accuracy.","The loss-landscape finding — a wider low-loss basin after disentanglement — invites a testable tie-in with flat-minima intuitions: if merged-model performance tracks the flatness of the joint loss basin, then entanglement measures other than pairwise cosine similarity might predict merging success equally well."],"forward_implications":["Any existing task-vector merger can be upgraded by a preprocessing pass that needs no task training data: AWD computes $\\delta$ from the task vectors alone, so it applies where the fine-tuning data is unavailable or privacy-restricted.","The benefit of orthogonalization grows with the number of merged tasks: on ViT-B/32 the gain over plain Task Arithmetic widens from 0.6 points with two tasks to 6.1 points with eight, matching the view that more tasks mean more pairwise interference to remove.","Orthogonality, not parameter pruning, is the active ingredient: the paper's control experiment that reverses the objective and increases cosine similarity collapses average accuracy by 37.3% on ViT-B/32 and 29.0% on ViT-L/14 even though the reconstructed single-task models remain accurate.","The approach transfers across model families — ViT-B/16, ViT-B/32, and ViT-L/14 for vision and RoBERTa-base/large on GLUE — and composes with both training-free and test-time-adaptation merging methods rather than replacing them."],"supporting_citations":[{"why":"Defines task vectors and Task Arithmetic, the base operation that AWD modifies and the source of the CLIP vision checkpoints and benchmark.","marker":"[10]"},{"why":"Introduces the Task Arithmetic Property and its tangent-space analysis, the framework that AWD relaxes into the Task Consistency Property.","marker":"[21]"},{"why":"Ties-Merging is the main training-free baseline that AWD+TA outperforms, and its redundancy argument motivates removing a shared redundant vector.","marker":"[39]"},{"why":"DARE motivates pruning redundant weights and justifies the first-order Taylor expansion by the small norms of task vectors.","marker":"[44]"},{"why":"AdaMerging is the test-time-adaptation baseline that AWD integrates with and outperforms.","marker":"[42]"},{"why":"Supplies the RoBERTa checkpoints and the normalized-score metric used for the language-model generalization experiments.","marker":"[18]"}],"fun_headline_variants":["Orthogonal task vectors boost merged-model performance","Adaptive Weight Disentanglement cuts merging interference","AWD: Make task vectors orthogonal for better merging","Subtract one vector to merge models with less interference"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing step is substituting each task's loss gradient at the pretrained weights with a negative scalar multiple of its task vector ($\\nabla_{\\Theta} L_i(\\Theta) \\approx k_i \\tau_i$, $k_i < 0$), together with a first-order Taylor expansion of the loss; if the true gradient is not aligned with the task vector, orthogonal task vectors need not eliminate interference, and the paper's guarantee is void.","fun_headline_variants_meta":{"raw":{"variants":["Orthogonal task vectors boost merged-model performance","Adaptive Weight Disentanglement cuts merging interference","AWD: Make task vectors orthogonal for better merging","Subtract one vector to merge models with less interference"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000182,"raw_usage":{"total_tokens":1388,"prompt_tokens":1100,"completion_tokens":288,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":716,"completion_tokens_details":{"reasoning_tokens":228}},"tokens_in":716,"tokens_out":288,"duration_ms":3600,"temperature":1.0,"reasoning_tokens":228,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:56:07.059457+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Estimate the true task gradients $\\nabla_{\\Theta} L_i(\\Theta)$ on validation data and compute $\\cos(\\nabla_{\\Theta} L_i(\\Theta), \\tau_i)$ for each task; values far from $\\pm 1$ would show the scalar-multiple substitution is not the operative mechanism. The decisive test is to pick or construct two tasks with exactly orthogonal task vectors ($\\cos(\\tau_1, \\tau_2) = 0$) whose loss gradients at $\\Theta$ are strongly correlated: the paper's equation then predicts a zero merging gap, so measurable accuracy loss on either task after merging would falsify the claim that orthogonality is sufficient.","supporting_citations":[{"cited_title":"Adamerging: Adap- tive model merging for multi-task learning","cited_arxiv_id":null,"evidence_quote":"AdaMerging is the test-time-adaptation baseline that AWD integrates with and outperforms."},{"cited_title":"Editing models with task arithmetic","cited_arxiv_id":null,"evidence_quote":"Defines task vectors and Task Arithmetic, the base operation that AWD modifies and the source of the CLIP vision checkpoints and benchmark."},{"cited_title":"Task arithmetic in the tangent space: Improved editing of pre-trained models","cited_arxiv_id":null,"evidence_quote":"Introduces the Task Arithmetic Property and its tangent-space analysis, the framework that AWD relaxes into the Task Consistency Property."},{"cited_title":"Ties-merging: resolving interfer- ence when merging models","cited_arxiv_id":null,"evidence_quote":"Ties-Merging is the main training-free baseline that AWD+TA outperforms, and its redundancy argument motivates removing a shared redundant vector."},{"cited_title":"Language models are super mario: Absorbing abilities from homologous models as a free lunch","cited_arxiv_id":null,"evidence_quote":"DARE motivates pruning redundant weights and justifies the first-order Taylor expansion by the small norms of task vectors."},{"cited_title":"Twin-merging: Dynamic integration of modular expertise in model merging","cited_arxiv_id":null,"evidence_quote":"Supplies the RoBERTa checkpoints and the normalized-score metric used for the language-model generalization experiments."}],"review_version":1}