{"id":"4801c355-330f-4000-9310-b29933a5bc4a","arxiv_id":"2412.04144","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Optimizing linear merge weights on 16 104B generalist checkpoints via CMA-ES reduces task tradeoffs and beats individual models and simple merge baselines.","lead":"The paper shows that search-optimized averaging of 16 large language model checkpoints (104B parameters each) can reduce tradeoffs between skills such as code generation and instruction following. The result is significant because it suggests discarded, suboptimal checkpoints can be cheaply recycled into better models without retraining.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline held-in gains (Tables 1 and 3) rest on a single stochastic CMA-ES trajectory with no repeated-seed variance estimates, so the reported 1-2 point improvements over baselines may not be distinguishable from search noise.","rationale":"The reader's weakest_assumption concerns parameter-space interpolatability (permutation symmetries, commensurability of training histories), which is a real precondition for any linear merge. However, the paper's checkpoints come from a single 104B development pipeline with shared architecture and presumably shared initialization, and the paper demonstrates empirically that linear interpolation yields sensible models. That concern, while legitimate, is partly mitigated by the setup and is not the most immediately decisive threat to the paper's central claim. The more load-bearing gap is the unreplicated stochastic search: all headline numbers come from one CMA-ES run per task combination, with no seed sensitivity analysis. For a paper whose contribution is a quantitative comparison of search-optimized merges against baselines, run-to-run variance is the first-order threat to validity. The reported 1-2 point gains could plausibly be within the spread of the optimizer itself, especially given the modest population size and the noisy evaluation setting. The reader's verdict of CONDITIONAL already captures this concern in part (absence of repeated runs and error bars), but the reader's stated weakest assumption is elsewhere. I therefore mark partial agreement. The concrete test—multiple seeds with reported distributions—would settle the concern directly. If the spread is small, the central claim holds and ACCEPT would be justified; if large, the specific Pareto-optimality and superiority claims should be softened to a qualitative demonstration. The absence of code release also hampers independent audit of the exact CMA-ES configuration (e.g., handling of the best-so-far solution), but the repeated-seed experiment is the single most decisive check.","tokens_in":16261,"tokens_out":2799,"duration_ms":32467,"concrete_test":"Re-run the MBPP-IFEval, MBPP-MUSR, MMLUPro-IFEval, and MBPP-IFEval-GSM8K searches at least 5 times with different CMA-ES random seeds, keeping all settings otherwise fixed, and report the distribution of held-in fitness and held-out scores over the final merges. If the inter-run standard deviation or interquartile range of held-in fitness exceeds roughly 30% of the reported gap between the search-optimized merge and the best baseline (e.g., 30% of 2.0 points for MBPP-IFEval), the quantitative head-to-head claims should be downgraded to conditional. Also report whether the Table 1/Table 3 rows are the best trajectory, the median trajectory, or the final mean of a single run.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central quantitative claim is that CMA-ES-optimized linear interpolation of 16 heterogeneous 104B checkpoints (θ_mrg = Σ α_i θ_i, §3.2) Pareto-improves held-in task tradeoffs without sacrificing held-out performance. This claim is load-bearing on the stability of the search, but the evidence is a single CMA-ES trajectory per task combination: 50 iterations with population size 4+3ln16 ≈ 12, and no repeated runs with different random seeds, no error bars on the final merges, and no explicit statement of whether the reported rows are the best-ever solution, the final mean, or a post-hoc selected trajectory. The observed gains—MBPP-IFEval 68.0 vs 66.0 uniform soup (Table 1) and MBPP-IFEval-GSM8K 72.1 vs 70.7 merge-best (Table 3)—are modest, and the fitness function (unweighted macro-average, §3.2) is evaluated on one greedy-decoding pass. If a second run with a different seed produces a weighting vector whose held-in average differs by more than roughly the reported 1-2 point margins, the specific Pareto-optimality and baseline-comparison claims are not robust. The paper does offer partial supporting evidence: fitness-vs-iteration plots (Fig. 9) show a positive trend, and the N=2/4/8/16 checkpoint ablation (Fig. 7) is consistent with the dense-merging story. But neither substitutes for repeated-search variance estimates. A related secondary concern is that the same 16 checkpoints are used both to search and to report the qualitative 'most models contribute' finding in §5.3, so the generality of that finding is untested on an independent pool. However, the primary, decisive gap is the absence of seed-sensitivity evidence for the headline numbers.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies whether linearly merging 16 large (104B) checkpoints harvested from different training runs can reduce performance tradeoffs across capabilities such as code generation, instruction following, math, and multi-step reasoning. The authors formulate the merge as a convex combination of model parameters, θ_mrg = Σ α_i θ_i, and use CMA-ES to optimize the weights α_i against a macro-average fitness over held-in tasks. They report that search-optimized merges outperform uniform soup and merge-best baselines on held-in tasks in two-task and three-task settings, maintain or improve performance on held-out tasks, and that the best merges use almost all checkpoints with nonzero weights. The paper also provides a compute-cost estimate suggesting that the merge search costs about 10% of a single SFT+PO training stage.","tokens_in":16658,"tokens_out":6051,"duration_ms":64175,"significance":"If the reported results are robust, this is a useful and timely contribution: it scales model merging to ~100B parameters, moves beyond the expert-merging setup to heterogeneous generalist checkpoints, and provides both a practical training-free recipe for recycling suboptimal models and an interesting qualitative finding that low-performing checkpoints can contribute to good merges. The inclusion of held-out tasks and the analysis of the learned weight distributions are particular strengths. The main caveat is that the evidence rests on single stochastic optimization runs with no variance estimates, and the search protocol is internally inconsistent in ways that affect both reproducibility and the computational-cost claim.","major_comments":[{"comment":"The search protocol is internally inconsistent. Section 4 states that the population size is set to 4+3lnN (≈12 for N=16), and Algorithm 1 samples λ offspring per iteration and evaluates each; but the text then says \"At each iteration, CMA-ES proposes a single weighting vector\" and the compute-cost calculation in App. D uses only 50 evaluations per task (50 × MBPP cost + 50 × IFEval cost). If the standard CMA-ES in Algorithm 1 is used, the true number of fitness evaluations is roughly 50×λ, which would increase the reported computational cost by an order of magnitude and undermine the \"10% of training compute\" claim. If only one candidate per iteration is evaluated, the algorithm is not the CMA-ES described in Algorithm 1. Please clarify the exact evaluation protocol and align Section 4, Algorithm 1, Fig. 9, and App. D.","section":"§4, Algorithm 1, App. D"},{"comment":"Each reported search-optimized merge is the result of a single CMA-ES run with no repeated seeds, no variance estimates, and no explicit statement of whether the reported numbers are the best-ever solution, the final mean, or a post-hoc selected trajectory. The headline margins are modest (e.g., 68.0 vs. 66.0 for MBPP-IFEval in Table 1; 72.1 vs. 70.7 for MBPP-IFEval-GSM8K in Table 3), and the fitness is evaluated on one greedy decoding pass. Without repeated runs, the claim that search-optimized merging \"always yields\" a Pareto-frontier model and reliably outperforms baselines is not supported. Please provide multiple seeds (or at least a clear selection rule and a sensitivity analysis), report mean/std or confidence intervals, and if possible compare the optimization signal to random search or a small number of independent restarts.","section":"§5.1, Tables 1 and 3"},{"comment":"The checkpoint-count ablation (N=2,4,8,16) and the \"most models contribute\" analysis are based on what appear to be single runs per configuration, with no repeated seeds or variance estimates. The centroids in Figs. 7-8 may therefore shift substantially under different random seeds, and the qualitative conclusion that dense merges are optimal is drawn from one or a few top-5 solution sets. Additionally, the captions of Fig. 7 and Fig. 8 both say the experiments optimize MBPP-MUSR, while the text indicates that Fig. 8 should show MBPP-IFEval; please correct this and clarify which configuration each figure reports.","section":"§5.3, Figs. 7 and 8"},{"comment":"The method assumes that a direct weighted average of 16 checkpoints from different SFT/PO runs, with different objectives, data mixtures, and possibly different base models, is a well-defined operation in parameter space. This requires that the checkpoints are commensurable and that interpolation is meaningful, but the paper provides no evidence on this point (e.g., whether the models share the same architecture and initialization, or whether permutation symmetries were addressed). If the checkpoints are not interpolatable, the optimized weights have no reliable effect. Please add a discussion or a simple control (e.g., showing that random weighted averages of the same checkpoints produce sensible models, and that the optimized solutions are not artifacts of ill-conditioned interpolation).","section":"§3.2"}],"minor_comments":[{"comment":"The text and caption say \"Models 1-8 are the result of supervised finetuning runs, while 8-16 from preference optimization,\" but the enumeration in Table 2 shows SFT models 1-8 and PO models 9-16; the second range should be 9-16.","section":"§4 and Fig. 2 caption"},{"comment":"The phrase \"Pareto-optimal\" is used without qualification. Since the search explores only a finite set of CMA-ES proposals, the claim should be stated as Pareto-optimality among the evaluated candidates/checkpoints, not as a global property of the entire weight simplex.","section":"Abstract and §5.1"},{"comment":"The paper states that the 2K MMLU-Pro subset correlates highly with the full dataset but reports no correlation coefficient or error estimate; please provide the quantitative evidence or a reference for this claim.","section":"Footnote 3"},{"comment":"If the CMA-ES population size is indeed λ≈12 as stated in Section 4, the fitness-vs-iteration plot should show multiple points per iteration; the current plot appears to show one point per iteration, which is inconsistent with the described algorithm and should be aligned with the clarified search protocol.","section":"Fig. 9"},{"comment":"The Spearman correlations in Fig. 4 are computed over only 16 checkpoints, and no significance levels or confidence intervals are reported; adding these would strengthen the tradeoff analysis.","section":"App. C"}],"recommendation":"major_revision","confidential_remarks":"The most pressing issue is the inconsistency between the stated CMA-ES population size and the reported number of fitness evaluations, as it directly affects the computational-cost claim and the reproducibility of the core result. Please ask the authors to clarify this before the paper can be considered further."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is worth a look for one reason: it takes evolutionary weight interpolation, previously shown on two 7B experts, and runs it on sixteen 104B generalist checkpoints from different SFT/PO stages. That scale and setup are new, and the descriptive finding that the best merges use nearly all checkpoints, including ones with zero standalone performance on a task, is genuinely interesting. The held-out evaluation on MT-Bench and LBPP is a good check that the search does not simply overfit to the held-in tasks.\n\nThe experiments are clean and the baselines (best single, uniform soup, merge-best) are fair. The compute estimate of about 10% of one SFT+PO stage is useful for practitioners.\n\nThe soft spots are real but not fatal. The main one: every reported search result comes from a single CMA-ES trajectory, 50 iterations, no repeated seeds, no error bars. The headline margins over uniform soup are 0.6 to 2.2 average points. For a stochastic optimizer, that is exactly the size of search noise you would worry about. The fitness-vs-iteration plots show a positive trend, and the N=2/4/8/16 ablation supports the qualitative story, but neither replaces repeated-seed variance estimates. Without them the specific numbers should be treated as suggestive, not robust.\n\nTwo smaller issues. The task pairs were chosen after looking at pairwise correlations, which is post-hoc selection and likely inflates the effect; the authors do not flag this. The 'Pareto-optimal' language is stronger than the evidence: they show the found merge is not dominated by the evaluated baselines and individual checkpoints, not that no better merge exists. Also, no code release, and MMLUPro uses a 2K subset.\n\nOn the commensurability worry: it is reasonable to ask whether linear interpolation of checkpoints from different training histories is meaningful, but the empirical results themselves are the answer here—the merges work, so the assumption is at least pragmatically satisfied.\n\nWho should read this: anyone doing model merging or LLM post-training. It is a solid empirical contribution that needs a revision adding variance estimates and toning down the Pareto claim. I would send it to peer review.\n\nRecommendation: engage with it, but ask for repeated runs with different seeds and error bars before the quantitative gains are accepted.","headline":"A useful empirical study of evolutionary merging at 104B scale, but the headline margins rest on single stochastic runs and need variance estimates before the numbers are trusted.","tokens_in":17235,"tokens_out":2869,"would_cite":true,"duration_ms":29298,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that discarded 100B-scale checkpoints, which individually trade off capabilities, can be recycled by optimizing a linear merge into a Pareto-optimal model without hurting held-out performance.","keywords":["model merging","model soups","Pareto-optimal","task tradeoffs","checkpoint recycling","evolutionary search","CMA-ES","large language models"],"falsifier":"Take the original 16 checkpoints, apply to each an independent random permutation of its hidden units (together with the corresponding weight-matrix permutations so each model still behaves identically), and rerun the same search on the permuted pool; if the optimized merge no longer beats uniform soup on the held-in average, the benefit depends on raw-parameter alignment rather than on the checkpoints' capabilities.","tokens_in":16088,"feed_emoji":"♻️","tokens_out":9981,"duration_ms":89525,"temperature":0.7,"pith_summary":"Model development produces many intermediate checkpoints that trade off one capability against another, and the usual response is to keep the best and discard the rest. This paper asks whether those leftovers can be recycled: it defines the merge as a weighted average of up to sixteen 104B checkpoints and uses an evolutionary search to tune the weights. On two- and three-task tradeoff pairs (code vs. instruction-following, code vs. multistep reasoning, knowledge vs. instruction-following, and the triple code/math/instruction-following), the optimized merge beats uniform averaging, merge-best, and the best single model on the tasks being balanced, and it does not degrade held-out benchmarks. The paper reports that good merges spread weight across almost all checkpoints, including ones that score worst on the target tasks, and that the whole search costs roughly 10% of one training stage's compute.","feed_headline":"Leftover 104B checkpoints merge into a Pareto-optimal model","feed_subtitle":"Search-tuned weight averaging beats uniform and best-model merging on task tradeoffs at about 10% of training cost","key_machinery":"The load-bearing object is linear weight interpolation: a convex combination of the raw parameters of the 16 checkpoints, $\\theta_{\\mathrm{mrg}} = \\sum_{i=1}^{16} \\alpha_i \\theta_i$ with $\\alpha_i \\ge 0$ and $\\sum_i \\alpha_i = 1$. This is the same operation as a \"model soup,\" but here the weights are not fixed by hand: they are searched by CMA-ES, an evolutionary strategy that samples weight vectors from a multivariate normal distribution, evaluates the resulting merged model on the held-in tasks, and adapts its sampling distribution toward higher macro-average fitness. The machinery does two jobs at once: it converts a discrete set of discarded training runs into a continuous family of candidate models, and it turns Pareto-optimality into a single scalar objective (the unweighted average of task scores) that a gradient-free optimizer can drive.","core_discovery":"On its own terms, the paper establishes that linear merging is a viable training-free way to reduce capability tradeoffs at the 100B scale. Given the 16 checkpoints, the optimization searches the simplex of non-negative weights summing to one to maximize the unweighted average of held-in task scores. The resulting models lie on the Pareto frontier over the evaluated candidates and outperform uniform soup and task-best merging on held-in averages (e.g., 68.0 vs 66.0 on MBPP-IFEval, and 72.1 vs 70.7 on the three-task combination), while maintaining comparable or better held-out scores. The paper also shows that the optimal solutions are dense: nearly all checkpoints receive nonzero weight, and a checkpoint's standalone performance does not predict its contribution to a good merge.","pith_inferences":["A natural next experiment is transferring a learned weight vector: use the optimized merge as the initialization for a short continued SFT/PO stage, then test whether the Pareto benefit survives further training, which the paper does not address.","The dense-weight finding hints that each checkpoint contributes a distinct axis in parameter space and that the optimal merge exploits the full span of these directions; this could be tested by measuring how well the merge direction aligns with task-performance gradients.","The recipe could be used outside checkpoint recycling, for example as a fast way to choose a final model during development by exploring the Pareto frontier across many candidate runs before selecting which checkpoint to ship.","The strong dependence on raw parameter alignment means the approach will work only within one model family; extending it to heterogeneous checkpoints with different tokenizers or architectures would require a different merge space, not just a better search."],"forward_implications":["Discarded intermediate checkpoints can be treated as a reusable resource: a frontier-model team can run the search recipe on whatever checkpoints it has and get a better balanced model without additional training.","Because good merges are dense, pruning the checkpoint pool by individual performance would remove inputs that contribute to the optimum; the pool should be kept broad, not filtered by score.","The method gives an upper-bound baseline for more complex merging techniques at 100B scale: if simple linear merging with searched weights is already Pareto-optimal over candidates, more elaborate methods must beat that to justify their extra cost.","The fact that held-out tasks do not degrade suggests the search is not overfitting to the exact evaluation prompts; the merged model generalises the balanced capability to other evaluations in the same family.","At roughly 10% of one SFT+PO stage's compute, this is a cheap way to probe the tradeoff frontier before committing to another training run."],"supporting_citations":[{"why":"Introduces linear weight averaging (\"model soups\"), the merge operation whose weights this paper optimizes.","marker":"Wortsman et al., 2022"},{"why":"Defines CMA-ES, the evolutionary optimizer used to search the weight simplex.","marker":"Hansen & Ostermeier, 2001"},{"why":"Shows evolutionary optimization of merging recipes; the search recipe here extends that idea to 16 much larger checkpoints.","marker":"Akiba et al., 2024"},{"why":"Reports that merging methods converge in performance at large scale, which the paper cites to justify using simple linear merging.","marker":"Yadav et al., 2024b"},{"why":"Introduces GSM8K, one of the held-in tasks used in the fitness function.","marker":"Cobbe et al., 2021"},{"why":"Introduces IFEval, one of the held-in tasks used in the fitness function.","marker":"Zhou et al., 2023"},{"why":"Introduces MMLU-Pro, one of the held-in tasks used in the fitness function.","marker":"Wang et al., 2024"},{"why":"Supplies the FLOPs formulas used to estimate that the merge search costs about 10% of one SFT+PO training stage.","marker":"Kaplan et al., 2020"}],"fun_headline_variants":["Recycled checkpoints yield Pareto-optimal 100B model","Merging 100B leftovers beats best single model","Optimized soup of 16 checkpoints tops Pareto frontier","Dense merging turns suboptimal 100B checkpoints into winner","Search-tuned checkpoint blending beats uniform merging"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the raw parameters of the 16 checkpoints can be averaged meaningfully, which requires a shared architecture and aligned parameter spaces despite different training stages, objectives, and data mixtures.","fun_headline_variants_meta":{"raw":{"variants":["Recycled checkpoints yield Pareto-optimal 100B model","Merging 100B leftovers beats best single model","Optimized soup of 16 checkpoints tops Pareto frontier","Dense merging turns suboptimal 100B checkpoints into winner","Search-tuned checkpoint blending beats uniform merging"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000617,"raw_usage":{"total_tokens":2842,"prompt_tokens":902,"completion_tokens":1940,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":1858}},"tokens_in":518,"tokens_out":1940,"duration_ms":13300,"temperature":1.0,"reasoning_tokens":1858,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:41:56.477152+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the original 16 checkpoints, apply to each an independent random permutation of its hidden units (together with the corresponding weight-matrix permutations so each model still behaves identically), and rerun the same search on the permuted pool; if the optimized merge no longer beats uniform soup on the held-in average, the benefit depends on raw-parameter alignment rather than on the checkpoints' capabilities.","supporting_citations":[],"review_version":1}