Pith. sign in

REVIEW 3 major objections 4 minor 39 references

Model Fusion via Neuron Transplantation

T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that Neuron Transplantation fuses an ensemble into a single network of member size that, after brief fine-tuning, beats each individual member and matches or exceeds optimal-transport fusion with less compute.

desk verdict New fusion trick with solid OT comparison, but the 'consistently beats members' claim is undercut by a missing fine-tuning control. read the letter →

arxiv 2502.06849 v1 pith:NKWQ4667 submitted 2025-02-07 cs.LG cs.AI

Muhammed Öz , Nicholas Kiefer , Charlotte Debus , Jasmin Hörter , Achim Streit , Markus Götz This is my paper · ORCID
classification cs.LGcs.AI
keywords modelfusionensemblelearningcompressionstructuredpruningneurontransplantationfine-tuningoptimaltransportmagnitude
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Training several copies of the same network and averaging their predictions improves accuracy, but multiplies memory and compute. This paper argues that the same benefit can be concentrated into one network of a member's size by Neuron Transplantation: concatenate the layers of all members, prune away the neurons with the smallest $L^2$ norms down to the original width, and fine-tune briefly. After fine-tuning, the transplanted model is claimed to consistently beat each individual ensemble member on the same architecture and capacity, and to match or exceed optimal-transport fusion while being faster, lighter, and needing fewer fine-tuning steps. The method's premise is that the pruned neurons are the unimportant ones and the surviving neurons from different models are complementary, which is why fusing a model with itself fails.

What carries the argument

The core mechanism is layer-wise concatenation followed by structured magnitude pruning. All non-output layers of the $k$ models are stacked and the output layers are averaged, so the resulting wide model is exactly equivalent to output averaging; then a pruning step removes the neurons with the smallest $L^2$ norm in each layer until only $1/k$ of the neurons remain, restoring the original architecture. The cross-weights connecting the concatenated layers are initialized to zero and learned during fine-tuning, which is what lets the transplanted neurons recombine into a model that outperforms any single member.

What would settle it

Fuse a model with an exact copy of itself: the paper reports that accuracy drops from 83.02% before fusion to 67.63% immediately afterward, recovering only to the single-model level after fine-tuning, which shows the complementarity premise is doing the work. A direct test would measure the overlap of top-$L^2$-norm neurons between two models; when the overlap is high, NT should fail to beat the best member, and when the overlap is low, it should succeed.

Watch

Extended reading notes

Core claim

The central claim is that ensemble performance can survive in a single model by selecting only the largest-norm neurons from each member rather than averaging aligned weights. The experiments show that the initial post-fusion loss is quickly recovered by fine-tuning: on SVHN, two-model NT reaches 84.37% accuracy after 20 fine-tuning epochs, against an ensemble score of 84.61% and a best-single-model score of 82.57%; in broader comparisons, NT reaches the best individual accuracy faster than OT-fused or vanilla-averaged models and then surpasses it. The paper also reports that merging before pruning works best, that fusing more than about eight models gives diminishing returns, and that NT's memory footprint stays flat near 3.8 MB while OT-fusion exceeds 38 GB for the widest tested layers.

Load-bearing premise

The load-bearing premise is that a neuron's importance is captured by the $L^2$ norm of its weights and that the high-norm neurons across the differently trained models are complementary; if the most important neurons are redundant between members, transplantation discards useful information without adding any.

Editorial extensions

If this is right

  • A single model with the same architecture can carry the accuracy of an ensemble after a short fine-tuning, cutting inference memory and time by a factor of $k$.
  • Fusion by NT is substantially cheaper than OT-fusion: roughly flat 3.8 MB memory overhead, no permutation matrices, and wall times under 1.4 seconds at the widest tested width where OT-fusion exceeds 38 GB or runs out of memory.
  • NT can be combined with knowledge distillation for a small additional accuracy boost, making it a practical drop-in replacement for alignment-based fusion.
  • Fusing more than about eight models yields diminishing returns, so practitioners should prefer hierarchical merging over iterative merging for larger ensembles.
  • For ensemble pruning the merging step can be omitted, but for model fusion the recommended order is merge, then prune, then fine-tune.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the $L^2$-norm criterion truly identifies the important neurons, then a cheap diversity check measuring overlap of top-norm neurons between members could predict when NT will work, turning the self-fusion failure into a diagnostic tool.
  • The zero-initialized cross-weights suggest a general recipe for mixing experts without permutation alignment, applicable to transformer layers or federated averaging when client models sit in different loss basins.
  • The width behaviour — smaller widths benefit more, opposite to the loss-barrier literature — hints that selection avoids the barrier by never interpolating weights; a controlled head-to-head on a common benchmark architecture would isolate whether the advantage comes from selection rather than averaging.
  • The paper leaves open whether jointly training the concatenated ensemble before pruning could create a better initialization; a testable extension is to compare NT against per-member prune-and-fine-tune baselines at fixed total compute.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Neuron Transplantation (NT), a weight-space fusion method that merges an ensemble of same-architecture networks into one model of the original size. Non-output layers are concatenated with zero-initialized cross-weights, the output layer is averaged, and structured L2-magnitude pruning removes neurons until the original width is reached; the resulting initialization is then fine-tuned or distilled on the full data. The authors evaluate NT on MLP, LeNet, VGG11, and ResNet18 across MNIST, CIFAR10, CIFAR100, and SVHN, compare it with vanilla averaging and OT-fusion, report post-fusion, fine-tuning, and distillation accuracies over five seeds, and measure runtime and peak memory. They also ablate the order of operations, number of models, reduction schemes, layer width, and depth, and identify a failure mode when fusing a model with itself or with very similar models.

Significance. NT is simple, cheap, and practical, and the paper's strengths include the breadth of ablations, five-seed error bars, a public code release, and concrete efficiency measurements showing that OT-fusion is far more expensive and can run out of memory in regimes where NT does not. If the central claim were established, NT would be a useful alternative to alignment-based fusion and to distillation for ensemble compression. The main problem is that the headline claim of consistently outperforming individual ensemble members is not supported by the experiments as designed, because the fine-tuning comparisons give the fused model extra training budget without a no-fusion control, and where such a control exists within the distillation tables, NT is not consistently better. The paper also honestly documents a fundamental limitation for similar models, which needs to be incorporated into the stated scope of the method.

major comments (3)
  1. [Section 4.3, Tables 5-7] The central claim that NT 'consistently outperform[s] individual ensemble members of the same model capacity and architecture' is confounded by the additional fine-tuning budget. In the fine-tuning experiments (Table 6), the NT-fused model is fine-tuned for 3-30 epochs and then compared against the original ensemble members at the end of their initial training, so any improvement could come from continued training rather than from transplantation. The paper needs a no-fusion control: fine-tune the best ensemble member, or a same-architecture model, for the same number of epochs with the same optimizer, learning-rate schedule, and data, and compare it to NT. The distillation tables (Table 7) do contain a same-budget individual-model column ('Model0'), and those results do not support the unqualified claim: for LeNet+CIFAR10 the best distillation accuracy is 64.23±0.36 for Model0 versus 63.51±0.37 for NT, and for VGG11+CIFAR10 it is 82.71±0.27 for Model0 versus 82.54±0.28 for NT. In the fine-tuning table, VGG11+CIFAR10 best NT is 81.12±0.27, below the original best member's 81.49±0.28. The sentence 'Any method can fail to do so in some cases' does not salvage the word 'consistently'; the claim should be replaced with a precise statement about relative performance under controlled training budgets.
  2. [Section 3.1 and Section 4.2] The method's only selection criterion is the L2 norm of neurons, and the paper's own experiments demonstrate that NT fails when models are too similar: fusing a model with itself drops from 83.02% to 67.63% before fine-tuning, and using NT as a synchronous-SGD averaging step with four models 'fails to train at all.' This makes the scope of the main claim ambiguous. Without a measurable definition of 'diverse enough,' a user cannot know when the method will work, and the abstract's unqualified 'consistently outperforming' is too broad. The authors should either restrict the central claim to models satisfying an explicit diversity condition or provide a heuristic or statistic that practitioners can compute before applying NT.
  3. [Section 4.3, Tables 5 and 6] Several performance comparisons are based on overlapping error bars or very high variance cells, which weakens the precision of the comparative conclusions. For example, Table 5 reports VGG11+CIFAR100 NT post-fusion accuracy as 36.0±18.88, and Table 6 reports VGG11+CIFAR100 fine-tuning accuracies of 59.33±2.93 for NT versus 56.59±2.0 for OT; with these spreads, 'comparable or better' is not a precise quantitative statement. The authors should report paired per-seed comparisons or effect sizes for the NT-versus-OT and NT-versus-best-member differences rather than only comparing averaged accuracies.
minor comments (4)
  1. [Tables 5 and 8] The '/' entries in Table 5 and the '>38.67 GB' entries in Table 8 should be explained in the corresponding captions as 'OT ran out of memory' rather than leaving the reader to infer it from the main text.
  2. [Figure 4] The caption uses 'Left' and 'Right' without labeling the subplots; adding '(a)' and '(b)' panel labels would make the discussion of the two panels easier to follow.
  3. [Section 4.1] The authors state that biases and batch normalization are removed 'since OT-fusion does not support biases and batch normalization.' Removing batch normalization from VGG11 and ResNet18 is a substantial architectural change; the paper should state whether the same modified architecture was used for all methods and whether this affects the interpretation of the reported absolute accuracies.
  4. [Section 5] In the final paragraph, 'It’s application to transformers' should read 'Its application to transformers'; there are also a few other typographical slips ('I' in the sentence 'I' is a fragment in the paragraph beginning 'In this section we conduct experiments').

Circularity Check

0 steps flagged · score 0.0 of 10

No circular reasoning found; the NT pipeline is evaluated empirically and no fitted parameter or self-citation predetermines the outcome.

full rationale

The paper's derivation chain is an explicit algorithmic pipeline: train ensemble members, concatenate non-output layers, prune by smallest L2-norm to recover single-model architecture, and fine-tune. The central claims about recovering and surpassing individual-member accuracy are empirical results measured on held-out test sets after fine-tuning, not consequences of how the fusion is defined. The importance of large-L2-norm neurons is presented as a hypothesis ('We theorize that only the large weights of the models are needed'), not as a fitted input that forces the comparison outcome. No parameters are fit to the test set, and no prediction is equivalent by construction to an input. The paper contains no load-bearing self-citations; references to prior work are background and comparative baselines. The absence of a control where individual members receive the same additional fine-tuning budget is a legitimate experimental-design limitation, but it is a correctness concern, not circularity. Therefore no circular step can be exhibited, and the appropriate score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The method introduces no new fitted parameters or invented entities. Its load-bearing assumptions concern the validity of magnitude pruning and the learnability of zero-initialized cross-weights under diversity.

assumptions (3)
  • domain assumption Structured magnitude pruning with L2 norm selects the most important neurons.
    Invoked in Section 3 (Pruning). The method's success depends on L2 norm being a valid importance measure; the paper notes L1 would likely work similarly.
  • domain assumption Zero-initialized cross-weights in the concatenated model can be learned by fine-tuning to recombine transplanted neurons.
    Stated in Section 3 and used throughout; the paper provides empirical evidence but no theoretical justification for this learnability.
  • domain assumption Ensemble members are sufficiently diverse, so the top-magnitude neurons are complementary rather than redundant.
    Stated in Section 3.1 and Discussion: the method fails when fusing a model with itself or highly similar models, showing this assumption is load-bearing.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Model Fusion via Neuron Transplantation." pith.science (2026). https://pith.science/paper/NKWQ4667

@misc{pith2026250206849,
  author       = {Pith},
  title        = {Pith review of: Model Fusion via Neuron Transplantation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NKWQ4667}},
  note         = {Machine review of arXiv:2502.06849}
}
read the original abstract

Ensemble learning is a widespread technique to improve the prediction performance of neural networks. However, it comes at the price of increased memory and inference time. In this work we propose a novel model fusion technique called \emph{Neuron Transplantation (NT)} in which we fuse an ensemble of models by transplanting important neurons from all ensemble members into the vacant space obtained by pruning insignificant neurons. An initial loss in performance post-transplantation can be quickly recovered via fine-tuning, consistently outperforming individual ensemble members of the same model capacity and architecture. Furthermore, NT enables all the ensemble members to be jointly pruned and jointly trained in a combined model. Comparing it to alignment-based averaging (like Optimal-Transport-fusion), it requires less fine-tuning than the corresponding OT-fused model, the fusion itself is faster and requires less memory, while the resulting model performance is comparable or better. The code is available under the following link: https://github.com/masterbaer/neuron-transplantation.

Figures

Figures reproduced from arXiv: 2502.06849 by the authors.

Figure 1
Figure 1. Neuron Transplantation. Low-magnitude neurons are replaced by large [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Pipeline of fusing multiple ensemble members. Multiple models are trained [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Concatenating 2D convolution layers. Channels are stacked, batch nor [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Left: Transplanting different neuron amounts of one model into another. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Mean accuracy plots for NT, OT and vanilla averaging for five differ [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 18 canonical work pages

  1. [1]

    Öz et al

    Ainsworth, S.K., Hayase, J., Srinivasa, S.: Git re-basin: Merging models modulo permutation symmetries (2023).https://doi.org/10.48550/arXiv.2209.04836 16 M. Öz et al

    arXiv doi:10.48550/arxiv.2209.04836 2023

  2. [2]

    CoRRabs/2012.09816 (2020), https://arxiv

    Allen-Zhu, Z., Li, Y.: Towards understanding ensemble, knowledge distillation and self-distillation in deep learning. CoRRabs/2012.09816 (2020), https://arxiv. org/abs/2012.09816

    arXiv 2020

  3. [3]

    Anwar, S., Hwang, K., Sung, W.: Structured pruning of deep convolutional neural networks. J. Emerg. Technol. Comput. Syst.13(3) (feb 2017),https://doi.org/ 10.1145/3005348

    doi:10.1145/3005348 2017

  4. [4]

    In: 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA) (12 2017).https://doi.org/10.1109/ICMLA.2017.00-72

    Bhagat Smith, J., Gashler, M.: Investigation of how neural networks learn from the experiences of peers through periodic weight averaging investigation of how neural networks learn from the experiences of peers through periodic weight averaging. In: 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA) (12 2017).https://doi....

    work page doi:10.1109/icmla.2017.00-72 2017

  5. [5]

    https://doi.org/10.48550/arXiv.2204.02311

    Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H.W., Sutton, C., Gehrmann, S., Schuh, P., Shi, K., Tsvyashchenko, S., Maynez, J., Rao, A., Barnes, P., Tay, Y., Shazeer, N., Prab- hakaran, V., Reif, E., Du, N., Hutchinson, B., Pope, R., Bradbury, J., Austin, J., Isard, M., Gur-Ari, G., Yin, P., Duke, T., Levsk...

    arXiv doi:10.48550/arxiv.2204.02311 2022

  6. [6]

    Das, D., Avancha, S., Mudigere, D., Vaidynathan, K., Sridharan, S., Kalamkar, D., Kaul, B., Dubey, P.: Distributed deep learning using synchronous stochastic gradient descent (2016).https://doi.org/10.48550/arXiv.1602.06709

    arXiv doi:10.48550/arxiv.1602.06709 2016

  7. [7]

    IEEE Signal Processing Magazine29(6), 141–142 (2012).https://doi

    Deng, L.: The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine29(6), 141–142 (2012).https://doi. org/10.1109/MSP.2012.2211477

    arXiv 2012

  8. [8]

    Dong, X., Yu, Z., Cao, W., Shi, Y., Ma, Q.: A survey on ensemble learn- ing. Front. Comput. Sci. 14(2), 241–258 (2020), https://doi.org/10.1007/ s11704-019-8208-z

    work page 2020

Show all 39 references
  1. [9]

    Entezari, R., Sedghi, H., Saukh, O., Neyshabur, B.: The role of permutation in- variance in linear mode connectivity of neural networks (2022).https://doi.org/ 10.48550/arXiv.2110.06296

    2022 doi arXiv

  2. [10]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and PatternRecognition.pp.16091–16101(2023)

    Fang,G.,Ma,X.,Song,M.,Mi,M.B.,Wang,X.:Depgraph:Towardsanystructural pruning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and PatternRecognition.pp.16091–16101(2023). https://doi.org/10.48550/arXiv. 2301.12900

    2023 doi arXiv

  3. [11]

    Journal of Machine Learning Research 22(78), 1–8 (2021), http:// jmlr.org/papers/v22/20-451.html

    Flamary, R., Courty, N., Gramfort, A., Alaya, M.Z., Boisbunon, A., Chambon, S., Chapel, L., Corenflos, A., Fatras, K., Fournier, N., Gautheron, L., Gayraud, N.T., Janati, H., Rakotomamonjy, A., Redko, I., Rolet, A., Schutz, A., Seguy, V., Sutherland, D.J., Tavenard, R., Tong, ...

    2021

  4. [12]

    CoRR abs/1512.03385 (2015), http://arxiv.org/abs/1512.03385

    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR abs/1512.03385 (2015), http://arxiv.org/abs/1512.03385

    2015 arXiv

  5. [13]

    https://doi.org/10.48550/arXiv.1503.02531 Model Fusion via Neuron Transplantation 17

    Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network (2015). https://doi.org/10.48550/arXiv.1503.02531 Model Fusion via Neuron Transplantation 17

    2015 doi arXiv

  6. [14]

    CoRRabs/1502.03167 (2015), http://arxiv

    Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. CoRRabs/1502.03167 (2015), http://arxiv. org/abs/1502.03167

    2015 arXiv

  7. [15]

    University of Toronto (05 2012)

    Krizhevsky, A.: Learning multiple layers of features from tiny images. University of Toronto (05 2012)

    2012

  8. [16]

    Proceedings of the IEEE86(11), 2278–2324 (1998).https: //doi.org/10.1109/5.726791

    Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE86(11), 2278–2324 (1998).https: //doi.org/10.1109/5.726791

    1998 doi

  9. [17]

    Neural Processing Letters51(1), 1–22 (Jul 2019), http://dx.doi.org/10.1007/s11063-019-10074-0

    Leontev, M.I., Islenteva, V., Sukhov, S.V.: Non-iterative knowledge fusion in deep convolutional neural networks. Neural Processing Letters51(1), 1–22 (Jul 2019), http://dx.doi.org/10.1007/s11063-019-10074-0

    2019 doi

  10. [18]

    In: International Conference on Learning Representations (2017),https: //openreview.net/forum?id=rJqFGTslg

    Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. In: International Conference on Learning Representations (2017),https: //openreview.net/forum?id=rJqFGTslg

    2017

  11. [19]

    https://doi.org/10.48550/arXiv.2309.15698

    Li, W., Peng, Y., Zhang, M., Ding, L., Hu, H., Shen, L.: Deep model fusion: A survey (2023). https://doi.org/10.48550/arXiv.2309.15698

    2023 doi arXiv

  12. [20]

    Liebenwein, L., Baykal, C., Carter, B., Gifford, D., Rus, D.: Lost in pruning: The effects of pruning neural networks beyond test accuracy (2021).https://doi.org/ 10.48550/arXiv.2103.03014

    2021 doi arXiv

  13. [21]

    In: Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., Sabato, S

    Liu, C., Lou, C., Wang, R., Xi, A.Y., Shen, L., Yan, J.: Deep neural network fusion via graph matching with applications to model ensemble and federated learning. In: Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., Sabato, S. (eds.) Proceedings of the 39th Inter...

    2022

  14. [22]

    https://doi.org/10.48550/arXiv

    Mishra, R., Gupta, H.P., Dutta, T.: A survey on deep neural network compression: Challenges, overview, and solutions (2020). https://doi.org/10.48550/arXiv. 2010.03954

    2020 doi arXiv

  15. [23]

    Journal of King Saud University - Computer and Information Sciences 35(2), 757–774 (2023), https://www.sciencedirect.com/ science/article/pii/S1319157823000228

    Mohammed, A., Kora, R.: A comprehensive review on ensemble deep learning: Opportunities and challenges. Journal of King Saud University - Computer and Information Sciences 35(2), 757–774 (2023), https://www.sciencedirect.com/ science/article/pii/S1319157823000228

    2023

  16. [24]

    NIPS (01 2011)

    Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.: Reading digits in natural images with unsupervised feature learning. NIPS (01 2011)

    2011

  17. [25]

    Neyshabur, B., Sedghi, H., Zhang, C.: What is being transferred in transfer learn- ing? CoRR abs/2008.11687 (2020), https://arxiv.org/abs/2008.11687

    2020 arXiv

  18. [26]

    NVIDIA, Vingelmann, P., Fitzek, F.H.: Cuda, release: 10.2.89 (2020),https:// developer.nvidia.com/cuda-toolkit

    2020

  19. [27]

    In: Advances in Neural Information Processing Systems 32, pp

    Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., Chintala, S.: Pytorch: An imperativ...

    2019

  20. [28]

    Future Genera- tion Computer Systems 150, 272–293 (2024), https://www.sciencedirect.com/ science/article/pii/S0167739X23003333 18 M

    Qi, P., Chiaro, D., Guzzo, A., Ianni, M., Fortino, G., Piccialli, F.: Model aggre- gation techniques in federated learning: A comprehensive survey. Future Genera- tion Computer Systems 150, 272–293 (2024), https://www.sciencedirect.com/ science/article/pii/S0167739X23003333 18...

    2024

  21. [29]

    In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence

    Qian, C., Yu, Y., Zhou, Z.H.: Pareto ensemble pruning. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence. p. 2935–2941. AAAI’15, AAAI Press (2015)

    2015

  22. [30]

    IEEE Transactions on Neural Networks 4(5), 740–747 (1993).https://doi.org/10.1109/72.248452

    Reed, R.: Pruning algorithms-a survey. IEEE Transactions on Neural Networks 4(5), 740–747 (1993).https://doi.org/10.1109/72.248452

    1993 doi

  23. [31]

    CoRR abs/1812.02425 (2018), http://arxiv.org/abs/1812.02425

    Shen, Z., He, Z., Xue, X.: MEAL: multi-model ensemble via adversarial learning. CoRR abs/1812.02425 (2018), http://arxiv.org/abs/1812.02425

    2018 arXiv

  24. [32]

    In: International Conference on Learning Representations (2015)

    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale im- age recognition. In: International Conference on Learning Representations (2015). https://doi.org/10.48550/arXiv.1409.1556

    2015 doi arXiv

  25. [33]

    org/10.48550/arXiv.1910.05653

    Singh, S.P., Jaggi, M.: Model fusion via optimal transport (2023).https://doi. org/10.48550/arXiv.1910.05653

    2023 doi arXiv

  26. [34]

    Sun, S., Chen, W., Bian, J., Liu, X., Liu, T.Y.: Ensemble-compression: A new method for parallel training of deep neural networks (2017).https://doi.org/ 10.48550/arXiv.1606.00575

    2017 doi arXiv

  27. [35]

    CoRRabs/1706.03762 (2017), http: //arxiv.org/abs/1706.03762

    Vaswani,A.,Shazeer,N.,Parmar,N.,Uszkoreit,J.,Jones,L.,Gomez,A.N.,Kaiser, L., Polosukhin, I.: Attention is all you need. CoRRabs/1706.03762 (2017), http: //arxiv.org/abs/1706.03762

    2017 arXiv

  28. [36]

    CoRRabs/2011.07449 (2020), https://arxiv.org/abs/ 2011.07449

    Walawalkar, D., Shen, Z., Savvides, M.: Online ensemble model compression using knowledge distillation. CoRRabs/2011.07449 (2020), https://arxiv.org/abs/ 2011.07449

    2020 arXiv

  29. [37]

    CoRR abs/2002.06440 (2020), https: //arxiv.org/abs/2002.06440

    Wang, H., Yurochkin, M., Sun, Y., Papailiopoulos, D.S., Khazaeni, Y.: Feder- ated learning with matched averaging. CoRR abs/2002.06440 (2020), https: //arxiv.org/abs/2002.06440

    2020 arXiv

  30. [38]

    Wortsman, M., Ilharco, G., Gadre, S.Y., Roelofs, R., Gontijo-Lopes, R., Morcos, A.S., Namkoong, H., Farhadi, A., Carmon, Y., Kornblith, S., Schmidt, L.: Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (2022).http...

    2022 doi arXiv

  31. [39]

    CoRR abs/2101.08387 (2021)

    Yang, Y., Lv, H.: Discussion of ensemble learning under the era of deep learning. CoRR abs/2101.08387 (2021). https://doi.org/10.1007/ s10462-022-10283-5, https://arxiv.org/abs/2101.08387

    2021 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.