Pith. sign in

REVIEW 3 major objections 6 minor 32 references

Sparse Training from Random Initialization: Aligning Lottery Ticket Masks using Weight Symmetry

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that lottery-ticket masks can be reused with new random initializations by permuting the mask to align with the new optimization basin, and supports the claim with experiments on CIFAR-10, CIFAR-100, and ImageNet.

desk verdict The permuted-mask result is consistent and worth taking seriously, but the basin-alignment explanation needs a random-permutation control before it can be believed. read the letter →

arxiv 2505.05143 v2 pith:U67H3UR6 submitted 2025-05-08 cs.LG

Mohammed Adnan , Rohan Jain , Ekansh Sharma , Rahul G. Krishnan , Yani Ioannou This is my paper · ORCID
classification cs.LG
keywords lotterytickethypothesissparsetrainingpermutationsymmetryactivationmatchinglinearmodeconnectivityweightrewindingfunctionaldiversitymodelmerging
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a lottery-ticket mask fails to transfer to a new random initialization not because the mask is tied to its original initialization, but because the mask's sparse optimization basin is misaligned with the new initialization's basin. The proposed fix is to permute the mask by a permutation $\pi$ obtained by activation-matching two dense models trained to convergence, so that the mask points into the new basin. Empirically, training a fresh initialization with the permuted mask $\pi(m_A)$ beats training it with the unpermuted mask $m_A$ across CIFAR-10, CIFAR-100, and ImageNet on VGG11 and ResNet20/50, with the gap narrowing as model width grows. If correct, this turns lottery-ticket masks into reusable assets for sparse training from arbitrary initializations rather than one-use tickets.

What carries the argument

The load-bearing object is the permutation $\pi$ computed by activation matching: for each layer, $\pi_l = \arg\max_\pi \langle \pi, Z_B (Z_A)^\top \rangle_F$, solved as a linear assignment problem with the Hungarian algorithm, which permutes the output neurons of one layer and the input neurons of the next. Applying this $\pi$ to the sparse mask reorients the mask into the new initialization's loss basin. Two supporting devices make the argument measurable: REPAIR, which renormalizes permuted activations to remove the variance-collapse barrier, and the loss barrier $B(\theta_1,\theta_2) = \sup_{\alpha \in [0,1]} [ L((1-\alpha)\theta_1+\alpha\theta_2) - ((1-\alpha)L(\theta_1)+\alpha L(\theta_2))]$, which quantifies whether two models share a basin.

What would settle it

At the rewind epoch $k$, compute the error barrier between $\pi(w_A^k)$ and $w_B^k$ for ResNet20 on CIFAR-10; if the barrier at $k$ stays high while the converged barrier is near zero, the permutation is not transferring to the rewind point and the accuracy gain of the permuted mask would need another explanation, and a second check should compare the permuted mask against a randomly permuted mask under identical training.

Watch

Extended reading notes

Core claim

The central claim is that weight-symmetry alignment, not the original initialization, is what makes a lottery-ticket mask effective. Given a mask $m_A$ derived from model $A$ by iterative magnitude pruning, and a fresh random initialization $w_B^0$, the authors find a permutation $\pi$ that aligns the activations of fully trained $A$ with fully trained $B$, then use $\pi(m_A)$ as the mask for sparse training of $B$ from a rewind checkpoint $w_B^k$. Across ResNet20 widths 1, 4, 8, 16, VGG11, and ResNet50 on ImageNet, the permuted runs consistently outperform naive runs that use $m_A$ unchanged, and the advantage widens at higher sparsity and larger width. The paper also argues that the dense solution and its LTH sparse solution lie in the same linearly connected basin once REPAIR corrects variance collapse, which differs from an earlier analysis that omitted that correction.

Load-bearing premise

The permutation found by matching two fully trained dense models is assumed to transfer to the much earlier rewind epoch and to the sparse masked training trajectory, so that permuting the mask by this $\pi$ places the sparse model inside the new initialization's basin.

Editorial extensions

If this is right

  • Sparse training from a new random initialization can reuse an existing lottery-ticket mask by permuting it, recovering much of the LTH accuracy without reusing the mask's original initialization.
  • The accuracy gap between the permuted solution and the true LTH solution shrinks as network width grows, because wider models allow more accurate permutation matching.
  • Ensembles of models trained with different permutations of the same mask are more functionally diverse than ensembles of LTH retrains, and the permuted ensemble reaches higher accuracy than the LTH ensemble.
  • The dense solution and its LTH sparse solution are linearly mode connected once variance collapse is repaired, sharpening the earlier claim that sparse masks tunnel into a separate basin.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A cheap approximation to the alignment permutation, such as matching at an early training checkpoint as the appendix tests, could make mask transfer practical; the early-matching results suggest most of the benefit survives matching at epoch 5.
  • If alignment is the true bottleneck, the same mechanism should allow masks to transfer across related settings, such as between different network widths or teacher-student pairs, whenever the permutation-merged model has a low loss barrier.
  • The paper's own loss-barrier plots predict that a better solver for the permutation problem, replacing the greedy Hungarian matching, would shrink the remaining LTH gap on ImageNet, where the barrier after permutation is highest.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript proposes that lottery ticket masks fail to transfer to new random initializations because the mask is not aligned with the optimization basin of the new initialization. The authors compute a permutation pi by activation matching between two dense models trained from different initializations, apply pi to the LTH mask, and train from the new initialization with the permuted mask (permuted). They compare this against training with the original mask (naive) and against the LTH baseline, across CIFAR-10, CIFAR-100, and ImageNet, for several sparsities, rewind points, and model widths. They report that permuted consistently outperforms naive, with the gap increasing with sparsity and width, and that ensembles of permuted models are more functionally diverse. They also show, using REPAIR, that the dense solution and the LTH solution lie in the same linearly connected mode.

Significance. The empirical pattern is broad and internally consistent: three-run error bars on CIFAR, multiple sparsities and rewind points, four width multipliers, and a public codebase. The diversity analysis is a useful additional contribution. If the mechanism were established, the work would be a meaningful step toward sparse training from random initializations and toward connecting weight-space symmetry to lottery-ticket transfer. However, the central causal interpretation is not yet supported because every comparison differs in the mask (mA versus pi(mA)) and no random-permutation control is included. The paper also acknowledges that Sharma et al. (2024) already demonstrated mask reuse modulo permutations, so the novel element is the basin-alignment mechanism and its empirical evaluation; the novelty boundary should be sharpened.

major comments (3)
  1. [Section 3, Tables 5-14] The paper's central claim is that permuting the LTH mask with the activation-matching permutation pi improves sparse training from a new random initialization relative to using the original mask. In every comparison, the two conditions differ only in the mask: mA versus pi(mA). There is no control in which the mask is transformed by a random permutation rho(mA) that preserves layerwise sparsity but is not aligned to the new initialization's basin. If rho(mA) also outperforms naive, then the improvement is not evidence for basin alignment but simply evidence that a different mask trains better from a new initialization. Please add at least one random-permutation baseline (ideally several) for a representative subset of the main CIFAR-10 ResNet20 settings, and report whether permuted > random > naive, or permuted is approximately equal to random. Without this control, the causal hypothesis that 'misalignment of basins is the reason why LTH masks do not generalize' remains untested.
  2. [Section 4.1 (ResNet50/ImageNet), Table 10] The ImageNet results in Table 10 are reported as single numbers with no error bars or number of seeds, in contrast to the three-run averages used for the CIFAR experiments. The VGG11 results also omit the high-sparsity naive baselines, described in Section 4.1 as unstable. Because the abstract claims that the method works on ImageNet and on VGG11, this asymmetry weakens the generality claim. Please either provide multiple-seed error bars for the ImageNet comparisons or explicitly state that these are single-run observations, and clarify the status of the omitted VGG11 high-sparsity runs.
  3. [Appendix B (Early Matching)] The transfer of pi computed on dense models to the sparse masked training trajectory is a load-bearing assumption. Appendix B shows that pi computed at earlier dense epochs (t=5,20,50,100) gives similar downstream accuracy for the permuted solution, but it does not test whether pi actually aligns the sparse basin at the rewind epoch. The only evidence connecting pi to the sparse basin is the accuracy gain, which is the same evidence used for the central claim and is therefore subject to the random-permutation concern in Major Comment 1. A direct test would be to compute loss barriers between the rewound sparse initialization with the permuted mask and the final sparse solution, and to compare with the barrier under a random permutation of the mask.
minor comments (6)
  1. [Table 1 caption] The caption uses 'Jenson-Shannon'; this should be 'Jensen-Shannon'.
  2. [Figure 11 caption] The caption contains the typo 'peroforming'; it should be 'performing'.
  3. [Figure captions (Figures 5-10)] The sentence 'The dashed (- -) line shows the dense model accuracy' is repeated verbatim in many captions; please vary or reduce the repetition.
  4. [Section 4.2] The permuted ensemble varies both the permutation and the random initialization, while the LTH ensemble fixes both and varies only the data order; this difference should be acknowledged when interpreting the diversity comparison, since the permuted ensemble confounds mask diversity with initialization diversity.
  5. [Appendix D] Appendix D states that the alignment process 'adds a small computational overhead,' but the method requires training two dense models to convergence before the permutation is computed; the phrasing should be revised to acknowledge that the total compute of the permuted solution is substantially higher than LTH or naive.
  6. [Section 2 (Background & Related Work)] The text says Sharma et al. (2024) 'showed that for certain neural networks, sparse mask obtained via weight rewinding can be reused modulo permutations without hurting the test performance'; the Introduction should state more explicitly how the present contribution differs from that result, beyond the specific activation-matching mechanism and the broader empirical study.

Circularity Check

0 steps flagged · score 2.0 of 10

No derivation-level circularity; permuted-vs-naive accuracy is an out-of-sample empirical result, with only a minor self-citation burden from Sharma et al. (2024).

full rationale

The paper's central comparison is not circular: the permutation pi is obtained by activation matching between two converged dense models (Section 3, Eq. 1), and the reported gains are the test accuracies of sparse models trained with pi(mA) versus mA from a new random initialization. Nothing in the construction forces that ordering; pi is not a fitted parameter whose value encodes the predicted accuracy, and no equation makes the claimed improvement follow by definition. The load-bearing transfer assumption, that a permutation found at dense convergence remains valid at the much earlier rewind epoch and on the sparse trajectory, is an empirical extrapolation. The paper partially tests it in Appendix B (Table 15), but it never compares against a random-permutation control; this is a missing control that weakens the causal 'basin misalignment' interpretation, not a circularity. The main self-citation, Sharma et al. (2024), which shares co-author Ekansh Sharma, already states a closely related result ('sparse mask obtained via weight rewinding can be reused modulo permutations') and is cited for activation-matching stability and early matching. That citation is background support, and the present paper's permuted-versus-naive results are new, externally evaluated, and do not logically reduce to it. Hence no step of the derivation is equivalent to its own input; the nonzero score reflects the minor self-citation burden rather than an actual circular reduction.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The method relies on permutation matching between dense models, the transfer of that permutation to sparse masks and earlier training checkpoints, and the validity of IMP masks after permutation. No numerical parameters are fitted to the results; all hyperparameters are taken from prior work or swept over rewind points.

assumptions (4)
  • domain assumption Independently trained neural networks converge to the same loss basin modulo permutation symmetry (Entezari conjecture, operationalized by Ainsworth et al. 2023).
    The method uses activation matching to find a permutation that aligns dense model A with dense model B; Sections 2 and 3 rely on this being approximately true.
  • ad hoc to paper The permutation obtained by matching two converged dense models transfers to the rewound sparse training trajectory and to the IMP mask.
    This transfer is load-bearing because the same pi is applied to the mask and used at rewind epoch k; it is validated only indirectly in Figures 3 and 4 and Appendix B, never derived.
  • domain assumption Unstructured IMP masks remain valid lottery masks after being permuted by the dense-model permutation.
    The paper gives no proof that pi(m_A) selects a trainable subnetwork; the empirical accuracy results are the only evidence.
  • domain assumption Variance collapse correction (REPAIR) removes the loss barrier between the dense solution and the IMP sparse solutions.
    Used in Figure 4 to support the claim that LTH solutions stay in the dense basin; relies on Jordan et al. 2023's REPAIR procedure.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sparse Training from Random Initialization: Aligning Lottery Ticket Masks using Weight Symmetry." pith.science (2026). https://pith.science/paper/U67H3UR6

@misc{pith2026250505143,
  author       = {Pith},
  title        = {Pith review of: Sparse Training from Random Initialization: Aligning Lottery Ticket Masks using Weight Symmetry},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U67H3UR6}},
  note         = {Machine review of arXiv:2505.05143}
}
read the original abstract

The Lottery Ticket Hypothesis (LTH) suggests there exists a sparse LTH mask and weights that achieve the same generalization performance as the dense model while using significantly fewer parameters. However, finding a LTH solution is computationally expensive, and a LTH sparsity mask does not generalize to other random weight initializations. Recent work has suggested that neural networks trained from random initialization find solutions within the same basin modulo permutation, and proposes a method to align trained models within the same loss basin. We hypothesize that misalignment of basins is the reason why LTH masks do not generalize to new random initializations and propose permuting the LTH mask to align with the new optimization basin when performing sparse training from a different random init. We empirically show a significant increase in generalization when sparse training from random initialization with the permuted mask as compared to using the non-permuted LTH mask, on multiple datasets (CIFAR-10, CIFAR-100 and ImageNet) and models (VGG11, ResNet20 and ResNet50).

Figures

Figures reproduced from arXiv: 2505.05143 by the authors.

Figure 1
Figure 1. Weight Symmetry and the Sparse Training Problem. A model with a single layer and only two parameters, w = (w0,w1), operating on a single input x0 has weight symmetry in the 2D loss landscape as illustrated above. In (a) the original dense model, wA, is trained from a random dense initialization, wt=0 A to a dense solution, wt=T A , which is then pruned using weight magnitude resulting in the mask mA = (1,0). In (b),… view at source ↗
Figure 2
Figure 2. The overall framework of the training procedure, begin￾ning with two distinct dense random weight initializations, w t=0 A , w t=0 B sampled from a normal distribution, N . The sparse training problem attempts to train the random initialization, w t=0 B using the naive mask mA, found by pruning a dense trained model, w t=T A . However, this results in poor generalization performance (Frankle et al., 2020). We propos… view at source ↗
Figure 3
Figure 3. Larger width exhibits better LMC. Plots showing linear interpolation between π(w t=T A ) and w t=T B where π was obtained through activation matching between two dense models for varying widths, w. As the width of the model increases, the permutation matching algorithm gets more accurate, thereby reducing the loss barrier (i.e., better LMC), which is evaluated on the test set. This shows that the permu￾tation matchi… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: LTH solution remains in the same linearly connected mode as the dense solution. In Figure 4a we plot the error barrier between the dense solution and the sparse solution (y-axis) vs the IMP iteration corresponding to the sparse solution (x-axis), for 90% sparsity. We o…
Figure 5
Figure 5. Figure 5: ResNet20×{w}/CIFAR-10.Test accuracy of sparse network solutions vs. increasing rewind points for different sparsity levels and widths, w. The dashed (- -) line shows the dense model accuracy. The effect of the rewind point on the test accuracy for different sparsities …
Figure 6
Figure 6. Figure 6: ResNet20×{w}/CIFAR-100. Test accuracy of sparse network solutions vs. increasing rewind points for different sparsity levels and widths, w. The dashed (- -) line shows the dense model accuracy. The effect of the rewind points on the test accuracy for different sparsiti…
Figure 7
Figure 7. Figure 7: ResNet50×{1}/ImageNet. Top-5 test accuracy vs. rewinds points of sparse network solutions at various sparsity levels. We observe the permuted solution consistently performing better than the naive solution for all sparsities. The dashed (- -) line shows the dense model…
Figure 8
Figure 8. Figure 8: VGG11×{1}/CIFAR-10. Test accuracy of sparse solutions at increasing rewind points for different sparsity levels. The dashed (- -) line shows the dense model accuracy. In Figure 8b, the permuted solution closely matches the LTH solution. However, beyond a certain rewind…
Figure 9
Figure 9. Figure 9: Accuracy vs sparsity trend for ResNet20×{w}/CIFAR-10.As the width increases, the gap between permuted and naive solutions increases, showing permuted masks help with sparse training. With increased width, we observe a more significant gap seen throughout Figures 9d, 9h…
Figure 10
Figure 10. Figure 10: Accuracy vs sparsity trend for ResNet20×{w}/CIFAR-100. Similar to the phenomenon seen in [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]
Figure 11
Figure 11. Figure 11: ResNet50×{1}/ImageNet. Top-1 test accuracy vs rewinds points of sparse network solutions at various sparsity levels. We observe the permuted solution consistently peroforming better than the naive solution for all sparsities. The dashed (- -) line shows the dense mode…
Figure 12
Figure 12. Figure 12: Weight Symmetry and the Sparse Training Problem (Full Figure). A model with a single layer and only two parameters, w= (w0,w1), operating on a single input x0 has the weight symmetry in the 2D loss landscape as illustrated above. In (a) the original dense model, wA, i…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 22 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    K., Hayase, J., and Srinivasa, S

    Ainsworth, S. K., Hayase, J., and Srinivasa, S. S. Git re-basin: Merging models modulo permutation symmetries. In The Eleventh International Conference on Learning Representations, (ICLR). OpenReview.net, 2023

    work page 2023

  3. [3]

    Network optimization: continuous and discrete models, volume 8

    Bertsekas, D. Network optimization: continuous and discrete models, volume 8. Athena Scientific, 1998

    work page 1998

  4. [4]

    Imagenet: A large-scale hierarchical image database

    Deng, J., Dong, W., Socher, R., Li, L., Li, K., and Fei - Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009), 20-25 June 2009, Miami, Florida, USA , pp.\ 248--255. IEEE Computer Society, 2009. doi:10.1109/CVPR.2009.5206848

    arXiv 2009

  5. [5]

    The role of permutation invariance in linear mode connectivity of neural networks

    Entezari, R., Sedghi, H., Saukh, O., and Neyshabur, B. The role of permutation invariance in linear mode connectivity of neural networks. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 . OpenReview.net, 2022

    work page 2022

  6. [6]

    A., Keskin, C., and Dauphin, Y

    Evci, U., Ioannou, Y. A., Keskin, C., and Dauphin, Y. Gradient flow in sparse neural networks and how lottery tickets win. In Proceedings of the 36th AAAI Conference on Artificial Intelligence, February 2022. doi:10.1609/aaai.v36i6.20611

    work page doi:10.1609/aaai.v36i6.20611 2022

  7. [7]

    Deep ensembles: A loss landscape perspective

    Fort, S., Hu, H., and Lakshminarayanan, B. Deep ensembles: A loss landscape perspective. arXiv preprint, 2020. doi:10.48550/arXiv.1912.02757

    arXiv doi:10.48550/arxiv.1912.02757 2020

  8. [8]

    and Carbin, M

    Frankle, J. and Carbin, M. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net, 2019

    work page 2019

Show all 32 references
  1. [9]

    K., Roy, D

    Frankle, J., Dziugaite, G. K., Roy, D. M., and Carbin, M. Linear mode connectivity and the lottery ticket hypothesis. In Proceedings of the 37th International Conference on Machine Learning, ICML'20. JMLR.org, 2020

    2020

  2. [10]

    The state of sparsity in deep neural networks

    Gale, T., Elsen, E., and Hooker, S. The state of sparsity in deep neural networks. arXiv preprint, abs/1902.09574, 2019. doi:10.48550/arXiv.1902.09574

    1902 doi arXiv

  3. [11]

    J., Bengio, Y., and Courville, A

    Goodfellow, I. J., Bengio, Y., and Courville, A. C. Deep Learning. Adaptive computation and machine learning. MIT Press, 2016. ISBN 978-0-262-03561-3

    2016

  4. [12]

    Han, S., Pool, J., Tran, J., and Dally, W. J. Learning both weights and connections for efficient neural network. In Cortes, C., Lawrence, N. D., Lee, D. D., Sugiyama, M., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 28: Annual Conference on Neural...

    2015

  5. [13]

    Han, S., Mao, H., and Dally, W. J. Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding. In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings , 2016

    2016

  6. [14]

    Deep residual learning for image recognition

    He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016 , pp.\ 770--778. IEEE Computer Society, 2016. doi:10.1109/CVPR.2016.90

    2016 doi

  7. [15]

    On The Algebraic Structure of Feedforward Network Weight Spaces

    Hecht-Nielsen, R. On The Algebraic Structure of Feedforward Network Weight Spaces. North-Holland, Amsterdam, 1990. ISBN 978-0-444-88400-8. doi:https://doi.org/10.1016/B978-0-444-88400-8.50019-4

    1990 doi

  8. [16]

    Characterising bias in compressed models

    Hooker, S., Moorosi, N., Clark, G., Bengio, S., and Denton, E. Characterising bias in compressed models. arXiv preprint, 2020. doi:10.48550/arXiv.2010.03058

    2020 doi arXiv

  9. [17]

    and Szegedy, C

    Ioffe, S. and Szegedy, C. Batch normalization: accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37, ICML'15, pp.\ 448–456. JMLR.org, 2015

    2015

  10. [18]

    Analysis of linear mode connectivity via permutation-based weight matching

    Ito, A., Yamada, M., and Kumagai, A. Analysis of linear mode connectivity via permutation-based weight matching. arXiv preprint, 2024. doi:10.48550/arxiv.2402.04051

    2024 doi arXiv

  11. [19]

    REPAIR: renormalizing permuted activations for interpolation repair

    Jordan, K., Sedghi, H., Saukh, O., Entezari, R., and Neyshabur, B. REPAIR: renormalizing permuted activations for interpolation repair. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.net, 2023

    2023

  12. [20]

    Learning multiple layers of features from tiny images

    Krizhevsky, A. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009

    2009

  13. [21]

    Kuhn, H. W. The hungarian method for the assignment problem. In J \" u nger, M., Liebling, T. M., Naddef, D., Nemhauser, G. L., Pulleyblank, W. R., Reinelt, G., Rinaldi, G., and Wolsey, L. A. (eds.), 50 Years of Integer Programming 1958-2008 - From the Early Years to the State...

    1958 doi

  14. [22]

    Convergent learning: Do different neural networks learn the same representations? In Storcheus, D., Rostamizadeh, A., and Kumar, S

    Li, Y., Yosinski, J., Clune, J., Lipson, H., and Hopcroft, J. Convergent learning: Do different neural networks learn the same representations? In Storcheus, D., Rostamizadeh, A., and Kumar, S. (eds.), Proceedings of the 1st International Workshop on Feature Extraction: Modern...

    2015

  15. [23]

    and Kolter, J

    Nagarajan, V. and Kolter, J. Z. Uniform convergence may be unable to explain generalization in deep learning. In Wallach, H., Larochelle, H., Beygelzimer, A., d Alch\' e -Buc, F., Fox, E., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 32. Cu...

    2019

  16. [24]

    and Forde, J

    Paganini, M. and Forde, J. Z. Streamlining tensor and network pruning in pytorch. arXiv preprint, abs/2004.13770, 2020. doi:10.48550/arXiv.2004.13770

    2004 doi arXiv

  17. [25]

    Pytorch: An imperative style, high-performance deep learning library

    Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. Pytorch: An impera...

    2019

  18. [26]

    W., Frankle, J., Ganguli, S., and Dziugaite, G

    Paul, M., Chen, F., Larsen, B. W., Frankle, J., Ganguli, S., and Dziugaite, G. K. Unmasking the lottery ticket hypothesis: What's encoded in a winning ticket's mask? In The Eleventh International Conference on Learning Representations, 2023

    2023

  19. [27]

    Comparing rewinding and fine-tuning in neural network pruning

    Renda, A., Frankle, J., and Carbin, M. Comparing rewinding and fine-tuning in neural network pruning. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020 . OpenReview.net, 2020

    2020

  20. [28]

    M., Rolnick, D., and Dziugaite, G

    Sharma, E., Kwok, D., Denton, T., Roy, D. M., Rolnick, D., and Dziugaite, G. K. Simultaneous linear connectivity of neural networks modulo permutation. In Machine Learning and Knowledge Discovery in Databases. Research Track, pp.\ 262--279, Cham, 2024. Springer Nature Switzerland

    2024

  21. [29]

    and Zisserman, A

    Simonyan, K. and Zisserman, A. Very deep convolutional networks for large-scale image recognition. In Bengio, Y. and LeCun, Y. (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings , 2015

    2015

  22. [30]

    Geometry of the loss landscape in overparameterized neural networks: Symmetries and invariances

    Simsek, B., Ged, F., Jacot, A., Spadaro, F., Hongler, C., Gerstner, W., and Brea, J. Geometry of the loss landscape in overparameterized neural networks: Symmetries and invariances. In Meila, M. and Zhang, T. (eds.), Proceedings of the 38th International Conference on Machine ...

    2021

  23. [31]

    Singh, S. P. and Jaggi, M. Model fusion via optimal transport. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, Dece...

    2020

  24. [32]

    J., Chen, P., Das, P., Melnyk, I., Sattigeri, P., and Lai, R

    Tatro, N. J., Chen, P., Das, P., Melnyk, I., Sattigeri, P., and Lai, R. Optimizing mode connectivity via neuron alignment. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Ne...

    2020

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.