Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Reinitializing weights vs units for maintaining plasticity in neural networks

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Selective weight reinitialization maintains plasticity in all four permuted MNIST settings studied, while unit reinitialization fails for small and layer-normalized networks.

desk verdict Weight-level reinitialization is a genuinely useful empirical finding, but the per-setting tuning protocol is the weak link. read the letter →

arxiv 2508.00212 v2 pith:DGEN2ZZV submitted 2025-07-31 cs.NE cs.AI

J. Fernando Hernandez-Garcia , Shibhansh Dohare , Jun Luo , Rich S. Sutton This is my paper · ORCID
classification cs.NEcs.AI
keywords lossofplasticitycontinuallearningweightreinitializationunitselectivepermutedMNISTlayernormalization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that when a continually learning network must give up some of its parameters to stay plastic, the level at which the reset happens matters: reinitializing individual weights is a more reliable scheme than reinitializing whole units. To test this, the authors introduce selective weight reinitialization, which periodically measures the utility of every weight and resamples the least useful ones from the original initialization distribution. In permuted MNIST they find that this weight-level scheme keeps performance flat in all four settings tested, whereas the unit-level algorithms continual backpropagation and ReDo lose plasticity in small networks and in networks with layer normalization. The authors conclude that reinitializing weights maintains plasticity across a wider variety of settings than reinitializing units, and that it is simpler to apply because it ignores connectivity structure. A reader should care because loss of plasticity is a fundamental failure mode for any system that must learn from a changing stream of data, and the paper identifies a cheap, architecture-agnostic fix.

What carries the argument

The central object is selective weight reinitialization, a reinitialization scheme that operates on individual weights instead of whole units. Its machinery has three components: a utility function that scores each weight (magnitude $|w|$ or gradient utility $|w \cdot g_w|$), a pruning function that selects which weights to reset (proportional pruning, which resets a fixed fraction, or threshold pruning, which resets all weights below $k$ times the layer's mean utility), and a reinitialization method that draws new values from the original initialization distribution (resample) or moves them to its mean (mean). Every $\tau$ updates, the least useful weights are replaced, restoring the small-magnitude initial conditions that promote learning while preserving the rest of the network's stored information. The granularity of resetting individual weights is what lets the algorithm survive settings where resetting entire units destroys too much of a small network or interacts badly with layer normalization.

What would settle it

Run continual backpropagation, ReDo, and selective weight reinitialization on permuted MNIST with a small layer-normalized network while selecting hyperparameters with a nested validation split rather than on the evaluation tasks; if the unit-reinitialization methods then match or beat weight reinitialization in average online accuracy on held-out tasks, the paper's central claim fails.

Watch

Extended reading notes

Core claim

The central claim is that reinitialization at the weight level is a more reliable plasticity-maintenance scheme than reinitialization at the unit level in continual supervised learning. The paper proposes selective weight reinitialization, which every $\tau$ updates ranks all weights in each layer by a utility function and reinitializes the fraction with lowest utility, using gradient utility (the product $|w \cdot g_w|$) with threshold pruning and resampling from the initialization distribution. In fully-connected permuted MNIST experiments, this method maintained average online accuracy across 1,000 tasks in a large network, a large network with layer normalization, a ten-unit-per-layer small network, and a small network with layer normalization; continual backpropagation and ReDo were stable only in the large network without layer normalization. In the class-incremental CIFAR-100 vision transformer experiment, selective weight reinitialization partially maintained plasticity and avoided the diverging runs that affected the unit-reinitialization baselines, though it did not fully close the gap to a network trained from scratch. The paper concludes that weight reinitialization succeeds in a wider variety of settings than unit reinitialization.

Load-bearing premise

The comparison rests on tuning each algorithm's hyperparameters on the evaluation tasks themselves and, for the vision transformer results, on using a reparameterized layer-norm variant that was introduced specifically for this comparison.

Editorial extensions

If this is right

  • A small or layer-normalized network can keep learning over thousands of tasks by periodically reinitializing its least useful weights, without changing the optimizer.
  • Unit-level reinitialization algorithms, continual backpropagation and ReDo, should not be the default choice in small networks or networks with layer normalization.
  • Selective weight reinitialization carries no architecture-specific connectivity assumptions, so the same mechanism transfers to transformers and other modern architectures, but only partially fixes plasticity loss there.
  • For continual learning under the big-world hypothesis, where networks cannot rely on spare capacity, weight-level reinitialization is a more dependable plasticity-preserving component than unit-level reinitialization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Weight-level reinitialization is a natural fit for dynamic sparse training: instead of resetting low-utility weights in place, prune and regrow them, which could maintain plasticity and sparsity simultaneously.
  • The layer-norm failure mode suggests normalization layers shrink the gradients reaching units, so unit-reset methods may be resetting units that already stopped learning; a testable extension is to check whether regularizing layer-norm scale parameters eliminates the advantage of weight-level reinitialization.
  • Selective weight reinitialization erases low-utility weights, so it likely amplifies forgetting; combining it with rehearsal or replay is the obvious next experiment for continual learning that must balance plasticity and memory.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes selective weight reinitialization (SWR), which prunes and reinitializes individual weights with the lowest utility, and compares it with two unit-level reinitialization methods, continual backpropagation and ReDo. The comparison is carried out on permuted MNIST in four settings (large/small networks, with/without layer norm) and on class-incremental CIFAR-100 with a vision transformer. The main empirical claims are that SWR maintains plasticity in all four permuted MNIST settings, whereas unit reinitialization fails in small networks and with layer norm, and that in the vision transformer setting SWR is the most stable method when combined with a reparameterized layer norm. The paper also contributes an analysis of correlates of plasticity loss and reports sensitivity curves for hyperparameters.

Significance. The paper addresses a timely problem, loss of plasticity in continual learning, and proposes a simple and modular weight-level reinitialization scheme that has not been systematically studied in this context. If the empirical findings hold, the paper provides a useful practical result: weight-level reinitialization is more robust than unit-level reinitialization in small networks and with layer norm, both of which are relevant to modern architectures and to the 'big world' hypothesis. The permuted MNIST experiments use 30 runs with standard errors, the code is publicly available, and the sensitivity analysis in Appendix B is a valuable addition. The main risks are the hyperparameter selection protocol, which is performed per setting on the evaluation metric itself, and the ad hoc reparameterized layer norm used in the vision transformer experiments; these issues are load-bearing for the comparative claims and need to be addressed before the conclusions can be considered robust.

major comments (3)
  1. [Section 4 / Appendix B] The comparative conclusion is weakened by the hyperparameter selection protocol. According to Appendix B, for each setting, the hyperparameters of every algorithm were chosen through a grid search with ten runs per configuration, selecting the values that maximize the average online accuracy (area under the curve) on the same 1,000 permutations used to produce Figure 2. This means the curves in Figure 2 show each method at its best-found configuration on the evaluation tasks, not its performance under a transferable or representative configuration. The sensitivity curves in Figure 9, while useful, are computed on the same selection metric and do not include a held-out split; they also use only 10 runs per configuration. Because the central claim is that weight reinitialization is 'more reliable' across settings, the authors should either (i) select hyperparameters on a validation subset of permutations and evaluate on held-out permutations, or (ii) fix one hyperparameter configuration per algorithm and transfer it across the four settings, and then report whether the ordering in Figure 2 is preserved. Without this, the observed advantage could be driven by selection noise or grid coverage rather than by the granularity of reinitialization.
  2. [Section 5 / Appendix D] The vision transformer comparison is contingent on a reparameterized layer norm that is introduced ad hoc. The paper observes that the scaling parameter of standard layer norm shrinks during incremental training (Figure 3a) and defines a modified operation y = (x - mean)/std * (1 + gamma) + beta. This changes the parameterization of a standard component, and no argument is given that it preserves the learning dynamics of standard layer norm; Appendix D shows that omitting regularization on layer norm parameters does not fully recover performance, so the reparameterization is load-bearing. The paper also states in Section 5 that without this modification none of the reinitialization algorithms decreased the gap relative to from-scratch training (Appendix F). The claim that weight reinitialization is more reliable in modern architectures should therefore be either (i) supported by an equivalence or invariance argument for the reparameterization, or (ii) explicitly restricted to the reparameterized architecture, with the standard-layer-norm results (Figure 18) presented as the primary comparison.
  3. [Appendix E] The claim that 'These numerical instabilities arose from using 32-bit floats and are not inherent to the reinitialization algorithms or the network architecture' is unsupported. The appendix only shows that the instabilities occur in the key/query matrices and MLP blocks, and that some algorithms suffer more than others. To make this claim, the authors should run the divergent configurations with 64-bit precision (or with a safe clipping strategy) and show that the instabilities disappear; otherwise the statement should be removed or softened.
minor comments (5)
  1. [Affiliations] There is a typo in the affiliation: 'Department of Computeing Science' should be 'Department of Computing Science'.
  2. [Throughout] The algorithm name is written inconsistently as 'ReDo' and 'ReDO'; please use a single spelling throughout.
  3. [Section 1] The citation 'Ash & Adams (Ash & Adams)' appears in incomplete form; it should include the year or a proper parenthetical reference.
  4. [Algorithm 1] The pseudocode contains a typo: 'utiltities' should be 'utilities'. Additionally, consider clarifying that the Bernoulli draw is independent at each reinitialization step.
  5. [Figure 9] The x-axis ranges for the threshold-pruning sensitivity plots differ across subfigures (e.g., some start at 1e-6 and others at 1e-5), which makes direct comparison more difficult; consider using identical axis ranges.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical benchmark comparison whose conclusions are supported by new experiments, not by definitional fits or a load-bearing self-citation chain.

full rationale

This paper is an empirical benchmarking study, not a derivation: it proposes selective weight reinitialization and compares it against continual backpropagation and ReDo on permuted MNIST and incremental CIFAR-100. There is no equation-level chain in which a claimed prediction is algebraically identical to an input. The algorithm's utility functions (magnitude and gradient utility) are standard pruning criteria defined from weights and gradients, not from the plasticity metric being reported, so the result is not self-definitional. Hyperparameters are tuned by grid search on the evaluation tasks, which is a limitation for generalization claims, but the paper does not present tuned configurations as predictions or as fitted parameters renamed as findings; it reports each method at its selected configuration, and the sensitivity curves in Appendix B provide some evidence about robustness. The self-citations to earlier work by Dohare, Sutton, and Hernandez-Garcia define the loss-of-plasticity phenomenon and the continual backpropagation baseline; these are external benchmarks and algorithm definitions rather than a load-bearing uniqueness argument, and the comparative claims are supported by the new experiments in this paper. The reparameterized layer norm introduced in Section 5 is presented frankly as an ad hoc fix and is not justified by a self-citation chain. Accordingly, no circular step meeting the quoted-evidence standard can be identified.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim is empirical; the main fitted quantities are hyperparameters selected per setting on the evaluation metric. No new physical entities are introduced.

free parameters (4)
  • SWR reinitialization frequency tau = 2^11 (large network, threshold pruning)
    Tuned per setting by grid search on average online accuracy (Appendix B, Table 2).
  • SWR reinitialization factor k = 1e-5 (large network, threshold pruning)
    Tuned per setting by grid search on average online accuracy.
  • ReDo reinitialization frequency and threshold = e.g., rf=2^7, rt=5e-4 in ViT
    Tuned per setting; selected for best AUC on evaluation data.
  • Layer norm placement (before or after activation) = after activation chosen
    Compared before and after activation; after activation selected per setting in Appendix B.
assumptions (4)
  • domain assumption Permuted MNIST with 1,000 permutations and one pass per task is a representative continual supervised learning benchmark.
    Sections 2 and 4 use this to draw general conclusions about plasticity maintenance.
  • domain assumption Average online accuracy over a task is a valid measure of plasticity and generalization.
    Section 2 defines loss of plasticity as lower performance than a freshly initialized network; online accuracy is the chosen metric.
  • ad hoc to paper Grid search on the evaluation metric yields a fair hyperparameter comparison across algorithms.
    Appendix B describes tuning each system for highest average online accuracy on the same tasks used in the reported curves; this can favor flexible methods.
  • ad hoc to paper Reparameterized layer norm (1+gamma) preserves the learning dynamics of standard layer norm while avoiding scale shrinkage.
    Section 5 introduces this modification after observing the scaling parameter shrink; the comparison assumes it does not favor any reinitialization method.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reinitializing weights vs units for maintaining plasticity in neural networks." pith.science (2026). https://pith.science/paper/DGEN2ZZV

@misc{pith2026250800212,
  author       = {Pith},
  title        = {Pith review of: Reinitializing weights vs units for maintaining plasticity in neural networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DGEN2ZZV}},
  note         = {Machine review of arXiv:2508.00212}
}
read the original abstract

Loss of plasticity is a phenomenon in which a neural network loses its ability to learn when trained for an extended time on non-stationary data. It is a crucial problem to overcome when designing systems that learn continually. An effective technique for preventing loss of plasticity is reinitializing parts of the network. In this paper, we compare two different reinitialization schemes: reinitializing units vs reinitializing weights. We propose a new algorithm, which we name \textit{selective weight reinitialization}, for reinitializing the least useful weights in a network. We compare our algorithm to continual backpropagation and ReDo, two previously proposed algorithms that reinitialize units in the network. Through our experiments in continual supervised learning problems, we identify two settings when reinitializing weights is more effective at maintaining plasticity than reinitializing units: (1) when the network has a small number of units and (2) when the network includes layer normalization. Conversely, reinitializing weights and units are equally effective at maintaining plasticity when the network is of sufficient size and does not include layer normalization. We found that reinitializing weights maintains plasticity in a wider variety of settings than reinitializing units.

Figures

Figures reproduced from arXiv: 2508.00212 by the authors.

Figure 1
Figure 1. Average online accuracy of selective weight reinitialization with gradient utility and ( [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Average online accuracy of selective weight reinitialization, ReDo, and continual backpropagation in four [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. (a) Average magnitude of the scaling parameter in layer norm, (b) highest accuracy per task, and (c) accuracy relative to the network with reparameterized layer norm trained from scratch. Each line is the average of 20 runs; the shaded regions correspond to the standard error. Only selective weight reinitialization mitigated plasticity loss and maintained a stable performance. To isolate the loss of plasticity effec… view at source ↗
Figures from the paper (15 more)
Figure 4
Figure 4. Figure 4: Average online accuracy of selective weight reinitialization with magnitude utility in the large network [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: Average online accuracy of selective weight reinitialization with gradient utility and proportional pruning in [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Average online accuracy of selective weight reinitialization with gradient utility and threshold pruning in [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Comparison with different optimizers in the large network setting. ( [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Sensitivity curve for selective weight reinitialization with proportional and threshold pruning. Each line is [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]
Figure 9
Figure 9. Figure 9: Sensitivity curve for selective reinitialization algorithms in all the settings studied in Section [PITH_FULL_IMAGE:figures/full_fig_p023_9.png]
Figure 10
Figure 10. Figure 10: Correlates of loss of plasticity in the initial assessment of selective weight reinitialization with gradient [PITH_FULL_IMAGE:figures/full_fig_p025_10.png]
Figure 11
Figure 11. Figure 11: Correlates of loss of plasticity in the large network setting. Each line represents the average of 30 runs, [PITH_FULL_IMAGE:figures/full_fig_p026_11.png]
Figure 12
Figure 12. Figure 12: Correlates of loss of plasticity in the large network with layer norm setting. Each line represents the [PITH_FULL_IMAGE:figures/full_fig_p026_12.png]
Figure 13
Figure 13. Figure 13: Correlates of loss of plasticity in the small network setting. Each line represents the average of 30 runs, [PITH_FULL_IMAGE:figures/full_fig_p026_13.png]
Figure 14
Figure 14. Figure 14: Correlates of loss of plasticity in the small network with layer norm setting. Each line represents the [PITH_FULL_IMAGE:figures/full_fig_p026_14.png]
Figure 15
Figure 15. Figure 15: (a) Top test accuracy in the task for each of the learning systems with different ways of using L2- regularization. (b) Average magnitude of the scaling factor of the layer norm parameter. Reparameterized layer norm without rescaling the regularization factor had the …
Figure 16
Figure 16. Figure 16: Summaries of the weight magnitude of several different layers in the vision transformer architecture: [PITH_FULL_IMAGE:figures/full_fig_p030_16.png]
Figure 17
Figure 17. Figure 17: We omit runs that diverged, so the measurements for ReDo are the average of 17 runs, for continual [PITH_FULL_IMAGE:figures/full_fig_p031_17.png]
Figure 18
Figure 18. Figure 18: Results of reinitialization systems with standard layer norm. The baseline used in the difference of accuracy [PITH_FULL_IMAGE:figures/full_fig_p031_18.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Calibrated Partial Resets: Preventing Policy Collapse in Continual Reinforcement Learning

    cs.LG 2026-07 conditional novelty 5.5 of 10

    Utility-scaled partial neuron resets prevent policy collapse in long-horizon continual RL while matching or beating binary-reset and uniform-decay baselines on several benchmarks.

Reference graph

Works this paper leans on

58 extracted references · 35 canonical work pages · cited by 1 Pith paper

  1. [1]

    Zaheer Abbas, Rosie Zhao, Joseph Modayil, Adam White, and Marlos C. Machado. Loss of plasticity in continual deep reinforcement learning. In Sarath Chandar, Razvan Pascanu, Hanie Sedghi, and Doina Precup (eds.), Proceedings of The 2nd Conference on Lifelong Learning Agents, volume 232 of Proceedings of Machine Learning Research, pp.\ 620--636. PMLR, 22--2...

    work page 2023

  2. [2]

    The impact of reinitialization on generalization in convolutional neural networks, 2021

    Ibrahim Alabdulmohsin, Hartmut Maennel, and Daniel Keysers. The impact of reinitialization on generalization in convolutional neural networks, 2021. Preprint available at https://arxiv.org/abs/2109.00267

    arXiv 2021

  3. [3]

    On warm-starting neural network training

    Jordan Ash and Ryan P Adams. On warm-starting neural network training. Advances in neural information processing systems, 33: 0 3884--3894, 2020

    2020

  4. [4]

    A better match for drivers and riders: Reinforcement learning at lyft

    Xabi Azagirre, Akshay Balwally, Guillaume Candeli, Nicholas Chamandy, Benjamin Han, Alona King, Hyungjun Lee, Martin Loncaric, S \'e bastien Martin, Vijay Narasiman, et al. A better match for drivers and riders: Reinforcement learning at lyft. INFORMS Journal on Applied Analytics, 54 0 (1): 0 71--83, 2024

    work page 2024

  5. [5]

    Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization, 2016. Preprint available at https://arxiv.org/abs/1607.06450

    arXiv 2016

  6. [6]

    What is the state of neural network pruning? Proceedings of machine learning and systems, 2: 0 129--146, 2020

    Davis Blalock, Jose Javier Gonzalez Ortiz, Jonathan Frankle, and John Guttag. What is the state of neural network pruning? Proceedings of machine learning and systems, 2: 0 129--146, 2020

    work page 2020

  7. [7]

    Dokania, Thalaiyasingam Ajanthan, and Philip H

    Arslan Chaudhry, Puneet K. Dokania, Thalaiyasingam Ajanthan, and Philip H. S. Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss (eds.), Computer Vision -- ECCV 2018, pp.\ 556--572, Cham, 2018. Springer International Publishing. ISBN 978-3-0...

    work page 2018

  8. [8]

    BERT: pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming - Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805, 2018. Preprint availabble at http://arxiv.org/abs/1810.04805

    arXiv 2018

Show all 58 references
  1. [9]

    The interplay of search and gradient descent in semi-stationary learning problems

    Shibhansh Dohare. The interplay of search and gradient descent in semi-stationary learning problems. Master's thesis, University of Alberta, 2020

    2020

  2. [10]

    Continual B ackprop: Stochastic gradient descent with persistent randomness, 2021

    Shibhansh Dohare, Richard S Sutton, and A Rupam Mahmood. Continual B ackprop: Stochastic gradient descent with persistent randomness, 2021. Preprint at https://arxiv.org/abs/2108.06325

    2021 arXiv

  3. [11]

    Loss of plasticity in deep continual learning

    Shibhansh Dohare, J Fernando Hernandez-Garcia, Qingfeng Lan, Parash Rahman, A Rupam Mahmood, and Richard S Sutton. Loss of plasticity in deep continual learning. Nature, 632 0 (8026): 0 768--774, 2024

    2024

  4. [12]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

    2021

  5. [13]

    Rupam Mahmood

    Mohamed Elsayed and A. Rupam Mahmood. Addressing loss of plasticity and catastrophic forgetting in continual learning. In The Twelfth International Conference on Learning Representations, 2024. Available at https://openreview.net/pdf?id=sKPzAXoylB

    2024

  6. [14]

    Rupam Mahmood

    Mohamed Elsayed, Qingfeng Lan, Clare Lyle, and A. Rupam Mahmood. Weight clipping for deep continual and reinforcement learning. Reinforcement Learning Journal, 5: 0 2198--2217, 2024

    2024

  7. [15]

    Rigging the lottery: Making all tickets winners

    Utku Evci, Trevor Gale, Jacob Menick, Pablo Samuel Castro, and Erich Elsen. Rigging the lottery: Making all tickets winners. In International conference on machine learning, pp.\ 2943--2952. PMLR, 2020

    2020

  8. [16]

    Depgraph: Towards any structural pruning

    Gongfan Fang, Xinyin Ma, Mingli Song, Michael Bi Mi, and Xinchao Wang. Depgraph: Towards any structural pruning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023

    2023

  9. [17]

    The lottery ticket hypothesis: Finding sparse, trainable neural networks

    Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=rJl-b3RcF7

    2019

  10. [18]

    Robert M. French. Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences , 3 0 (4): 0 128--135, 1999. ISSN 1364-6613

    1999

  11. [19]

    Understanding the difficulty of training deep feedforward neural networks

    Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp.\ 249--256. JMLR Workshop and Conference Proceedings, 2010

    2010

  12. [20]

    Deep learning, volume 1

    Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning, volume 1. MIT Press, 2016

    2016

  13. [21]

    An empirical investigation of catastrophic forgetting in gradient-based neural networks

    Ian J Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empirical investigation of catastrophic forgetting in gradient-based neural networks. In 2nd International Conference on Learning Representations, 2014. Avialable at https://arxiv.org/abs/1312.6211

    2014 arXiv

  14. [22]

    Taylor, Mykola Pechenizkiy, and Decebal Constantin Mocanu

    Bram Grooten, Ghada Sokar, Shibhansh Dohare, Elena Mocanu, Matthew E. Taylor, Mykola Pechenizkiy, and Decebal Constantin Mocanu. Automatic noise filtering with dynamic sparse training in deep reinforcement learning. In Proceedings of the 2023 International Conference on Autono...

    2023

  15. [23]

    Delving deep into rectifiers: Surpassing human-level performance on imagenet classification

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pp.\ 1026--1034, 2015

    2015

  16. [24]

    Gvfs in the real world: making predictions online for water treatment

    Muhammad Kamran Janjua, Haseeb Shah, Martha White, Erfan Miahi, Marlos C Machado, and Adam White. Gvfs in the real world: making predictions online for water treatment. Machine Learning, pp.\ 1--31, 2023

    2023

  17. [25]

    Khurram Javed and Richard S. Sutton. The big world hypothesis and its ramifications for artificial intelligence. In Finding the Frame: An RLC Workshop for Examining Conceptual Frameworks, 2024. Preprint available at https://openreview.net/forum?id=Sv7DazuCn8

    2024

  18. [26]

    Spine dynamics in the brain, mental disorders and artificial neural networks

    Haruo Kasai, Noam E Ziv, Hitoshi Okazaki, Sho Yagishita, and Taro Toyoizumi. Spine dynamics in the brain, mental disorders and artificial neural networks. Nature Reviews Neuroscience, 22 0 (7): 0 407--422, 2021

    2021

  19. [27]

    Learning multiple layers of features from tiny images, 2009

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images, 2009. Retrieved from https://www.cs.toronto.edu/ kriz/learning-features-2009-TR.pdf

    2009

  20. [28]

    Implicit under-parameterization inhibits data-efficient deep reinforcement learning

    Aviral Kumar, Rishabh Agarwal, Dibya Ghosh, and Sergey Levine. Implicit under-parameterization inhibits data-efficient deep reinforcement learning. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=O9bnihsFfXU

    2021

  21. [29]

    Maintaining plasticity in continual learning via regenerative regularization

    Saurabh Kumar, Henrik Marklund, and Benjamin Van Roy. Maintaining plasticity in continual learning via regenerative regularization. In Proceedings of the 3rd Conference on Lifelong Learning Agents. PMLR, 2024

    2024

  22. [30]

    Plastic: Improving input and label plasticity for sample efficient reinforcement learning

    Hojoon Lee, Hanseul Cho, Hyunseung Kim, Daehoon Gwak, Joonkee Kim, Jaegul Choo, Se-Young Yun, and Chulhee Yun. Plastic: Improving input and label plasticity for sample efficient reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024 a

    2024

  23. [31]

    Slow and steady wins the race: Maintaining plasticity with hare and tortoise networks

    Hojoon Lee, Hyeonseo Cho, Hyunseung Kim, Donghu Kim, Dugki Min, Jaegul Choo, and Clare Lyle. Slow and steady wins the race: Maintaining plasticity with hare and tortoise networks. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Sca...

    2024

  24. [32]

    Learning continually by spectral regularization, 2024

    Alex Lewandowski, Saurabh Kumar, Dale Schuurmans, Andr \'a s Gy \"o rgy, and Marlos C Machado. Learning continually by spectral regularization, 2024. Preprint available at https://arxiv.org/pdf/2406.06811

    2024 arXiv

  25. [33]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=Bkg6RiCqY7

    2019

  26. [34]

    Understanding and preventing capacity loss in reinforcement learning

    Clare Lyle, Mark Rowland, and Will Dabney. Understanding and preventing capacity loss in reinforcement learning. In International Conference on Learning Representations, 2022. Avialable at https://openreview.net/forum?id=ZkC8wKoLbQ7

    2022

  27. [35]

    Understanding plasticity in neural networks

    Clare Lyle, Zeyu Zheng, Evgenii Nikishin, Bernardo Avila Pires, Razvan Pascanu, and Will Dabney. Understanding plasticity in neural networks. In International Conference on Machine Learning, pp.\ 23190--23211. PMLR, 2023

    2023

  28. [36]

    Normalization and effective learning rates in reinforcement learning

    Clare Lyle, Zeyu Zheng, Khimya Khetarpal, James Martens, Hado van Hasselt, Razvan Pascanu, and Will Dabney. Normalization and effective learning rates in reinforcement learning. arXiv preprint arXiv:2407.01800, 2024. Preprint available at https://arxiv.org/abs/2407.01800

    2024 arXiv

  29. [37]

    Representation search through generate and test

    Ashique Rupam Mahmood and Richard S Sutton. Representation search through generate and test. In AAAI Workshop: Learning Rich Representations from Low-Level Sensors, 2013

    2013

  30. [38]

    Torchvision: Pytorch's computer vision library

    TorchVision maintainers and contributors. Torchvision: Pytorch's computer vision library. https://github.com/pytorch/vision, 2016

    2016

  31. [39]

    Catastrophic interference in connectionist networks: The sequential learning problem

    Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of Learning and Motivation, 24: 0 109--165, 1989

    1989

  32. [40]

    Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science

    Decebal Constantin Mocanu, Elena Mocanu, Peter Stone, Phuong H Nguyen, Madeleine Gibescu, and Antonio Liotta. Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science. Nature communications, 9 0 (1): 0 2383, 2018

    2018

  33. [41]

    The primacy bias in deep reinforcement learning

    Evgenii Nikishin, Max Schwarzer, Pierluca D’Oro, Pierre-Luc Bacon, and Aaron Courville. The primacy bias in deep reinforcement learning. In International conference on machine learning, pp.\ 16828--16847. PMLR, 2022

    2022

  34. [42]

    Deep reinforcement learning with plasticity injection

    Evgenii Nikishin, Junhyuk Oh, Georg Ostrovski, Clare Lyle, Razvan Pascanu, Will Dabney, and Andre Barreto. Deep reinforcement learning with plasticity injection. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=jucDLW6G9l

    2023

  35. [43]

    GPT -4 technical report, 2023

    OpenAI. GPT -4 technical report, 2023. Preprint available at https://arxiv.org/abs/2303.08774

    2023 arXiv

  36. [44]

    Toward generate-and-test algorithms for continual feature discovery

    Parash Rahman. Toward generate-and-test algorithms for continual feature discovery. Master's thesis, University of Alberta, 2021

    2021

  37. [45]

    i C a RL : Incremental classifier and representation learning

    Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. i C a RL : Incremental classifier and representation learning. In IEEE Conference on Computer Vision and Pattern Recognition, pp.\ 2001--2010, 2017

    2001

  38. [46]

    The dormant neuron phenomenon in deep reinforcement learning

    Ghada Sokar, Rishabh Agarwal, Pablo Samuel Castro, and Utku Evci. The dormant neuron phenomenon in deep reinforcement learning. In International Conference on Machine Learning, pp.\ 32145--32168. PMLR, 2023

    2023

  39. [47]

    Overtrained language models are harder to fine-tune, 2025

    Jacob Mitchell Springer, Sachin Goyal, Kaiyue Wen, Tanishq Kumar, Xiang Yue, Sadhika Malladi, Graham Neubig, and Aditi Raghunathan. Overtrained language models are harder to fine-tune, 2025. Preprint available at https://arxiv.org/abs/2503.19206

    2025 arXiv

  40. [48]

    Sutton and Shibhansh Dohare

    Richard S. Sutton and Shibhansh Dohare. Maintaining plasticity in deep continual learning. Keynote presentation at Conference on Lifelong Learning Agents, 2022. Available at https://www.youtube.com/watch?v=p_zknyfV9fY

    2022

  41. [49]

    Knowledge evolution in neural networks

    Ahmed Taha, Abhinav Shrivastava, and Larry Davis. Knowledge evolution in neural networks. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp.\ 12838--12847, 2021. doi:10.1109/CVPR46437.2021.01265

    2021

  42. [50]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Info...

    2017

  43. [51]

    Same accuracy, twice as fast: continuous training surpasses retraining from scratch, 2025

    Eli Verwimp, Guy Hacohen, and Tinne Tuytelaars. Same accuracy, twice as fast: continuous training surpasses retraining from scratch, 2025. Preprint available at https://arxiv.org/abs/2502.21147

    2025 arXiv

  44. [52]

    I Can't Believe It's Not Better! - Understanding Deep Learning Through Empirical Falsification

    Sheheryar Zaidi, Tudor Berariu, Hyunjik Kim, Jorg Bornschein, Claudia Clopath, Yee Whye Teh, and Razvan Pascanu. When does re-initialization work? In Javier Antorán, Arno Blaas, Fan Feng, Sahra Ghalebikesabi, Ian Mason, Melanie F. Pradier, David Rohde, Francisco J. R. Ruiz, an...

    2022

  45. [53]

    Continual learning through synaptic intelligence

    Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International conference on machine learning, pp.\ 3987--3995. PMLR, 2017

    2017

  46. [54]

    Fortuitous forgetting in connectionist networks

    Hattie Zhou, Ankit Vani, Hugo Larochelle, and Aaron Courville. Fortuitous forgetting in connectionist networks. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=ei3SY1_zYsE

    2022

  47. [55]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  48. [56]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  49. [57]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  50. [58]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.