Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

In Praise of Stubbornness: An Empirical Case for Cognitive-Dissonance Aware Continual Update of Knowledge in LLMs

T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Training a language model on facts that contradict what it already knows can destroy up to 80% of its unrelated knowledge, while adding new facts leaves it largely intact.

desk verdict Solid, well-executed empirical study; the catastrophic-forgetting asymmetry is real, but the paper hasn't yet isolated contradiction from overwrite and extra training. read the letter →

arxiv 2502.04390 v2 pith:IKW7DNHK submitted 2025-02-05 cs.CL cs.AIcs.LGq-bio.NC

Simone Clemente , Zied Ben Houidi , Alexis Huet , Dario Rossi , Giulio Franzese , Pietro Michiardi This is my paper · ORCID
classification cs.CLcs.AIcs.LGq-bio.NC
keywords catastrophicforgettingknowledgeeditingcognitivedissonancecontinuallearningselectiveplasticitycontradictiondetectionfactualtransformerlanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that large language models do not treat all new information alike: adding facts that do not contradict what the model knows is usually safe, but training on facts that directly contradict known facts triggers catastrophic forgetting of unrelated knowledge. The authors report that as few as 10-100 contradictory facts can wipe out up to 80% of the model's previously learned knowledge, across GPT-2-small, GPT-2-XL, and GPT-J-6B, under both full fine-tuning and LoRA. They also show that selective-plasticity strategies that protect frequently used neurons preserve old knowledge for non-contradictory updates (98% vs 93%) but fail completely for contradictory ones. Finally, they show that simple classifiers on model activations, gradients, or output probabilities can distinguish novel, known, and contradictory facts with 95%+ accuracy, suggesting a protective mechanism. The human analogy of cognitive dissonance frames the claim as a fundamental asymmetry in gradient-based learning.

What carries the argument

The load-bearing object is the counterfact update itself, built from the COUNTERFACT dataset's fact/counterfact pairs, plus the neuron-usage tracker that classifies neurons as stubborn or plastic by cumulative gradient magnitude during baseline training. Stubborn neurons are those with high historical gradients; plastic neurons are those with low historical gradients; candidate neurons are those with high gradients for the new facts; specific neurons are candidate neurons that avoid stubborn ones. The machinery works by freezing non-selected neurons during fine-tuning (zeroing their gradients before the optimizer step) and by comparing retention of 2,000 baseline facts after non-dissonant versus dissonant updates. The argument's core move is the contrast: the same targeting strategies that preserve knowledge for benign additions fail categorically when the added facts contradict known ones.

What would settle it

Run the epoch-matched control the paper mentions: after the same initial training, fine-tune on a third round of non-contradictory facts for the same number of epochs used for the dissonant round, and measure retention of the 2,000 baseline facts. If retention drops as much as it does after dissonant updates, the catastrophic effect is caused by repeated fine-tuning effort rather than by contradiction; if retention stays high, the asymmetry claim survives. A second check: repeat the dissonant protocol on a model that has never seen the counterfact subjects at all, to verify that the damage requires a real conflict with stored knowledge.

Watch

Extended reading notes

Core claim

The central discovery is a sharp asymmetry in how gradient-based updates affect transformer language models. When the model is fine-tuned on new, non-contradictory facts, it integrates them while retaining roughly 93% of previously learned unrelated facts; when it is fine-tuned on counterfacts that contradict facts it already knows—such as learning "Paris is the capital of Italy" after knowing "Paris is the capital of France"—retention of unrelated knowledge collapses, with standard fine-tuning destroying up to 80% of it even for just 10-100 contradictory facts. The authors argue this is not a capacity or training-approach artifact: the pattern holds across model scales, across full fine-tuning and LoRA, and independently of whether updates target stubborn, plastic, candidate, specific, or random neurons. They frame the failure as the absence of a cognitive-dissonance filter: unlike humans, the model does not resist or contextualize contradictions, it overwrites, and the overwrite radiates damage through unrelated knowledge. They further show the damage is detectable in advance: activation, gradient, and output-distribution features classify novel, known, and contradictory facts with 95%+ accuracy.

Load-bearing premise

The paper's causal story—that contradiction itself, not extra training on a small set, causes the catastrophic forgetting—rests on a control experiment that is described in Section 2.1 but never shown, and on treating the 2,000 tracked facts as a representative sample of the model's knowledge.

Editorial extensions

If this is right

  • Knowledge-editing methods that overwrite facts by gradient fine-tuning will silently corrupt unrelated knowledge whenever the edit contradicts the model's existing beliefs; this applies to both full fine-tuning and LoRA.
  • Selective plasticity—freezing frequently used neurons—is a viable protective strategy only for non-contradictory additions; it should not be relied on for contradiction resolution.
  • Contradiction detection is feasible before training: simple classifiers on internal or output features reach 95%+ accuracy, so a protective filter that blocks or recontextualizes dissonant updates is within reach.
  • The 10-100 fact result implies that even tiny batches of contradictory data—for example, from a data-poisoning or misinformation campaign—pose an outsized risk to deployed models.
  • The authors propose that models should follow human-like append-only updates, preserving old and new versions with episodic context rather than attempting destructive overwrites.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported asymmetry may extend beyond factual knowledge to other learned structures such as ethical guidelines or behavioral alignments; the paper's own discussion of misalignment-after-editing hints at this, but it is not tested here.
  • The causal attribution to contradiction itself remains the main thing to verify: the paper asserts an epoch-matched control but does not present its results, so repeated fine-tuning effort is not yet excluded as a partial cause.
  • A testable extension is to run the same dissonant-update protocol on instruction-tuned or RLHF models, where the model has been explicitly trained to hedge or admit uncertainty; the paper's results are limited to GPT-family base models.
  • The 95% detection accuracy on balanced fact/counterfact pairs may not transfer to long-text, implicit, or context-dependent contradictions; the paper flags this in its limitations, and a dedicated benchmark would be the natural next step.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper experimentally compares two kinds of knowledge updates in GPT-2-small, GPT-2-XL, and GPT-J-6B: adding non-contradictory facts versus training on COUNTERFACT counterfacts that contradict previously learned facts. It reports that non-dissonant updates preserve old knowledge while dissonant updates destroy a large fraction of tracked unrelated knowledge, that selective neuron targeting protects old knowledge for non-dissonant but not dissonant updates, and that a simple classifier on activations and gradients can distinguish known, novel, and contradictory facts with high accuracy. The paper frames these results as evidence that LLMs lack a cognitive-dissonance-like protective mechanism and argues for append-only, non-overwriting update strategies.

Significance. The empirical phenomenon, if the causal attribution is correct, would be an important and actionable finding for continual learning and model editing. The paper has real strengths: 5-fold cross-validation, three model scales, five targeting strategies plus full fine-tuning and LoRA baselines, standard deviations on the main selective-plasticity results, comparisons to ROME and MEMIT, and released code. However, the central causal claim that contradiction rather than update structure or training effort causes the forgetting is not yet supported by the experiments as presented. The missing controls are feasible, and the main text already asserts one of them without showing results. I therefore see the contribution as significant but conditional on additional control experiments.

major comments (4)
  1. [Sec. 2.1] The decisive control ruling out repeated-update and training-duration effects is asserted but never shown: the text states that "control experiments with a third round of non-dissonant updates, instead of dissonant, confirm the catastrophic effect is due to contradictions, not repeated updates," but no results are displayed. Sec. 3 then notes that full fine-tuning needed twice as many epochs to learn dissonant information. Without an epoch-matched or otherwise training-effort-matched third-round control with reported results, the retention loss cannot be attributed specifically to contradiction rather than to longer or repeated training on a small fact set. Please report this control quantitatively.
  2. [Sec. 2.1 and Sec. 3] The dissonant and non-dissonant conditions differ in two ways simultaneously. Non-dissonant updates introduce 1,000 new facts with new subject-relation prompts, whereas dissonant updates reuse the exact subject-relation prompts learned in the immediately preceding stage and change only the object. A non-contradictory overwrite control is therefore required before the forgetting can be blamed on contradiction: for example, updating the same subject-relation prompts with near-synonymous or otherwise compatible objects, or updating the same subjects with new non-contradictory relations, and comparing retention. This is load-bearing because the paper itself cites Hiratani (2024) for the analytical result that high-input-overlap/low-output-overlap updates cause catastrophic forgetting, which is precisely the structural condition of the dissonant condition. That alternative account predicts the observed damage without any appeal to contradiction; if it is correct, the paper's proposed protective mechanism of detecting contradictions would not address the actual cause.
  3. [Abstract, Sec. 3] The claim that "up to 80% of unrelated knowledge" is destroyed is measured only on the 2,000-fact tracked set, which App. F.3 acknowledges is a tiny fraction of the knowledge of larger models. The main text should say "up to 80% of the 2,000 tracked facts" or otherwise qualify the quantity; the current wording overstates what the probe can measure.
  4. [Fig. 3] The central cross-scale demonstration is presented for "one of our folds" without error bars or aggregate statistics, even though the stated protocol is 5-fold cross-validation. Please show the cross-validated mean and standard deviation, or per-fold points, for the Fig. 3 scatter positions, so that the claim that the asymmetry persists across model scales is supported by the full protocol rather than a single fold.
minor comments (6)
  1. [Sec. 2.3 vs App. C.1] Sec. 2.3 says the novel facts were generated with GPT4, while App. C.1 says GPT-3.5; please align these statements.
  2. [App. C.4, App. E.3, Sec. 6] There are typos: "curiousy" should be "curiously" in App. C.4, "neeed" should be "need" in App. E.3, and "limiations" should be "limitations" in Sec. 6.
  3. [Table 4 vs Sec. 5] Table 4 uses "Null" for the no-normalization condition while Sec. 5 calls it "R"; please use one name consistently.
  4. [App. D.2] Hyperparameters are selected by fine-tuning on 10k facts, but the update experiments use 2,000- and 1,000-fact stages; please justify the transfer of these hyperparameters or repeat the selection at the experimental scale.
  5. [App. A.3] App. A.3 cites "Hu et al. [2025]" twice for what appear to be two different papers (insecure-code finetuning and superposition analysis), but the reference list contains only one Hu et al. entry; please check whether both citations point to the same work or add the missing reference.
  6. [Sec. 4] The statement that random selection protects knowledge "likely by avoiding stubborn neurons by chance" is not quantified; please report the overlap between the random and stubborn neuron sets, or soften the claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: all central claims are empirical measurements against external COUNTERFACT counterfact data, with no fitted parameter renamed as a prediction and no load-bearing self-citation.

full rationale

The paper's central claims—that non-dissonant updates preserve unrelated knowledge while dissonant updates destroy it, that selective plasticity helps only in the non-dissonant case, and that contradictions are detectable from model features—are all empirical findings measured against COUNTERFACT's external fact/counterfact pairs, not consequences of a definition or of a fitted constant. The update conditions are defined by external ground truth (whether the new object contradicts the known fact), and the retention metrics are measured on held-out factual accuracy. The asserted but unreported control experiment with a third round of non-dissonant updates is a methodological weakness and a possible confound between contradiction and input-output overlap, but it is not circularity: the claimed effect is not enforced by construction. The candidate-neuron selection uses a single gradient backpropagation on the new facts, but the subsequent measurements of old-knowledge retention and new-knowledge acquisition are independent outcomes, so no quantity reduces to its own input. There are no load-bearing self-citations, no imported uniqueness theorems, and no renaming of a known pattern as a derivation. The paper is self-contained empirical work whose conclusions could be falsified by independent runs, so the appropriate circularity score is 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The paper is empirical; the main unstated premises are the representativeness of COUNTERFACT facts, the proxy metric, and the validity of the unshown control. The only hand-chosen quantities are training hyperparameters and the neuron-count sweep, none of which define the central claim by construction.

free parameters (4)
  • learning rate for GPT-2-small = 5e-4
    Chosen via hyperparameter search (App. D.2) to ensure successful learning of target facts; affects all update experiments.
  • learning rate for GPT-2-XL = 1e-5
    Picked from Table 6 as best for 10k facts; used for full fine-tuning and targeted updates.
  • batch size / epochs for GPT-2-XL = 16 / 10
    From the bold entry in Table 6; part of the training recipe.
  • number of updated neurons N = 2k-20k (or 20k-200k for XL)
    Swept as a design-space variable, not fitted to data; the central asymmetry persists across the sweep.
assumptions (5)
  • domain assumption COUNTERFACT facts are general knowledge that pre-trained models were likely exposed to, so fine-tuning on them approximates updating existing knowledge.
    Sec. 2.1: this justifies calling counterfacts 'doubly dissonant' and treating non-dissonant facts as safe; if false, the dissonance framing weakens.
  • domain assumption The 2,000 baseline facts are a representative sample of model knowledge; retention on them measures corruption of unrelated knowledge.
    Sec. 2.1 and App. F.3: the authors acknowledge the tracked set is tiny relative to GPT-2-XL and GPT-J, so the extrapolation to '80% of unrelated knowledge' relies on this proxy.
  • domain assumption The third-round non-dissonant control isolates contradiction as the cause of forgetting.
    Sec. 2.1 asserts this control exists but does not show results; the causal reading of the central claim depends on it.
  • domain assumption Factual accuracy on COUNTERFACT prompts adequately captures the model's knowledge.
    Used throughout as the sole evaluation metric; no other knowledge probes are used.
  • domain assumption GPT-generated novel facts are truly novel and similar in structure to known facts.
    Sec. 2.3 and App. C.1: classification results depend on this; if novel facts are trivially separable, the 95% accuracy is inflated.
invented entities (1)
  • none
    purpose: no new physical or architectural entities are postulated
    The 'stubborn' and 'plastic' neuron labels are operational definitions based on accumulated gradients, not new mechanistic entities.

how reviews work

0 comments
Cite this review

Pith. "Pith review of In Praise of Stubbornness: An Empirical Case for Cognitive-Dissonance Aware Continual Update of Knowledge in LLMs." pith.science (2026). https://pith.science/paper/IKW7DNHK

@misc{pith2026250204390,
  author       = {Pith},
  title        = {Pith review of: In Praise of Stubbornness: An Empirical Case for Cognitive-Dissonance Aware Continual Update of Knowledge in LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IKW7DNHK}},
  note         = {Machine review of arXiv:2502.04390}
}
read the original abstract

Through systematic empirical investigation, we uncover a fundamental and concerning property of Large Language Models: while they can safely learn facts that don't contradict their knowledge, attempting to update facts with contradictory information triggers catastrophic corruption of unrelated knowledge. Unlike humans, who naturally resist contradictory information, these models indiscriminately accept contradictions, leading to devastating interference, destroying up to 80% of unrelated knowledge even when learning as few as 10-100 contradicting facts. To understand whether this interference could be mitigated through selective plasticity, we experiment with targeted network updates, distinguishing between previously used (stubborn) and rarely used (plastic) neurons. We uncover another asymmetry: while sparing frequently-used neurons significantly improves retention of existing knowledge for non-contradictory updates (98% vs 93% with standard updates), contradictory updates trigger catastrophic interference regardless of targeting strategy. This effect which persists across tested model scales (GPT-2 to GPT-J-6B), suggests a fundamental limitation in how neural networks handle contradictions. Finally, we demonstrate that contradictory information can be reliably detected (95%+ accuracy) using simple model features, offering a potential protective mechanism. These findings motivate new architectures that can, like humans, naturally resist contradictions rather than allowing destructive overwrites.

Figures

Figures reproduced from arXiv: 2502.04390 by the authors.

Figure 1
Figure 1. Overview of our empirical investigation pipeline [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Illustration of selective plasticity strategies based on [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Safe Non-dissonant vs. Catastrophic Dissonant Updates. Results shown in one of our folds, for GPT-2 Small 3a, GPT-2 XL 3b, and GPT-J-6B 3c, comparing full fine-tuning (stars) and LoRA (crosses) approaches. The stark contrast between dissonant (red) and non-dissonant (green) updates persists across facts, model scales and training methods. Note that for full finetuning, models were trained, as per our protocol, until… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: The Selective Plasticity Asymmetry (GPT-2-small). While avoiding stubborn neurons preserves old knowledge during non-dissonant updates, all strategies fail catastrophically with dissonant updates. See [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Dissonance awareness. Feature importance showing the higher importance of gradient￾related features for finetuned models. turns out that in the finetuned scenario, gradient-based features are substantially more important. This is likely due to the fact that finetuning …
Figure 6
Figure 6. Figure 6: Block Importance. Albeit differences are visible, the tendency is not as marked as for the activation vs gradient based feature importance in Fig.5 - GPT2-small 21 [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 7
Figure 7. Figure 7: Distribution of stubborn neurons across GPT2-XL transformer blocks for different neuron [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]
Figure 8
Figure 8. Figure 8: Lottery ticket 2. Non-Lottery Neurons: Neurons underutilized during the initial training on Facts H. 3. Random Neurons: Neurons selected randomly from the entire network [PITH_FULL_IMAGE:figures/full_fig_p024_8.png]
Figure 9
Figure 9. Figure 9: Non-Dissonant updates with GPT2-XL under various conditions. Overall the same trends as GPT2-small are confirmed: targeting stubborn neurons destroys old knowledge more and plastic neurons need more space or time to learn [PITH_FULL_IMAGE:figures/full_fig_p025_9.png]
Figure 10
Figure 10. Figure 10: Non-Dissonant updates with GPT2-XL compared to small, a different visualization. Scatter plot of old (x) vs new (y) knowledge during non-dissonant updates. Same conditions as in [PITH_FULL_IMAGE:figures/full_fig_p026_10.png]
Figure 11
Figure 11. Figure 11: Dissonant updates with GPT2-small - impact of the number of conflicting facts. Each row represents a distinct metric: accuracy on the Generalization side dataset (paraphrased versions of the new facts), accuracy on New Knowledge, and Accuracy on old unrelated knowledg…
Figure 12
Figure 12. Figure 12: Dissonant updates with GPT2-XL: whether the model learns new knowledge or not, old unrelated knowledge is severely destroyed regardless of the strategy Experiments with 1000 facts using the best learning rate we found for Full Finetuning. Interestingly, this degradati…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ScienceMeter: Tracking Scientific Knowledge Updates in Language Models

    cs.CL 2025-05 reject novelty 6.0 of 10

    ScienceMeter evaluates language model knowledge updates across three axes, preservation of old scientific claims, acquisition of new claims, and projection to future findings, and finds all current methods fall short.

Reference graph

Works this paper leans on

42 extracted references · 19 canonical work pages · cited by 1 Pith paper

  1. [1]

    Continual learning with global prototypes: Beyond the scope of task supervision

    Xueying Bai, Jinghuan Shang, Yifan Sun, and Niranjan Balasubramanian. Continual learning with global prototypes: Beyond the scope of task supervision. NeurIPS, 2024

    work page 2024

  2. [2]

    Continual learning with the neural tangent ensemble

    Ari Benjamin, Christian-Gernot Pehle, and Kyle Daruwalla. Continual learning with the neural tangent ensemble. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 58816--58840. Curran Associates, Inc., 2024. URL https://proceedings.neurips.cc/pap...

    work page 2024

  3. [3]

    A functional role for adult hippocampal neurogenesis in spatial pattern separation

    Claire D Clelland, Minee Choi, CCGJ Romberg, GD Clemenson Jr, Alexandra Fragniere, Pamela Tyers, S Jessberger, LM Saksida, RA Barker, FH Gage, et al. A functional role for adult hippocampal neurogenesis in spatial pattern separation. Science, 325 0 (5937): 0 210--213, 2009

    work page 2009

  4. [4]

    Dissonance arousal: physiological evidence

    Robert T Croyle and Joel Cooper. Dissonance arousal: physiological evidence. Journal of personality and social psychology, 45 0 (4): 0 782, 1983

    work page 1983

  5. [5]

    Knowledge neurons in pretrained transformers

    Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neurons in pretrained transformers. arXiv preprint arXiv:2104.08696, 2021. https://arxiv.org/abs/2104.08696

    arXiv 2021

  6. [6]

    Editing factual knowledge in language models

    Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. arXiv preprint arXiv:2104.08164, 2021. https://arxiv.org/pdf/2104.08164.pdf

    arXiv 2021

  7. [7]

    Addressing loss of plasticity and catastrophic forgetting in continual learning

    Mohamed Elsayed and A Rupam Mahmood. Addressing loss of plasticity and catastrophic forgetting in continual learning. arXiv preprint arXiv:2404.00781, 2024

    arXiv 2024

  8. [8]

    A theory of cognitive dissonance row

    Leon Festinger. A theory of cognitive dissonance row. Peterson and company, 1957

    work page 1957

Show all 42 references
  1. [9]

    The lottery ticket hypothesis: Finding sparse, trainable neural networks

    Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprint arXiv:1803.03635, 2018

    2018 arXiv

  2. [10]

    Transformer feed-forward layers are key-value memories

    Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. arXiv preprint arXiv:2012.14913, 2020. https://arxiv.org/abs/2012.14913

    2012 arXiv

  3. [11]

    Disentangling and mitigating the impact of task similarity for continual learning

    Naoki Hiratani. Disentangling and mitigating the impact of task similarity for continual learning. arXiv preprint arXiv:2405.20236, 2024

    2024 arXiv

  4. [12]

    Knowledge in superposition: Unveiling the failures of lifelong knowledge editing for large language models

    Chenhui Hu, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. Knowledge in superposition: Unveiling the failures of lifelong knowledge editing for large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 24086--24094, 2025

    2025

  5. [13]

    Reasons and solutions for the decline in model performance after editing

    Xiusheng Huang, Jiaxiang Liu, Yequan Wang, and Kang Liu. Reasons and solutions for the decline in model performance after editing. arXiv preprint arXiv:2410.23843, 2024

    2024 arXiv

  6. [14]

    Clap4clip: Continual learning with probabilistic finetuning for vision-language models

    Saurav Jha, Dong Gong, and Lina Yao. Clap4clip: Continual learning with probabilistic finetuning for vision-language models. arXiv preprint arXiv:2403.19137, 2024

    2024 arXiv

  7. [15]

    Vector quantization prompting for continual learning

    Li Jiao, Qiuxia Lai, Yu Li, and Qiang Xu. Vector quantization prompting for continual learning. arXiv preprint arXiv:2410.20444, 2024

    2024 arXiv

  8. [16]

    Overcoming catastrophic forgetting in neural networks

    James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of scien...

    2017

  9. [17]

    Auditory critical periods: a review from system’s perspective

    Andrej Kral. Auditory critical periods: a review from system’s perspective. Neuroscience, 247: 0 117--133, 2013

    2013

  10. [18]

    Incremental learning of retrievable skills for efficient continual task adaptation

    Daehee Lee, Minjong Yoo, Woo Kyung Kim, Wonje Choi, and Honguk Woo. Incremental learning of retrievable skills for efficient continual task adaptation. arXiv preprint arXiv:2410.22658, 2024

    2024 arXiv

  11. [19]

    Continual learning in the presence of spurious correlations: Analyses and a simple baseline

    Donggyu Lee, Sangwon Jung, and Taesup Moon. Continual learning in the presence of spurious correlations: Analyses and a simple baseline. In The Twelfth International Conference on Learning Representations

  12. [20]

    Unveiling the pitfalls of knowledge editing for large language models

    Zhoubo Li, Ningyu Zhang, Yunzhi Yao, Mengru Wang, Xi Chen, and Huajun Chen. Unveiling the pitfalls of knowledge editing for large language models. arXiv preprint arXiv:2310.02129, 2023

    2023 arXiv

  13. [21]

    Gradient episodic memory for continual learning

    David Lopez-Paz and Marc'Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017

    2017

  14. [22]

    Packnet: Adding multiple tasks to a single network by iterative pruning

    Arun Mallya and Svetlana Lazebnik. Packnet: Adding multiple tasks to a single network by iterative pruning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 7765--7773, 2018

    2018

  15. [23]

    Locating and editing factual associations in gpt

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems, 35: 0 17359--17372, 2022 a

    2022

  16. [24]

    Locating and editing factual associations in gpt

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems, 35: 0 17359--17372, 2022 b

    2022

  17. [25]

    Mass-editing memory in a transformer

    Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory in a transformer. arXiv preprint arXiv:2210.07229, 2022 c . https://arxiv.org/pdf/2210.07229.pdf

    2022 arXiv

  18. [26]

    Fast model editing at scale

    Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning. Fast model editing at scale. In International Conference on Learning Representations, 2022. URL https://openreview.net/pdf?id=0DcZxeWfOPt

    2022

  19. [27]

    Young dentate granule cells mediate pattern separation, whereas old granule cells facilitate pattern completion

    Toshiaki Nakashiba, Jesse D Cushman, Kenneth A Pelkey, Sophie Renaudineau, Derek L Buhl, Thomas J McHugh, Vanessa Rodriguez Barrera, Ramesh Chittajallu, Keisuke S Iwamoto, Chris J McBain, et al. Young dentate granule cells mediate pattern separation, whereas old granule cells ...

    2012

  20. [28]

    Scalable language model with generalized continual learning

    Bohao Peng, Zhuotao Tian, Shu Liu, Mingchang Yang, and Jiaya Jia. Scalable language model with generalized continual learning. arXiv preprint arXiv:2404.07470, 2024

    2024 arXiv

  21. [29]

    icarl: Incremental classifier and representation learning

    Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 2001--2010, 2017

    2001

  22. [30]

    Progressive neural networks

    Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.04671, 2016

    2016 arXiv

  23. [31]

    Train-attention: Meta-learning where to focus in continual knowledge learning

    Yeongbin Seo, Dongha Lee, and Jinyoung Yeo. Train-attention: Meta-learning where to focus in continual knowledge learning. arXiv preprint arXiv:2407.16920, 2024

    2024 arXiv

  24. [32]

    Overcoming catastrophic forgetting with hard attention to the task

    Joan Serra, Didac Suris, Marius Miron, and Alexandros Karatzoglou. Overcoming catastrophic forgetting with hard attention to the task. In International conference on machine learning, pages 4548--4557. PMLR, 2018

    2018

  25. [33]

    Massive editing for large language models via meta learning

    Chenmien Tan, Ge Zhang, and Jie Fu. Massive editing for large language models via meta learning. arXiv preprint arXiv:2311.04661, 2023

    2023 arXiv

  26. [34]

    Neural activity predicts attitude change in cognitive dissonance

    Vincent Van Veen, Marie K Krug, Jonathan W Schooler, and Cameron S Carter. Neural activity predicts attitude change in cognitive dissonance. Nature neuroscience, 12 0 (11): 0 1469--1474, 2009

    2009

  27. [35]

    Easyedit: An easy-to-use knowledge editing framework for large language models

    Peng Wang, Ningyu Zhang, Xin Xie, Yunzhi Yao, Bozhong Tian, Mengru Wang, Zekun Xi, Siyuan Cheng, Kangwei Liu, Guozhou Zheng, et al. Easyedit: An easy-to-use knowledge editing framework for large language models. arXiv preprint arXiv:2308.07269, 2023

    2023 arXiv

  28. [36]

    A unified and general framework for continual learning

    Zhenyi Wang, Yan Li, Li Shen, and Heng Huang. A unified and general framework for continual learning. arXiv preprint arXiv:2403.13249, 2024

    2024 arXiv

  29. [37]

    Single-cell responses in striate cortex of kittens deprived of vision in one eye

    Torsten N Wiesel and David H Hubel. Single-cell responses in striate cortex of kittens deprived of vision in one eye. Journal of neurophysiology, 26 0 (6): 0 1003--1017, 1963

    1963

  30. [38]

    Advancing cross-domain discriminability in continual learning of vison-language models

    Yicheng Xu, Yuxin Chen, Jiahao Nie, Yusong Wang, Huiping Zhuang, and Manabu Okumura. Advancing cross-domain discriminability in continual learning of vison-language models. arXiv preprint arXiv:2406.18868, 2024

    2024 arXiv

  31. [39]

    Lifelong learning with dynamically expandable networks

    Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547, 2017

    2017 arXiv

  32. [40]

    Continual learning of context-dependent processing in neural networks

    Guanxiong Zeng, Yang Chen, Bo Cui, and Shan Yu. Continual learning of context-dependent processing in neural networks. Nature Machine Intelligence, 1 0 (8): 0 364--372, 2019

    2019

  33. [41]

    Safe: Slow and fast parameter-efficient tuning for continual learning with pre-trained models, 2024

    Linglan Zhao, Xuerui Zhang, Ke Yan, Shouhong Ding, and Weiran Huang. Safe: Slow and fast parameter-efficient tuning for continual learning with pre-trained models, 2024. URL https://arxiv.org/abs/2411.02175

    2024 arXiv

  34. [42]

    Modifying memories in transformer models

    Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix Yu, and Sanjiv Kumar. Modifying memories in transformer models. arXiv preprint arXiv:2012.00363, 2020. https://arxiv.org/pdf/2012.00363.pdf

    2012 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.