{"id":"5286cdaf-e48f-4ad9-bc32-d703a0ed7bf0","arxiv_id":"2411.11932","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Continually trained LLMs can recover near-full performance on supposedly forgotten tasks with the right instruction-level prompts, indicating pseudo forgetting rather than erased capabilities.","lead":"The paper argues that many apparent cases of catastrophic forgetting in large language models are not true knowledge loss: the old skill is still in the model, but the instruction no longer activates it. It supports this with prompt-level recovery experiments and proposes a replay method that spends old-task examples where this pseudo forgetting is worst.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The GCG suffix-recovery experiment (Section 2.1) lacks a control for search artifacts: a per-sample adversarially optimized suffix that forces a target rationale cannot by itself prove old-task capability is retained, especially since one target is the ground-truth answer.","rationale":"The paper's central conceptual claim—that apparent forgetting can be an instruction-activation failure—is worth testing, and the replay experiments provide some independent value. However, the headline evidence in Section 2.1 does not currently distinguish capability retention from GCG search artifacts. The reader's weakest-assumption identification is correct: the recovery experiment's validity rests on the target tokens and the suffix being information-free, and the only check (GPT4o) is a subjective proxy that tests the partial rationale, not the suffix. I would sharpen this further: because GCG is an adversarial optimization procedure, the per-sample optimization itself is a confound. A model that has 'forgotten' an old task may still be steered to emit arbitrary target text by an optimized suffix; the high recovery rate is therefore not direct evidence of retained capability. The 'Answer' optimization target in Table 8 is particularly problematic, as it optimizes the suffix to make the model produce the correct answer, making recovery on that target tautological. The proposed wrong-target control would settle this: if the same search can force incorrect rationales/answers at similar rates, then the correct-target recovery is not diagnostic. Until such a control is run, the central claim should be treated as plausible but unproven, which is consistent with the reader's conditional accept recommendation.","tokens_in":17127,"tokens_out":11427,"duration_ms":123414,"concrete_test":"Run the identical GCG recovery procedure on the same forgetting model and same 100 forgotten samples, but with a target that is the first 20% of an incorrect rationale (or a wrong-answer prefix) for the same input. Measure the rate at which the model then produces the wrong final answer. If this wrong-target recovery rate is as high as the reported ~90% correct recovery, the experiment shows only that GCG can force target content, not that the old-task capability is retained; if the wrong-target rate is near chance while the correct-target rate stays ~90%, the pseudo-forgetting interpretation is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central evidence for pseudo forgetting is the A2 recovery experiment (Section 2.1, Eqs. 1-3). For each forgotten sample, GCG searches a suffix S_i that minimizes the loss of generating a target T—either the ground-truth answer, the first 20% of the ground-truth rationale, or the first 20% of the pre-forgetting rationale (Table 8)—conditioned on [instruction, S_i]. Recovery rate (Eq. 3) then measures whether M_{a-f}([I_i, S_i]) produces A_i. This protocol conflates 'the model has the old capability' with 'GCG can find an adversarial trigger that forces the desired output.' The 'Answer' target is tautological: the suffix is optimized to make the model emit the correct answer, so near-100% recovery on that target is expected and proves nothing about retained ability. For the partial-rationale targets, the load-bearing premise is that the first 20% of the rationale and the suffix carry no task-relevant information. The only check (Appendix C.1, Table 9) is a GPT4o judgment that the partial rationale 'directly leads' to the answer in only ~5% of cases; this is a subjective proxy, not a certificate, and it does not test the suffix at all. 'Does not directly lead' is weaker than 'contains no task-relevant information.' Since the suffix is optimized per sample on the same model, the high recovery rate could reflect the model's susceptibility to prompt optimization rather than an intact old-task capability. A non-optimized control (natural instruction reformulations) or a wrong-target control is needed before the 90% recovery can be attributed to capability retention.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces and empirically argues for 'pseudo forgetting' in continual learning of LLMs: the observed performance drop on previously learned tasks is claimed to stem not from a loss of task capabilities, but from the original instructions failing to activate those intact capabilities. The evidence is threefold: (A1) providing the first k tokens of the ground-truth rationale partially restores task accuracy; (A2) appending per-sample GCG-searched suffixes to the original instructions yields recovery rates above 90% on forgotten samples; and (2.2) attribution analysis shows reduced instruction dependence during rationale generation after continual learning. Based on this diagnosis, the paper proposes the Rationale-Guidance Difficulty (RGD) metric and an RGD-based replay allocation strategy (RGD-R), evaluated on the Long Sequence Benchmark across several LLM families and sizes.","tokens_in":17419,"tokens_out":6868,"duration_ms":65214,"significance":"If the central claim is established, the paper would meaningfully refine the understanding of catastrophic forgetting in LLMs by shifting attention from parametric erasure to instruction-activation failures, with practical consequences for replay-based continual learning. The manuscript has clear strengths: it attacks the problem with multiple complementary analyses, evaluates across five model sizes/families, includes attribution-based mechanistic evidence, and reports a replay-allocation method with code and data availability. However, the strongest direct evidence for pseudo forgetting rests on the GCG suffix-recovery experiment, which currently lacks the controls needed to distinguish retained capability from susceptibility to per-sample adversarial optimization. The theoretical derivation of the RGD metric is also not mathematically sound as written. The phenomenon is plausible and the paper is worth publishing after substantial revision, but the central claim is not yet convincingly supported.","major_comments":[{"comment":"The GCG suffix-recovery experiment conflates retained capability with susceptibility to per-sample adversarial optimization. Because S_i is optimized to minimize the loss of the target T (Eq. 2) and the recovery rate (Eq. 3) is evaluated on the same (I_i, A_i) used in that optimization, the 'Answer' target is close to a tautology: a high success rate is expected whenever GCG can find any token string that steers the model, regardless of whether the old-task capability is intact. The partial-rationale targets inherit this concern unless it is verified that neither the first 20% of the rationale nor the optimized suffix carries task-relevant information. The manuscript provides no control with a random or fixed non-optimized suffix, no wrong-target control (e.g., a suffix optimized to force an incorrect answer), no transfer test of suffixes across samples of the same task, and no automated check of suffix semantics. The suffixes shown in Table 10 already contain meaningful English fragments such as 'Sounds striking' and 'phony provisions'. These controls are needed before the 90% recovery rate can be read as evidence for retained parametric capability.","section":"Section 2.1, A2 (Eqs. 1-3)"},{"comment":"The only leakage check is a GPT4o judgment on the partial rationale, not on the suffix and not on the combined prompt [I, S_i]. The criterion 'directly leads to the correct answer' is weaker than 'contains no task-relevant information', and Table 9 reports that at k=0.2, 3-6% of partial rationales do leak key information. The paper's statement in Appendix C.1 that it uses the k value that 'does not leak any information' is therefore inaccurate. This matters because the A2 experiment's load-bearing premise is that recovered predictions reflect parametric capability rather than leaked cues.","section":"Appendix C.1, Table 9"},{"comment":"The theoretical derivation of the RGD metric is not mathematically sound. Eq. (9) writes Pθ(c∗|i)=p(c1,...,cm|i)=Σ_m p(cm|i), conflating the joint probability over capabilities with a marginal sum; Eq. (10) similarly writes Pθ(r∗)=p(r∗|c1,...,cn)=Σ_n p(r∗|cn). Eq. (11) does not follow from the independence assumption in Eq. (8): setting cross terms p(r|cn)p(cm|i)=0 for m≠n does not imply the factorization Pθ(r∗|i)=(Σ_n p(r∗|cn))(Σ_m p(cm|i)). Because this proof is presented as the justification for the RGD score, it should be corrected or replaced by a direct empirical validation of RGD as a heuristic.","section":"Section 3.2, Eqs. (7)-(12)"},{"comment":"The RGD score divides PPL_{a-f}(Rg|I) by PPL_{b-f}(Rg), where the denominator is not conditioned on the instruction I. To measure instruction-activation difficulty, the natural baseline is PPL_{b-f}(Rg|I) or a matched control; using the unconditional probability means the ratio can change merely because of domain-shift effects on token probabilities, independent of any instruction-activation failure. This weakens the construct validity of the metric, although the empirical replay results could still be useful if the metric is treated as a heuristic.","section":"Section 3.1, Eq. (5)"}],"minor_comments":[{"comment":"There is a typo: 'the model dose not forget' should be 'the model does not forget'.","section":"Section 2.1, Results and Analysis"},{"comment":"The phrase '0.2 portions' is unclear; it should say 'the first 20% of the rationale'.","section":"Figure 5 caption"},{"comment":"The summation notation 'Pi−1 k=1' in the text is a typo; it should be a summation symbol with limits k=1 to i−1.","section":"Section 3.3, Eq. (13)"},{"comment":"Several displayed suffixes are not obviously 'semantically meaningless' (e.g., 'Sounds striking', 'phony provisions'); a quantitative check of semantic content or a human study would strengthen the claim that the suffixes carry no task-relevant information.","section":"Table 10"},{"comment":"The main results report single runs without standard deviations or significance tests. Given the small differences between RGD-R and EA (e.g., Qwen2-0.5B FAP 65.99 vs 64.13; Mistral-7B FAP 74.91 vs 72.15), repeated seeds are needed to establish the stability of the improvement.","section":"Tables 1 and 3"},{"comment":"The code link is given as 'available at here' without an actual URL; a working repository link should be provided.","section":"Section 1"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses an interesting and timely question, and the multi-pronged evidence is a strength. The main concern is that the GCG suffix-recovery experiment, which is the most direct evidence for the paper's central claim, lacks the controls needed to rule out search artifacts and prompt-optimization effects. These issues are addressable with additional experiments (random-suffix controls, wrong-target controls, cross-sample transfer, and suffix semantics evaluation) and with a corrected or softened theoretical section. I therefore recommend major revision rather than rejection. The authors should also clarify the relationship between 'pseudo forgetting' and the known task-inference hypothesis, since the paper's definition overlaps with but is not cleanly distinguished from prior work."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper makes a solid conceptual point: apparent forgetting in continually trained LLMs can reflect failed instruction-capability coupling rather than erased knowledge. The partial-rationale recovery and the attribution analyses are genuinely useful, and they extend Kotha et al.'s 'task inference' hypothesis from synthetic setups to natural-language benchmarks and real LLM scales. The RGD metric and RGD-R replay allocation are simple, well-motivated, and show consistent if modest gains over equal replay across five models. That part deserves a serious read.\n\nThe soft spot is exactly what the stress-test flags. The A2 recovery experiment fits a per-sample GCG suffix to minimize the loss of the target (the answer or the first 20% of a rationale) and then reports high recovery as evidence of retained capability. That conflates 'the model can be steered by an adversarial trigger' with 'the model still knows this task.' The Answer target is nearly tautological, and for the partial-rationale targets the only leakage check is a GPT4o judgment on the rationale prefix, not on the optimized suffix. 'Does not directly lead' is weaker than 'contains no task-relevant information.' A non-optimized control—natural paraphrases, random suffixes, or a wrong-target suffix—would be the right test, and the paper would be much stronger with it.\n\nOther issues are minor. The main tables show single runs without seed-level variance, and the theoretical analysis in Section 3.2 relies on an unverified independence assumption; the proof is really just a restatement of that assumption. The paper's own limitations section is honest about these gaps.\n\nAll told, the pseudo-forgetting framing and the RGD metric are worth taking seriously, and the paper deserves a careful referee who will push for the missing control. I would accept it for peer review, but with an eye toward revision of the recovery experiment before the claim is fully established.","headline":"Pseudo forgetting is a real and useful framing, but the GCG suffix-recovery experiment is the load-bearing weak spot and needs a non-optimized control before the 90% recovery claim is taken at face value.","tokens_in":18017,"tokens_out":2460,"would_cite":true,"duration_ms":25463,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Performance drops in continually trained LLMs are often instruction-activation failures, not forgotten abilities.","keywords":["pseudo forgetting","continual learning","large language models","instruction dependence","catastrophic forgetting","replay-based methods","rationale guidance","GCG suffix"],"falsifier":"A decisive check would be to run the recovery experiment with random tokens in place of the GCG-searched suffix and measure whether recovery collapses; if it does not, the suffix's specific guidance is not what restores performance, and a classifier trained on the first 20% of ground-truth rationales that predicts the answer above chance would show the partial rationale leaks task-relevant cues, defeating the capability-retention reading.","tokens_in":16842,"feed_emoji":"🧠","tokens_out":7790,"duration_ms":70411,"temperature":0.7,"pith_summary":"The paper argues that what looks like catastrophic forgetting in continually trained large language models is often \"pseudo forgetting\": old-task performance drops not because the model has lost the relevant ability, but because the original instruction no longer triggers that ability. To test this, the authors show that a forgetting model can recover old-task accuracy when given a short correct-rationale prefix, and that appending a task-irrelevant token suffix found by automated search restores correct behavior on more than 90% of forgotten examples. Attribution analysis indicates the cause is reduced dependence on instructions during rationale generation, concentrated in shallow layers. Building on this, the paper introduces a Rationale-Guidance Difficulty score and a replay-allocation framework, RGD-R, that gives more replay data to tasks whose instructions are hardest to use; experiments across five model families report improved final accuracy and lower forgetting while keeping forward transfer.","feed_headline":"LLM 'forgetting' is often just a prompt-activation failure","feed_subtitle":"Adding meaningless suffixes to old instructions restores >90% of 'forgotten' accuracy, the paper reports.","key_machinery":"The load-bearing object is the pseudo-forgetting diagnosis itself, operationalized through three instruments: partial-rationale guidance, where the first 20% of the ground-truth rationale is inserted after the assistant token; GCG, Greedy Coordinate Gradient search, which optimizes a short, task-irrelevant suffix so that the original instruction plus suffix drives the model toward a correct rationale; and attribution scores over self-attention layers, computed with integrated gradients, that quantify instruction-to-rationale dependency. The RGD score, $\\mathrm{RGD}(I,R_g,A_g)=\\frac{\\mathrm{PPL}_{a-f}(R_g\\mid I)}{\\mathrm{PPL}_{b-f}(R_g)}$, turns the diagnosis into a per-task number used by RGD-R to set replay ratios. The theoretical link is a conditional-probability decomposition in which the probability of activating the correct capability equals the probability of generating the correct rationale given the instruction divided by its probability without the instruction, so RGD approximates activation difficulty.","core_discovery":"On the paper's own terms, the central discovery is that continual instruction tuning does not erase old-task capabilities; it degrades the instruction-to-capability activation path. Evidence comes from two interventions on models after sequential learning: first, feeding the first k tokens of the correct rationale as external guidance lets the model regenerate the rest of the reasoning and recover pre-forgetting accuracy as k grows; second, searching for a semantically meaningless suffix with gradient-based coordinate search, then appending it to the original instruction, lets the model actively produce the correct rationale and answer, with recovery rates above 90% across tasks. Attribution scores computed by integrated gradients on attention show that pseudo-forgetting models rely less on instruction tokens when generating rationales, especially in shallow layers, which the paper identifies as the mechanism. The paper then defines the RGD score as the ratio of the pseudo-forgetting model's perplexity for the ground-truth rationale under the instruction to the pre-forgetting model's perplexity, and uses it to allocate replay data proportionally to each old task's activation difficulty. RGD-R outperforms equal replay allocation on final average performance, forgetting rate, and backward transfer while preserving forward transfer.","pith_inferences":["Editorial extension: standard continual-learning benchmarks that report only final accuracy may systematically understate retained knowledge, because re-testing old tasks with prompt variants or minimal guidance would likely produce a different forgetting ranking.","Editorial extension: the RGD ratio could be combined with parameter-preservation or layer-specific regularization, targeting the layers where instruction dependence drops most, rather than relying on replay alone.","Editorial extension: a natural stress test is to run the same suffix-recovery experiment on open-ended generation and non-English tasks, since the paper's evidence comes from English classification tasks.","Editorial extension: the fact that arbitrary token suffixes can wake old-task behavior also raises prompt-robustness and controllability questions, as small prompt changes could shift a model between different learned behaviors."],"forward_implications":["If pseudo forgetting is real, a drop in old-task accuracy after continual learning should not be read as evidence that the old ability is gone; the model needs a prompt or guidance that re-activates it.","Replay-based continual learning can be made more efficient by allocating replay samples to tasks where instruction-to-capability activation is hardest, which is what RGD-R does.","Instruction dependence is concentrated in shallow layers, so interventions targeting instruction following in those layers may mitigate forgetting without storing old data.","The suffix-recovery result means old-task performance can be elicited without retraining, purely by prompt modification."],"supporting_citations":[{"why":"Supplies the GCG suffix-search procedure used to build task-irrelevant prompts in the active-recovery experiment.","marker":"(Zou et al., 2023)"},{"why":"Provides the instruction-following decomposition and shallow-layer observation that ground the attribution analysis and theoretical model.","marker":"(Wu et al., 2024)"},{"why":"Proposes the task-inference hypothesis that this paper extends to natural-language LLM continual learning.","marker":"(Kotha et al., 2024)"},{"why":"Offers the instruction-following versus knowledge-loss account whose prefix-completion probing this paper contrasts with its instruction-following evidence.","marker":"(Jiang et al., 2024)"},{"why":"Supplies the attribution algorithm used to compute instruction-to-rationale dependency scores.","marker":"(Li et al., 2024a)"},{"why":"Supports the claim that learned capabilities are stored parametrically, which the paper invokes when arguing recovery reflects retained ability.","marker":"(Dai et al., 2022)"},{"why":"Provides InsCL, the instruction-similarity replay-allocation baseline RGD-R is compared with.","marker":"(Wang et al., 2024)"},{"why":"Provides the Long Sequence Benchmark task order and replay setup used in the RGD-R experiments.","marker":"(Razdaibiedina et al., 2023a)"}],"fun_headline_variants":["LLM 'forgetting' is just a prompt-activation failure","Add meaningless suffixes to old tasks to restore LLM accuracy","Recover forgotten LLM skills: a nonsense suffix does the trick","Old LLM abilities stay, only the instructions fail to trigger them"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the first 20% of the ground-truth rationale and the searched-for meaningless suffix carry no task-relevant information, so a recovered prediction reflects retained parametric ability rather than leaked cues, yet the paper's own automated check finds about 5% of the short rationales do leak key information.","fun_headline_variants_meta":{"raw":{"variants":["LLM 'forgetting' is just a prompt-activation failure","Add meaningless suffixes to old tasks to restore LLM accuracy","Recover forgotten LLM skills: a nonsense suffix does the trick","Old LLM abilities stay, only the instructions fail to trigger them"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000623,"raw_usage":{"total_tokens":2894,"prompt_tokens":962,"completion_tokens":1932,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":578,"completion_tokens_details":{"reasoning_tokens":1871}},"tokens_in":578,"tokens_out":1932,"duration_ms":13754,"temperature":1.0,"reasoning_tokens":1871,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:19:53.465003+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive check would be to run the recovery experiment with random tokens in place of the GCG-searched suffix and measure whether recovery collapses; if it does not, the suffix's specific guidance is not what restores performance, and a classifier trained on the first 20% of ground-truth rationales that predicts the answer above chance would show the partial rationale leaks task-relevant cues, defeating the capability-retention reading.","supporting_citations":[],"review_version":1}