{"id":"2503d823-71dc-4996-b447-f1741870c835","arxiv_id":"2506.15076","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"How knowledge is encoded during LLM fine-tuning strongly affects later unlearning: paraphrased training data helps unlearning, while entangled chunks hinder selective forgetting.","lead":"This paper studies whether the way a fact is written during fine-tuning changes how easily that fact can later be erased from an LLM. It finds that training on several paraphrases makes facts easier to unlearn, while packing facts into long text chunks makes selective unlearning much harder.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"FT-Single vs FT-Mul is confounded by 3x training examples and, for UL-Exact, 3x forget examples, so the 'encoding' claim is not yet isolated.","rationale":"The paper is carefully designed around synthetic knowledge, and the chunk-entanglement finding has internal control in the TOFU+ versus Eval-DU+ comparison. However, the first and most general finding—the practical recommendation to paraphrase—rests on a contrast that cannot distinguish encoding from exposure. The authors state in Section 3.2 that the number of paraphrases is 3 in FT-Mul but do not match count or tokens in FT-Single. They also reuse exact training texts for UL-Exact, so the unlearning data size differs. This is not just a statistical-noise issue; it is a structural confound in the independent variable. A single additional control condition would settle it. Because the paper's novelty and practical strategy I both depend on this comparison, the central first claim should not be accepted as established. The rest of the paper (chunk entanglement, split alignment, isolation) is less affected by this particular confound, and those findings remain suggestive, so a conditional posture is still appropriate. The reader's CONDITIONAL verdict is therefore unchanged, but the required experiment is now clearly specified.","tokens_in":25006,"tokens_out":7160,"duration_ms":75787,"concrete_test":"Create a control condition FT-Single-3x: for each fact, repeat the same single description three times to match FT-Mul's number of training examples, tokens, and per-epoch gradient steps (same learning rate and epochs); similarly, for UL-Exact use three copies of the identical description so the forget-set size matches FT-Mul's. Compare Norm-AUC/AUC on Eval-DU+ with Llama2-7B under all six unlearning configurations. If FT-Single-3x reaches the same unlearning performance as FT-Mul, the reported advantage is a data-volume artifact, not evidence for paraphrase-based encoding. If FT-Mul still wins, rerun with multiple seeds and report means and confidence intervals; if the advantage vanishes with error bars, the claim is statistically unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's first finding—that FT-Mul (multiple paraphrases per fact) unlearns better than FT-Single—is presented as evidence that learning-time knowledge encoding shapes unlearning. But FT-Mul differs from FT-Single in at least two variables besides encoding. Section 3.2 fixes the number of paraphrases at 3 for FT-Mul while FT-Single is one description; fine-tuning uses the same learning rate and epochs (Table 9), so FT-Mul receives 3x the training examples, 3x the tokens, and 3x the optimizer updates per epoch. Table 2 shows FT-Mul also reaches higher test probabilities (e.g., 0.68 vs 0.47 for Llama2/Eval-DU+), exactly what one would expect from more data. Second, for the UL-Exact unlearning configuration (a third of all comparisons in Table 3), Section 2.2 defines UL-Exact as the exact training texts; hence for FT-Mul it contains three descriptions per target fact versus one for FT-Single, so the unlearning algorithm also sees 3x more forget data. Either confound alone could raise Norm-AUC without any effect of paraphrase diversity. The 44/48 win rate is therefore not a clean test of the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies whether the way knowledge is textually encoded during fine-tuning affects the later efficacy of post-hoc unlearning. The authors construct augmented versions of Eval-DU and TOFU, fine-tune Llama2-7B, Llama3-8B, and Gemma2-2B on three encodings (FT-Single, FT-Mul, FT-Mul-Chunk), and then apply gradient ascent and task-vector unlearning with three choices of unlearning data (UL-Exact, UL-Single, UL-Mul). They report two main findings: (1) fine-tuning with multiple paraphrased descriptions (FT-Mul) improves unlearning effectiveness relative to a single description, and (2) unlearning individual knowledge embedded in a chunk that also contains retained knowledge is significantly harder, with supporting hypotheses that chunk-aligned unlearn splits and isolated knowledge descriptions make unlearning easier. The paper proposes two practical data-design strategies, paraphrasing and separating, and releases code.","tokens_in":25179,"tokens_out":5689,"duration_ms":60542,"significance":"If the claims are valid, the paper would be a useful early contribution showing that training-time textual encoding is a controllable factor in post-hoc unlearning, with concrete implications for privacy and data curation. The strengths of the design are real: the use of fictitious knowledge bases controls the underlying facts; the experiments span three model families, two datasets, two unlearning algorithms, and three unlearning-data constructs; the evaluation includes held-out paraphrases; the chunk-entanglement hypotheses are tested with internal controls (aligned splits and isolated sentences); and the code is publicly released. However, the headline first finding rests on a confounded comparison, and the quantitative support is weakened by the absence of error bars and significance tests. These issues are load-bearing for the central claim and require additional experiments before the paper can be accepted.","major_comments":[{"comment":"The FT-Single versus FT-Mul comparison does not isolate knowledge encoding. Section 3.2 states that the number of paraphrases is 3 in FT-Mul and 1 in FT-Single, while Table 9 shows the same learning rate and number of epochs for both. Consequently, FT-Mul receives three times as many training examples, three times as many tokens, and three times as many optimizer updates per epoch, and Table 2 indeed shows higher test probabilities for FT-Mul. Furthermore, for the UL-Exact configurations (8 of the 48 entries in Table 3), Section 2.2 defines Dul as the exact fine-tuning texts, so FT-Mul also has three times as many forget examples. Both confounds could produce the reported 44/48 advantage in Table 3 without any effect of paraphrase diversity. The authors should add a control such as FT-Single with the same single description repeated three times, or otherwise match the number of training examples and total tokens across conditions, and for UL-Exact match the size of the forget set; if FT-Mul still dominates, the claim would be supported.","section":"§3.2, §2.2, Table 9, Table 3"},{"comment":"The quantitative evidence for the paper's main directional claim is thin because all comparisons appear to be single runs with no error bars, standard deviations, or significance tests, and many reported differences are very small. For example, in Table 3 the Llama2-7B/Eval-DU+ GA UL-Single row shows Norm-AUC 0.62 versus 0.63 for FT-Single versus FT-Mul, and several AUC differences are 0.01, while some entries favor FT-Single. The statement that FT-Mul 'outperforms or matches' FT-Single in 44/48 cases depends on how ties are counted and is not a substitute for statistical evidence. The authors should run multiple seeds, report variances, and use paired tests across the unlearn-retain configurations, or explicitly argue why single runs are sufficient at these effect sizes.","section":"Table 3, §4.1"},{"comment":"The text in Section 4.2 claims that unlearning from FT-Mul-Chunk 'consistently results in lower Norm-AUC scores than FT-Single', but Table 4 reports only FT-Mul-Chunk values; there is no table or plot in the main text directly comparing FT-Mul-Chunk with FT-Single under the same evaluation protocol. A cross-table comparison with Table 3 is not controlled because the FT-Mul and FT-Mul-Chunk fine-tuning data differ in both structure and content. The authors should present a matched comparison, for example FT-Mul versus FT-Mul-Chunk with identical unlearning configurations and evaluation prompts, or qualify the claim as a between-experiment observation.","section":"§4.2, Table 4"}],"minor_comments":[{"comment":"The word 'prodcures' should be 'procedures'.","section":"§3.2"},{"comment":"The entry 'Gradieng Ascent' should be 'Gradient Ascent'.","section":"Table 1"},{"comment":"The table header uses 'FT-Mul-Trunk' while the rest of the paper uses 'FT-Mul-Chunk'; the names should be made consistent.","section":"Table 7"},{"comment":"The phrase 'text trunk' appears twice where 'text chunk' is meant, and the appendix has the typo 'prbability'.","section":"Figure 1 and Appendix A"},{"comment":"The code URL in the main text (github.com/...) and the anonymous URL in the appendix (anonymous.4open.science/...) should be reconciled so the reader knows which repository is canonical.","section":"Appendix C"},{"comment":"The legends in these figures mix three curves (FT-Mul-Chunk, FT-Mul-Chunk (New Split), FT-Mul-Chunk-Iso) with labels such as 'FT-Single (New Split)' in the Llama3 panel; the curve labels and captions should be made consistent and unambiguous.","section":"Figures 11-13"}],"recommendation":"major_revision","confidential_remarks":"The paper's first finding is its headline, and the FT-Single/FT-Mul comparison is currently confounded by data quantity and, under UL-Exact, by forget-set size. This is fixable with additional control experiments, so I favor major_revision over rejection. The chunk-entanglement findings and their internal controls are the stronger part of the paper and could be foregrounded while the paraphrase claim is being deconfounded. I would not request pretraining-scale experiments, since the authors clearly scope the study to fine-tuning, but the title and abstract should be tightened to avoid overgeneralizing to all LLMs."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nWhat you should know: this is a genuinely useful empirical study of whether training-time text encoding affects post-hoc unlearning, and the chunk-entanglement half is mostly convincing. But the first headline result—FT-Mul unlearns better than FT-Single—is not yet clean, because FT-Mul differs from FT-Single by more than encoding. It has three times as many training examples per fact, and under UL-Exact, the unlearning algorithm is also given three times as many forget examples. The stress-test note lands: either difference could raise Norm-AUC without paraphrase diversity doing anything. This is the main thing to fix.\n\nWhat is new and good: prior unlearning work treats the trained model as fixed; this paper varies how the same facts are written, on synthetic biographies where memorization is controlled. The Eval-DU+ and TOFU+ extensions are sensible, code is released, and the design spans three model families, two datasets, two algorithms, and three unlearning-data choices. The chunk section is the stronger contribution. FT-Mul-Chunk gives Norm-AUC near 0.5 on Eval-DU+ while TOFU+ is higher, and the two follow-up manipulations—aligning the unlearn split with chunk boundaries, and isolating descriptions within chunks—both improve scores. Those comparisons are cleaner and support the practical advice to separate likely forget and retain content in training data. Citation pattern looks fine; the self-citation to Eval-DU is relevant.\n\nSoft spots: the FT-Single/FT-Mul comparison is confounded as above; all runs are single-run with no error bars or significance tests, and many differences are small (e.g., 0.59 vs 0.60). That is addressable but should be stated as directional. The Chunk-Iso table omits UL-Exact, which is a missing baseline for Hypothesis 2. The paper's own limitations section is candid about fine-tuning-only scope and lack of pretraining validation.\n\nWho it is for: people working on machine unlearning benchmarks and training-data design. With a matched-example control (e.g., three copies of the same single description per fact) and repeated runs, the paraphrasing claim would be worth serious attention. As is, it deserves a serious referee—conditional acceptance is right—but the first finding needs rework before the stronger conclusions can stand.\n\nRecommendation: send it to review; ask for confound control and uncertainty quantification.","headline":"A useful study of how chunk structure affects unlearning, but the paraphrasing result is confounded by 3x training data and, for UL-Exact, 3x forget data.","tokens_in":25796,"tokens_out":3463,"would_cite":true,"duration_ms":35330,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that how a fact is written during fine-tuning—one description, several paraphrases, or a shared text chunk—determines how effectively that fact can later be unlearned from the model.","keywords":["machine unlearning","large language models","knowledge encoding","paraphrasing","fine-tuning","unlearn-retain trade-off","text chunks","factual knowledge"],"falsifier":"Train two additional models on the same knowledge set: one with each fact written once, and one with the identical single description repeated three times per fact. If the repeated model shows the same Norm-AUC advantage over the single model that FT-Mul shows, then paraphrase diversity is not the cause of the reported effect. For the chunk result, a version that shuffles the same facts across chunks while preserving boundary sizes would test whether content alignment or chunk size drives the entanglement finding.","tokens_in":24705,"feed_emoji":"🧹","tokens_out":9827,"duration_ms":93048,"temperature":0.7,"pith_summary":"This paper asks whether the way a fact is written during fine-tuning changes how hard it is to erase that fact later, independently of the unlearning algorithm. Using synthetic biography datasets where the same knowledge can be written as a single description, several paraphrases, or a multi-fact chunk, it finds that training with multiple paraphrased descriptions produces consistently better unlearn–retain trade-offs than training with a single description. It also finds that a fact sharing a text chunk with facts that must be kept is hard to remove selectively: forget and retain scores fall at nearly the same rate, and unlearning improves when the forget set matches chunk boundaries or when facts are written as separate sentences inside the chunk. If the findings hold, choosing how to write data at training time becomes a practical lever for later deletion requests, and unlearning benchmarks that fix the trained model ignore half of the problem.","feed_headline":"Paraphrase training text so LLMs can forget facts later","feed_subtitle":"Multiple phrasings ease later erasure; facts tangled in shared paragraphs barely budge.","key_machinery":"The carrying mechanism is a controlled fine-tuning testbed built by extending two unlearning datasets (Eval-DU+ and TOFU+) with fictitious subjects, so the same atomic facts can be encoded as a single description, as three paraphrases, or inside multi-fact chunks. The comparison instrument is the unlearn–retain trade-off curve: for two unlearning algorithms (gradient ascent and task vector) and three choices of unlearning text (the exact training text, one unseen paraphrase, or several unseen paraphrases), checkpoints of increasing unlearning strength are scored by token-level probability on the forget set and the retain set. Norm-AUC is the summary statistic: it normalizes scores by the fine-tuned model's starting values and computes the area under the curve, with 0.5 indicating that forget and retain are falling at equal rates. The second half of the argument relies on two constructed interventions—a forget split aligned to chunk boundaries and an isolated-chunk variant built by concatenating sentence-level descriptions—to show that entanglement, not chunk size or model family, is what blocks selective unlearning.","core_discovery":"On the paper's own terms, the central discovery is that learning-time textual encoding is a first-order determinant of post-hoc factual unlearning in fine-tuned LLMs. Empirically, fine-tuning each knowledge piece with three paraphrased descriptions (FT-Mul) matches or beats single-description training (FT-Single) in 44 of 48 extraction-trade-off comparisons and 39 of 48 memorization-trade-off comparisons across three model families and two datasets, while also making the model less able to answer unseen paraphrased prompts. When the same knowledge is embedded in multi-fact chunks, unlearning an individual fact becomes markedly harder: Norm-AUC values hover near 0.5 on the entangled dataset, meaning unlearning removes target and retained knowledge at the same rate. Two interventions confirm the mechanism the paper proposes—aligning the unlearn split with chunk boundaries and isolating each fact as its own sentence both raise Norm-AUC consistently. The paper extracts two training-time strategies from these results: paraphrase each fact during fine-tuning, and structure data so likely unlearn targets are separated from retained content.","pith_inferences":["The cleanest challenge to the first finding is a repetition confound: FT-Mul differs from FT-Single in paraphrase variety and in having three training examples per fact. A condition that repeats the identical description three times would tell whether diversity or sheer count drives the unlearning gain.","If the effect extends to pretraining, corpus-level surface-form diversity and duplication become compliance-relevant data-curation choices, not just memorization correlates; that is a much larger claim than the fine-tuning evidence in this paper supports.","The chunk-entanglement result suggests a concrete mechanism worth testing directly: gradient-ascent and task-vector updates may attenuate shared parameter directions or overlapping representations, so measuring gradient overlap between forget and retain facts in the same chunk could predict unlearning failure.","The separation strategy could be automated with a detector that marks likely deletion targets at curation time, but the paper only proposes the strategy, not the detector."],"forward_implications":["If a deployment expects deletion requests, adding paraphrases of each sensitive fact to the fine-tuning set will make later gradient-ascent or task-vector unlearning more effective for the same retention of unrelated knowledge.","Training chunks that weave forget-prone and retained facts together are a liability: single-fact erasure will degrade both, so separating those facts into standalone text is the safer layout.","Choosing the to-forget set to align with natural chunk boundaries in the training data converts a near-0.5 Norm-AUC unlearning run into a measurably more selective one.","Unlearning algorithm comparisons are only meaningful relative to a training encoding; a benchmark that fixes the trained model cannot observe the differences this paper reports."],"supporting_citations":[{"why":"Supplies the Eval-DU factual-triple knowledge base and initial descriptions that Eval-DU+ extends.","marker":"[17]"},{"why":"Supplies the TOFU fictitious-author QA dataset and its original unlearn–retain split that TOFU+ extends.","marker":"[19]"},{"why":"Provides prior evidence that paraphrased encodings change how robustly LLMs internalize knowledge, motivating the FT-Single versus FT-Mul comparison.","marker":"[28]"},{"why":"Supports the premise that different textual encodings influence both memorization and extraction of knowledge.","marker":"[27]"},{"why":"Defines the gradient ascent unlearning algorithm used as one of the two unlearning methods.","marker":"[9]"},{"why":"Defines the task vector unlearning method used as the other unlearning algorithm.","marker":"[23]"},{"why":"Motivates the UL-Single and UL-Mul choices of unlearning data as defenses against extraction attacks.","marker":"[18]"},{"why":"Supplies the prior observation that entanglement of retain and unlearn content in prompts degrades unlearning evaluations, which informs the chunk experiments.","marker":"[55]"}],"fun_headline_variants":["Training-time paraphrase improves LLM unlearning","How you encode facts during training affects unlearning","LLM unlearning depends on learning-time wording","Paraphrased facts are easier to unlearn from LLMs","Chunked knowledge blocks LLM unlearning; paraphrase helps"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that paraphrasing helps unlearning rests on comparing training with one description of each fact to training with several paraphrases; but the paraphrased condition also uses three times as many example sentences per fact, so the observed gain could be caused by repetition or data quantity rather than by variation in wording.","fun_headline_variants_meta":{"raw":{"variants":["Training-time paraphrase improves LLM unlearning","How you encode facts during training affects unlearning","LLM unlearning depends on learning-time wording","Paraphrased facts are easier to unlearn from LLMs","Chunked knowledge blocks LLM unlearning; paraphrase helps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000366,"raw_usage":{"total_tokens":1938,"prompt_tokens":885,"completion_tokens":1053,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":501,"completion_tokens_details":{"reasoning_tokens":976}},"tokens_in":501,"tokens_out":1053,"duration_ms":9228,"temperature":1.0,"reasoning_tokens":976,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:44:20.407968+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train two additional models on the same knowledge set: one with each fact written once, and one with the identical single description repeated three times per fact. If the repeated model shows the same Norm-AUC advantage over the single model that FT-Mul shows, then paraphrase diversity is not the cause of the reported effect. For the chunk result, a version that shuffles the same facts across chunks while preserving boundary sizes would test whether content alignment or chunk size drives the entanglement finding.","supporting_citations":[{"cited_title":"Tofu: A task of fictitious unlearning for llms","cited_arxiv_id":null,"evidence_quote":"Supplies the TOFU fictitious-author QA dataset and its original unlearn–retain split that TOFU+ extends."},{"cited_title":"Physics of language models: Part 3.2, knowledge manipula- tion","cited_arxiv_id":null,"evidence_quote":"Provides prior evidence that paraphrased encodings change how robustly LLMs internalize knowledge, motivating the FT-Single versus FT-Mul comparison."},{"cited_title":"Physics of language models: Part 3.1, knowledge storage and extraction","cited_arxiv_id":null,"evidence_quote":"Supports the premise that different textual encodings influence both memorization and extraction of knowledge."},{"cited_title":"Can sensitive information be deleted from llms? objectives for defending against extraction attacks","cited_arxiv_id":null,"evidence_quote":"Motivates the UL-Single and UL-Mul choices of unlearning data as defenses against extraction attacks."}],"review_version":2}