Pith. sign in

REVIEW 10 cited by

Unlearn What You Want to Forget: Efficient Unlearning for LLMs

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2310.20150 v1 pith:CVS32SIJ submitted 2023-10-31 cs.CL cs.AI

Jiaao Chen , Diyi Yang This is my paper · ORCID
classification cs.CLcs.AI
keywords dataunlearningllmsdifferentefficientforgetissueslayers
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large language models (LLMs) have achieved significant progress from pre-training on and memorizing a wide range of textual data, however, this process might suffer from privacy issues and violations of data protection regulations. As a result, the ability to easily remove data related to individual users from such models while not deteriorating their predictive quality after the removal becomes increasingly important. To address these issues, in this work, we propose an efficient unlearning framework that could efficiently update LLMs without having to retrain the whole model after data removals, by introducing lightweight unlearning layers learned with a selective teacher-student objective into the transformers. In addition, we introduce a fusion mechanism to effectively combine different unlearning layers that learns to forget different sets of data to handle a sequence of forgetting operations. Experiments on classification and generation tasks demonstrate the effectiveness of our proposed methods compared to the state-of-the-art baselines.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Maximizing Local Entropy Where It Matters: Prefix-Aware Localized LLM Unlearning

    cs.CL 2026-01 conditional novelty 6.0 of 10

    PALU shows that unlearning only needs local intervention—the first few tokens of the sensitive span and the top-k logits—not full-sequence, full-vocabulary suppression.

  2. Distribution Preference Optimization: A Fine-grained Perspective for LLM Unlearning

    cs.LG 2025-10 conditional novelty 6.0 of 10

    DiPO is a distribution-level unlearning method that constructs preference distributions from the model's own high-confidence logits and achieves state-of-the-art forget quality on TOFU while preserving utility.

  3. Towards Evaluation for Real-World LLM Unlearning

    cs.AI 2025-08 conditional novelty 6.0 of 10

    DCUE evaluates LLM unlearning by comparing core-token confidence score distributions of the unlearned model and the original model, corrected by a validation set, using the Kolmogorov-Smirnov test.

  4. The Safety Gap Toolkit: Evaluating Hidden Dangers of Open-Source Models

    cs.CY 2025-07 conditional novelty 6.0 of 10

    On Llama-3 and Qwen-2.5, removing safety guardrails sharply raises compliance with dangerous bio, chem, and cyber requests, and the resulting safety gap grows with model scale.

  5. Model Collapse Is Not a Bug but a Feature in Machine Unlearning for LLMs

    cs.LG 2025-07 conditional novelty 6.0 of 10

    A new method, Partial Model Collapse, iteratively fine-tunes an LLM on its own self-generated responses to conditionally collapse its output distribution on forget queries, removing private answers without the true la...

  6. Unlearning Isn't Invisible: Detecting Unlearning Traces in LLMs from Model Outputs

    cs.LG 2025-06 conditional novelty 6.0 of 10

    Unlearning in LLMs leaves detectable 'fingerprints' that let a simple classifier distinguish an unlearned model from its original, even on unrelated prompts.

  7. GROM: Gradient-Free Rapid One-Shot Machine Unlearning

    cs.LG 2026-08 conditional novelty 5.0 of 10

    A single closed-form ridge update to selected MLP layers removes targeted knowledge from LLMs in seconds, with state-of-the-art forgetting-utility trade-offs and quantization robustness.

  8. Reliable Unlearning Harmful Information in LLMs with Metamorphosis Representation Projection

    cs.LG 2025-08 unverdicted novelty 5.0 of 10

    An LLM unlearning method that projects hidden states so harmful information is irreversibly removed while useful knowledge is preserved.

  9. Lifting Data-Tracing Machine Unlearning to Knowledge-Tracing for Foundation Models

    cs.CV 2025-06 conditional novelty 4.0 of 10

    A position paper urging a shift from data-tracing to knowledge-tracing machine unlearning for foundation models, supported by a CLIP case study that shows current methods struggle to generalize.

  10. UCD: Unlearning in LLMs via Contrastive Decoding

    cs.CL 2025-06 conditional novelty 4.0 of 10

    UCD steers an LLM away from forget-set content at inference time by mixing in the difference between forget-tuned and retain-tuned small models.

Pith tools