Pith. sign in

REVIEW 8 cited by

Massive Editing for Large Language Models via Meta Learning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2311.04661 v3 pith:PI4J4F2L submitted 2023-11-08 cs.CL cs.LG

Chenmien Tan , Ge Zhang , Jie Fu This is my paper
classification cs.CLcs.LG
keywords editinglanguagefactshyper-networkknowledgemalmenlargelearning
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

While large language models (LLMs) have enabled learning knowledge from the pre-training corpora, the acquired knowledge may be fundamentally incorrect or outdated over time, which necessitates rectifying the knowledge of the language model (LM) after the training. A promising approach involves employing a hyper-network to generate parameter shift, whereas existing hyper-networks suffer from inferior scalability in synchronous editing operation amount. To mitigate the problem, we propose the MAssive Language Model Editing Network (MALMEN), which formulates the parameter shift aggregation as the least square problem, subsequently updating the LM parameters using the normal equation. To accommodate editing multiple facts simultaneously with limited memory budgets, we separate the computation on the hyper-network and LM, enabling arbitrary batch size on both neural networks. Our method is evaluated by editing up to thousands of facts on LMs with different architectures, i.e., BERT-base, GPT-2, T5-XL (2.8B), and GPT-J (6B), across various knowledge-intensive NLP tasks, i.e., closed book fact-checking and question answering. Remarkably, MALMEN is capable of editing hundreds of times more facts than strong baselines with the identical hyper-network architecture and outperforms editor specifically designed for GPT. Our code is available at https://github.com/ChenmienTan/malmen.

Discussion (0). Sign in to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DisarmRAG: Stealthy Retriever-Centric Poisoning to Disable Self-Correction in Retrieval-Augmented Generation (Extended Version)

    cs.CR 2025-08 conditional novelty 7.0 of 10

    DisarmRAG compromises the retriever to inject anti-self-correction instructions, achieving over 90% attack success across six LLMs while evading basic detection.

  2. ForgetBench: Benchmarking Forgetting Dynamics of Long-Term Parametric Memory in Language Models

    cs.CL 2026-07 conditional novelty 6.0 of 10

    ForgetBench is a sequential-editing benchmark showing that existing knowledge-editing methods trade long-term retention against generalization, while structured contexts mask parametric forgetting.

  3. Don't Use a Cannon to Kill a Fly: Lightweight Model Editing for LLMs to Correct Deprecated API Recommendations

    cs.SE 2025-11 conditional novelty 6.0 of 10

    AdaLoRA-L restricts edits to API-specific layers and raises specificity by 33–836% (relative) on a new 3,000+ instance benchmark while staying close to AdaLoRA's effectiveness.

  4. One for All: Update Parameterized Knowledge Across Multiple Models

    cs.CL 2025-06 conditional novelty 6.0 of 10

    One fine-tuned small model plus an ensemble step can update a fact across multiple large language models with a single edit, outperforming separate per-model editing.

  5. Break Through the Compression Bottleneck: From Theory to Practice

    cs.CL 2026-05 reject novelty 5.0 of 10

    The paper asserts a first proof that low-rank decomposition and quantization are non-orthogonal tools for LLM compression, recommends low-rank-first ordering, and adds a diagonal scaling fix (DAM) that reduces the com...

  6. PREE: Towards Harmless and Adaptive Fingerprint Editing in Large Language Models via Knowledge Prefix Enhancement

    cs.CR 2025-08 conditional novelty 5.0 of 10

    PREE edits a tiny fraction of an LLM's weights so the model answers fake facts when triggered by specially selected prefix phrases, enabling robust ownership verification.

  7. The Compositional Architecture of Regret in Large Language Models

    cs.CL 2025-06 reject novelty 5.0 of 10

    The paper claims that regret in LLMs is encoded by interacting neuron groups detectable in the final hidden layer, using new S-CDI, RDS, and GIC metrics.

  8. A Survey on Proactive Defense Strategies Against Misinformation in Large Language Models

    cs.IR 2025-07 reject novelty 3.0 of 10

    A survey claims proactive defenses against LLM misinformation outperform post-hoc detection by up to 63%, but no meta-analysis details are provided to support the claim.

Pith tools