Pith. sign in

REVIEW 3 major objections 5 minor 48 references

NeuralDB: Scaling Knowledge Editing in LLMs to 100,000 Facts with Neural KV Database

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Knowledge editing in LLMs scales to 100,000 facts once linear weight updates are replaced by a gated key-value lookup that only activates on edited inputs.

desk verdict A real 100k-fact memory-editing scaling result undercut by an unverified gate-separation assumption and an overclaim against AlphaEdit. read the letter →

arxiv 2507.18028 v1 pith:FTMQXEBL submitted 2025-07-24 cs.CL cs.AI

Weizhi Fei , Hao Shi , Jing Xu , Jingchen Peng , Jiazheng Li , Jingzhao Zhang , Bo Bai , Wei Han
show 2 more authors
Zhenyuan Chen Xueyan Niu
This is my paper · ORCID
classification cs.CLcs.AI
keywords knowledgeeditinglargelanguagemodelskey-valuedatabasegatedretrievalmassivemodelscalabilityCounterFactZsRE
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the dominant Locate-and-Edit family of knowledge editing methods, which insert linear perturbations into feedforward weights, is best understood as querying a key-value database: the edited facts provide keys and residuals, and the weight update is a weighted average of those residuals. From that perspective, the linear weighting is the bottleneck, because a linear system must protect a sampled set of preserved examples and cannot prevent interference at scale. NeuralDB replaces the linear perturbation with a non-linear gated retrieval module that returns the stored residual only when the current hidden state matches an edited fact, and returns nothing otherwise. The authors report that on CounterFact and ZsRE, across GPT-2 XL, GPT-J, and Llama-3-8B, the method keeps efficacy near 95 to 99 percent while preserving performance on six general text tasks, and that it stays effective when scaled from 10,000 to 100,000 edited facts. If these results hold, the practical obstacle to updating LLM knowledge is no longer the number of edits but the storage cost of the key-value tables.

What carries the argument

The neural KV database is the pair $(K_1, R_1)$, where each edited fact contributes a key $k_i$ computed from the subject prompt and a residual $r_i$ that decodes the new object. The retrieval module is the gated argmax in Eq. (11): it selects the stored key with the highest cosine similarity to the incoming key and returns that key's residual only if the similarity exceeds $\gamma$, set to 0.65 throughout the experiments. The threshold is what enforces the preservation rule, keeping unedited inference on the original computation path, while the argmax is what enforces the editing rule, letting paraphrases retrieve the right residual through semantic similarity of hidden states.

What would settle it

Record, for a held-out set of inputs that never mention any edited subject, the distribution of $\max_i \cos(k, k_i)$ against the stored key matrix after a large edit run; if a nontrivial fraction of these unedited queries exceeds $\gamma = 0.65$, the gate fires and alters their outputs, directly falsifying the claim that general abilities are fully preserved. Equivalently, one can compare six-task scores with the gate active versus forced to zero.

Watch

Extended reading notes

Core claim

The central claim is that a single feedforward layer equipped with the gated retrieval function $g(k; K_1, R_1) = r_j \cdot \mathbb{1}[\cos(k, k_j) > \gamma]$ with $j = \arg\max_i \cos(k, k_i)$ can carry simultaneously edited facts without retraining, because each edit is stored as an explicit pair of a key vector and a residual vector and is activated only when the incoming key is closest to it and above the threshold. The paper states sufficient conditions for editing: an edited fact or its paraphrase must match a stored key and return that key's residual, while an unrelated input must match no stored key and receive the zero vector. Empirically, the paper shows this separation holds for 10,000 edits on three models and two datasets, and that the gate keeps the pre-edit model's behavior intact on six general language benchmarks, with only marginal accuracy decline when the edit count grows to 100,000.

Load-bearing premise

The result rests on a clean cosine separation between edited and unedited inputs: any paraphrase of an edited fact must produce a hidden-state key closer than 0.65 to exactly one stored key, while every unrelated input must stay below 0.65 for all stored keys.

Editorial extensions

If this is right

  • Editing volume can grow from roughly one thousand edited facts to 100,000 facts with only a small accuracy drop, whereas the linear baselines compared in the paper decay sharply by 4,000 edits.
  • General language ability, as measured by six text understanding and generation tasks, stays essentially flat as the edit count grows because unedited queries never trip the gate.
  • The stored database can be maintained incrementally: appending, deleting, or revising a fact becomes a table operation rather than a re-optimization of model weights.
  • Storage grows linearly with the number of edits, and the paper reports about 2.2 percent of Llama-3-8B model size for 10,000 facts and about 20 percent for 100,000 facts.
  • Because the method does not modify the original weights, the pre-edit model remains available as the zero-retrieval case, giving an interpretable separation between edited and unedited behavior.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the cosine-threshold separation is an empirical property rather than a proven guarantee; one can test it directly by measuring the maximum similarity of unrelated queries to stored keys across diverse inputs and edit sets.
  • Beyond the paper: because the gate is a hard switch, a softer or learned gate may buy higher generalization on paraphrases at the cost of some specificity, which the paper's own $\gamma$ ablation already hints at.
  • Beyond the paper: the database metaphor suggests compositional editing, where multi-hop facts are stored as multiple residual entries and chained by retrieval; the paper does not explore this direction.
  • Beyond the paper: the method's success depends on hidden-state keys clustering by subject, so models with weaker key separation may require lower thresholds or multiple database layers to retain the same edit capacity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes NeuralDB, a knowledge-editing framework that stores edited facts as a neural key-value database (K1, R1) and injects a nonlinear gated retrieval module into one FFN layer: g(k; K1, R1) = r_j * 1[cos(k,k_j) > gamma] with j = argmax cos(k, k_i). The gate is intended to return the stored residual only for prompts related to an edited fact, leaving all other forward passes unchanged. The authors reinterpret MEMIT and AlphaEdit as linear KV-querying updates, then evaluate NeuralDB on Counterfact and ZsRE for GPT2-XL, GPT-J, and Llama-3-8B, reporting stable editing metrics up to 100,000 facts and flat scores on six general-ability benchmarks.

Significance. NeuralDB is a simple and attractive alternative to linear L&E: it avoids solving a large constrained least-squares update, supports incremental add/delete, and reports strong scaling results (efficacy about 99% at 10k and 95.5% at 100k on Llama-3). The KV-database reinterpretation of MEMIT and AlphaEdit is instructive, and the paper includes ablation studies, memory/time measurements, and comparisons against four mass-editing baselines. However, the headline claim of 'fully preserving general abilities' depends on a separation property of the gate that is asserted but never measured, and the 100k evidence is limited to MMLU. If the separation property is confirmed, this would be a practical contribution; as it stands, the paper's strongest claims outrun its evidence.

major comments (3)
  1. [Section 5.1, Table 1] The abstract's claim that NeuralDB 'excels in editing efficacy, generalization, specificity, fluency, and consistency' is contradicted by the 2,000-edit rows of Table 1: on Counterfact generalization, AlphaEdit scores 94.0 versus 86.6 for Llama-3 and 96.3 versus 94.6 for GPT-J, and on ZsRE generalization the two methods tie for GPT-J at 95.9. The paper should either restrict the superiority claim to the 10,000-edit setting or report the comparison honestly by metric and scale.
  2. [Section 4.2, Eq. (11), Rule II] The gate's preservation guarantee requires that for every unedited input, max_j cos(k, k_j) remains below gamma = 0.65, but the paper provides no distributional evidence for this separation. The six benchmarks in Fig. 4 and Table 7 are a limited probe; with 100,000 stored keys the relevant quantity is a maximum over 100,000 comparisons, and a false positive silently adds a learned residual and corrupts the model's output. I request a max-cosine histogram on held-out text (for example, Wikipedia or a mixed-domain corpus), the false-positive rate as a function of the number of edits m, and an explicit failure analysis for near-duplicate subjects, typos, and topic shifts.
  3. [Section 5.3, Table 2] The claim that scaling to 100,000 facts 'fully preserv[es] general language understanding and generation capabilities' is supported only by MMLU, because Table 2 reports no other benchmark and Fig. 4 stops at 10,000 edits. The generalization to Lambada, WSC273, and generation tasks is therefore unsupported; the 100k experiment should include at least the generation and reasoning tasks used elsewhere, or the claim should be downgraded to what the data actually show.
minor comments (5)
  1. [Abstract and Section 1] The abstract says '50x more than in prior work' while Section 1 says '45x more edited facts than AlphaEdit'; these numbers should be reconciled, and the relationship between the 10,000-fact headline and the 100,000-fact scaling experiment should be stated consistently.
  2. [Appendix G] The sentence 'the additional memory for 1M facts is only 2.2%' appears to be a scaling error: for Llama-3 8B, 10,000 facts cost about 150M parameters, so 1M facts would cost about 15B parameters, not 2.2%. This should refer to 10,000 facts and should be reconciled with Appendix A's 20% figure for 100,000 facts.
  3. [Throughout] There are typographical issues that should be corrected: 'Defination 1' should be 'Definition 1', 'Border impact' should be 'Broader impact', and 'lm-evaluation-hardness' in Appendix I.1 should be 'lm-evaluation-harness'.
  4. [Table 5 and Table 3] The gamma and layer ablations are performed only on Llama-3, yet the paper uses gamma = 0.65 for all three models; the choice of l* = 7 for Llama-3 also differs from the causal-trace layer 17 reported in Table 3, and the selection procedure for these hyperparameters should be stated explicitly.
  5. [Section 7] The conclusion says the method can edit 'hundreds of thousands of facts', but the largest experiment is 100,000 facts; the wording should be aligned with the evidence.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the gate is a designed mechanism, the preservation claim is benchmark-tested, and the only self-citation is background.

full rationale

NeuralDB's derivation chain is not circular in the scoring sense. The KV-database re-derivation of MEMIT and AlphaEdit (Eqs. 6-10) is exact algebra from the cited methods' own closed forms, not an imported conclusion. The nonlinear gate in Eq. 11 is the authors' own mechanism; Rule II's cosine-separation premise is an empirical assumption, not a result smuggled in by self-citation, and the paper tests it on six external benchmarks (Fig. 4, Table 7). The per-fact residual r_i is fitted by Eq. 21 to the edited prompt, so the efficacy metric in Eq. 13 partly reports training-fit behavior; however, the paper does not present efficacy as a predicted consequence, and its central claims—paraphrase generalization, neighborhood specificity, and preservation of general ability at 10k-100k edits—are out-of-sample or external. The one self-citation ([27], used only as associative-memory background) is not load-bearing. Thus the score is 1.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method relies on a thresholded nearest-neighbor lookup, so the key assumptions are about geometric separation of keys in one FFN layer. The only tuned free parameters are the gate threshold, the edited layer, and the unstated prefix count in the key/residual estimation. No new physical or theoretical entities are introduced.

free parameters (3)
  • gamma (cosine gate threshold) = 0.65
    Tuned and ablated (Table 5); the gate in Eq. 11 fires only when cosine similarity exceeds this value. It is a single global threshold across all models, chosen by search.
  • target layer l* = GPT2-XL: 17, GPT-J: 8, Llama-3: 7
    Table 3 lists causal-trace-found layer 17 for all models, but the method uses a different layer for GPT-J and Llama-3, indicating the layer was selected by validation, not derived.
  • prefix count N = not reported
    Eq. 20 and Eq. 21 average over N random prefixes to compute keys and residuals; N is never given, so key estimation has an unstated hyperparameter.
assumptions (3)
  • domain assumption Unrelated inputs have low cosine similarity to all edited-fact keys, and paraphrases of edited facts have high similarity to their own key.
    Rule I and Rule II (Section 4.1) depend on this separation; Eq. 11's threshold gamma realizes it, but no proof or failure analysis is provided.
  • domain assumption Per-fact residuals remain effective when stored jointly and selected by argmax over cosine similarity.
    Eq. 21 optimizes each residual independently; the paper does not analyze interactions between keys, so overlapping or contradictory edits could retrieve the wrong value.
  • domain assumption Editing a single FFN layer is sufficient for correctness at large scale.
    Section 4.2 and Appendix H justify this empirically; Appendix H.2 shows multi-layer variants change the accuracy/storage trade-off, so single-layer is a design choice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of NeuralDB: Scaling Knowledge Editing in LLMs to 100,000 Facts with Neural KV Database." pith.science (2026). https://pith.science/paper/FTMQXEBL

@misc{pith2026250718028,
  author       = {Pith},
  title        = {Pith review of: NeuralDB: Scaling Knowledge Editing in LLMs to 100,000 Facts with Neural KV Database},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FTMQXEBL}},
  note         = {Machine review of arXiv:2507.18028}
}
read the original abstract

Efficiently editing knowledge stored in large language models (LLMs) enables model updates without large-scale training. One possible solution is Locate-and-Edit (L\&E), allowing simultaneous modifications of a massive number of facts. However, such editing may compromise the general abilities of LLMs and even result in forgetting edited facts when scaling up to thousands of edits. In this paper, we model existing linear L\&E methods as querying a Key-Value (KV) database. From this perspective, we then propose NeuralDB, an editing framework that explicitly represents the edited facts as a neural KV database equipped with a non-linear gated retrieval module, % In particular, our gated module only operates when inference involves the edited facts, effectively preserving the general abilities of LLMs. Comprehensive experiments involving the editing of 10,000 facts were conducted on the ZsRE and CounterFacts datasets, using GPT2-XL, GPT-J (6B) and Llama-3 (8B). The results demonstrate that NeuralDB not only excels in editing efficacy, generalization, specificity, fluency, and consistency, but also preserves overall performance across six representative text understanding and generation tasks. Further experiments indicate that NeuralDB maintains its effectiveness even when scaled to 100,000 facts (\textbf{50x} more than in prior work).

Figures

Figures reproduced from arXiv: 2507.18028 by the authors.

Figure 1
Figure 1. The proposed NeuralDB scales the number of edited facts up to 10, 000 with almost no performance loss. Left: Average of efficacy, generalization, and specificity on the MCF dataset compared with AlphaEdit, MEMIT, and RECT. As the number of edits grows, NeuralDB experiences only a slight decline, whereas the other three methods undergo progressively larger decreases. Right: Average performance on six tasks (MMLU, Sci… view at source ↗
Figure 2
Figure 2. Visualization of weighted scores ω = KT 1 Sk using MEMIT and AlphaEdit for three models. The boxplots are generated from the mean and variance of weight scores, with the center line indicating the mean, boxes showing ±1 standard deviation, and whiskers ±1.5. When inferring the i-th edited fact, ωi serves as the positive sample, while the remaining elements of ω are negative samples. The high positive score ensures t… view at source ↗
Figure 3
Figure 3. Overview of NeuralDB editing framework. We replace the linear system in L&E with a non-linear gated retrieval module in the FFN layer. The key K1 and residual matrix R1, defined in Eq. 4 and Eq. 5, are computed to construct the neural KV database. During inference, our non-linear gated function g(·; K1, R1) only returns the most matched residual rj when post-edited models infer one edited fact and involve key vector… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Results of general abilities after massive editing. The performance of NeuralDB is compared [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Visualization of weighted scores for paraphrased facts and neighborhood facts, using [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 30 canonical work pages

  1. [1]

    Is your llm outdated? evaluating llms at temporal generalization

    Chenghao Zhu, Nuo Chen, Yufei Gao, Yunyi Zhang, Prayag Tiwari, and Benyou Wang. Is your llm outdated? evaluating llms at temporal generalization. arXiv preprint arXiv:2405.08460, 2024

    arXiv 2024

  2. [2]

    Openagi: When llm meets domain experts

    Yingqiang Ge, Wenyue Hua, Kai Mei, Juntao Tan, Shuyuan Xu, Zelong Li, Yongfeng Zhang, et al. Openagi: When llm meets domain experts. Advances in Neural Information Processing Systems, 36:5539–5568, 2023

    work page 2023

  3. [3]

    An empirical study of catastrophic forgetting in large language models during continual fine-tuning

    Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint arXiv:2308.08747, 2023

    arXiv 2023

  4. [4]

    Does fine-tuning llms on new knowledge encourage hallucinations? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 7765–7784, 2024

    Zorik Gekhman, Gal Yona, Roee Aharoni, Matan Eyal, Amir Feder, Roi Reichart, and Jonathan Herzig. Does fine-tuning llms on new knowledge encourage hallucinations? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 7765–7784, 2024

    work page 2024

  5. [5]

    Knowledge editing for large language models: A survey

    Song Wang, Yaochen Zhu, Haochen Liu, Zaiyi Zheng, Chen Chen, and Jundong Li. Knowledge editing for large language models: A survey. ACM Computing Surveys, 57(3):1–37, 2024

    work page 2024

  6. [6]

    Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D. Manning. Fast model editing at scale. In ICLR. OpenReview.net, 2022

    work page 2022

  7. [7]

    Can we edit factual knowledge by in-context learning? CoRR, abs/2305.12740, 2023

    Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. Can we edit factual knowledge by in-context learning? CoRR, abs/2305.12740, 2023

    arXiv 2023

  8. [8]

    Andonian, Yonatan Belinkov, and David Bau

    Kevin Meng, Arnab Sen Sharma, Alex J. Andonian, Yonatan Belinkov, and David Bau. Mass- editing memory in a transformer. In ICLR. OpenReview.net, 2023

    work page 2023

Show all 48 references
  1. [9]

    Locating and editing factual associations in GPT

    Kevin Meng, David Bau, Alex J Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems , 2022. URL https:// openreview.net/f...

    2022

  2. [10]

    PMET: precise model editing in a transformer

    Xiaopeng Li, Shasha Li, Shezheng Song, Jing Yang, Jun Ma, and Jie Yu. PMET: precise model editing in a transformer. In AAAI, pages 18564–18572. AAAI Press, 2024. 10

    2024

  3. [11]

    Alphaedit: Null-space constrained model editing for language models

    Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Jie Shi, Xiang Wang, Xiangnan He, and Tat-Seng Chua. Alphaedit: Null-space constrained model editing for language models. In The Thirteenth International Conference on Learning Representations, 2025. URL https: //openreview.n...

    2025

  4. [12]

    Transformer feed-forward layers are key-value memories

    Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen- tau Yih, editors,Proceedings of the 2021 Conference on Empirical Methods in Natural Language P...

    2021 doi

  5. [13]

    Reasons and solutions for the decline in model performance after editing

    Xiusheng Huang, Jiaxiang Liu, Yequan Wang, and Kang Liu. Reasons and solutions for the decline in model performance after editing. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id= xjXYgdFM5M

    2024

  6. [14]

    Language models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019

    2019

  7. [15]

    Mesh-Transformer-JAX: Model-Parallel Implementation of Transformer Lan- guage Model with JAX

    Ben Wang. Mesh-Transformer-JAX: Model-Parallel Implementation of Transformer Lan- guage Model with JAX. https://github.com/kingoflolz/mesh-transformer-jax , May 2021

    2021

  8. [16]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  9. [17]

    MELO: enhancing model editing with neuron- indexed dynamic lora

    Lang Yu, Qin Chen, Jie Zhou, and Liang He. MELO: enhancing model editing with neuron- indexed dynamic lora. In AAAI, pages 19449–19457. AAAI Press, 2024

    2024

  10. [18]

    Editing large language models via adaptive gradient guidance

    Xiaojie Gu, Guangxu Chen, Shuliang Liu, Jungang Li, Aiwei Liu, Sicheng Tao, Junyan Zhang, and Xuming Hu. Editing large language models via adaptive gradient guidance. In AAAI 2025 Workshop on Preventing and Detecting LLM Misinformation (PDLM), 2025

    2025

  11. [19]

    Zero-shot relation extraction via reading comprehension

    Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. Zero-shot relation extraction via reading comprehension. In CoNLL, pages 333–342. Association for Computational Linguistics, 2017

    2017

  12. [20]

    Liu, and Matt Gardner

    Johannes Welbl, Nelson F. Liu, and Matt Gardner. Crowdsourcing multiple choice science questions. In Leon Derczynski, Wei Xu, Alan Ritter, and Tim Baldwin, editors, Proceedings of the 3rd Workshop on Noisy User-generated Text , pages 94–106, Copenhagen, Denmark, September 2017...

    2017 doi

  13. [21]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum? id=d7KBjmI3GmQ

    2021

  14. [22]

    CommonsenseQA: A question answering challenge targeting commonsense knowledge

    Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. CommonsenseQA: A question answering challenge targeting commonsense knowledge. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of ...

    2019 doi

  15. [23]

    Think you have solved question answering? try arc, the ai2 reasoning challenge

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018. 11

    2018 arXiv

  16. [24]

    A surprisingly robust trick for the Winograd schema challenge

    Vid Kocijan, Ana-Maria Cretu, Oana-Maria Camburu, Yordan Yordanov, and Thomas Lukasiewicz. A surprisingly robust trick for the Winograd schema challenge. In Anna Ko- rhonen, David Traum, and Lluís Màrquez, editors, Proceedings of the 57th Annual Meet- ing of the Association fo...

    2019 doi

  17. [25]

    The LAMBADA dataset: Word prediction requiring a broad discourse context

    Denis Paperno, Germán Kruszewski, Angeliki Lazaridou, Ngoc Quan Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernández. The LAMBADA dataset: Word prediction requiring a broad discourse context. In Katrin Erk and Noah A. Smith, editors, Proc...

    2016 doi

  18. [26]

    A framework for few-shot language model evaluation, 07 2024

    Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang S...

    2024

  19. [27]

    Beyond scaling laws: Understanding transformer performance with associative memory

    Xueyan Niu, Bo Bai, Lei Deng, and Wei Han. Beyond scaling laws: Understanding transformer performance with associative memory. arXiv preprint arXiv:2405.08707, 2024

    2024 arXiv

  20. [28]

    Should we really edit language models? on the evaluation of edited language models

    Qi Li, Xiang Liu, Zhenheng Tang, Peijie Dong, Zeyu Li, Xinglin Pan, and Xiaowen Chu. Should we really edit language models? on the evaluation of edited language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openrevie...

    2024

  21. [29]

    Editing the mind of giants: An in-depth exploration of pitfalls of knowledge editing in large language models

    Cheng-Hsun Hsueh, Paul Kuo-Ming Huang, Tzu-Han Lin, Che Wei Liao, Hung-Chieh Fang, Chao-Wei Huang, and Yun-Nung Chen. Editing the mind of giants: An in-depth exploration of pitfalls of knowledge editing in large language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung ...

    2024 doi

  22. [30]

    Model editing harms general abilities of large language models: Regularization to the rescue

    Jia-Chen Gu, Hao-Xiang Xu, Jun-Yu Ma, Pan Lu, Zhen-Hua Ling, Kai-Wei Chang, and Nanyun Peng. Model editing harms general abilities of large language models: Regularization to the rescue. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors,Proceedings of the 2024 Conf...

    2024

  23. [31]

    Perturbation- restrained sequential model editing

    Jun-Yu Ma, Hong Wang, Hao-Xiang Xu, Zhen-Hua Ling, and Jia-Chen Gu. Perturbation- restrained sequential model editing. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=bfI8cp8qmk

    2025

  24. [32]

    Editing factual knowledge in language models

    Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. arXiv preprint arXiv:2104.08164, 2021

    2021 arXiv

  25. [33]

    Fast model editing at scale

    Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning. Fast model editing at scale. arXiv preprint arXiv:2110.11309, 2021

    2021 arXiv

  26. [34]

    Instructedit: Instruction-based knowledge editing for large language models

    Ningyu Zhang, Bozhong Tian, Siyuan Cheng, Xiaozhuan Liang, Yi Hu, Kouying Xue, Yanjie Gou, Xi Chen, and Huajun Chen. Instructedit: Instruction-based knowledge editing for large language models. arXiv preprint arXiv:2402.16123, 2024

    2024 arXiv

  27. [35]

    Memory-based model editing at scale

    Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D Manning, and Chelsea Finn. Memory-based model editing at scale. In International Conference on Machine Learning, pages 15817–15831. PMLR, 2022. 12

    2022

  28. [36]

    Transformer-patcher: One mistake worth one neuron

    Zeyu Huang, Yikang Shen, Xiaofeng Zhang, Jie Zhou, Wenge Rong, and Zhang Xiong. Transformer-patcher: One mistake worth one neuron. arXiv preprint arXiv:2301.09785, 2023

    2023 arXiv

  29. [37]

    Aging with grace: Lifelong model editing with discrete key-value adaptors

    Tom Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. Aging with grace: Lifelong model editing with discrete key-value adaptors. Advances in Neural Information Processing Systems, 36:47934–47959, 2023

    2023

  30. [38]

    Melo: Enhancing model editing with neuron- indexed dynamic lora

    Lang Yu, Qin Chen, Jie Zhou, and Liang He. Melo: Enhancing model editing with neuron- indexed dynamic lora. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 38, pages 19449–19457, 2024

    2024

  31. [39]

    Mindbridge: Scalable and cross-model knowledge editing via memory-augmented modality

    Shuaike Li, Kai Zhang, Qi Liu, and Enhong Chen. Mindbridge: Scalable and cross-model knowledge editing via memory-augmented modality. arXiv preprint arXiv:2503.02701, 2025

    2025 arXiv

  32. [40]

    Memory-assisted prompt editing to improve GPT-3 after deployment

    Aman Madaan, Niket Tandon, Peter Clark, and Yiming Yang. Memory-assisted prompt editing to improve GPT-3 after deployment. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors, Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages...

    2022 doi

  33. [41]

    Can we edit factual knowledge by in-context learning? In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023

    Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. Can we edit factual knowledge by in-context learning? In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023. URL https://openreview.net/forum? id=hsjQHAM8MV

    2023

  34. [42]

    MQuAKE: Assessing knowledge editing in language models via multi-hop questions

    Zexuan Zhong, Zhengxuan Wu, Christopher D Manning, Christopher Potts, and Danqi Chen. MQuAKE: Assessing knowledge editing in language models via multi-hop questions. In The 2023 Conference on Empirical Methods in Natural Language Processing , 2023. URL https://openreview.net/f...

    2023

  35. [43]

    PokeMQA: Programmable knowledge editing for multi-hop question answering

    Hengrui Gu, Kaixiong Zhou, Xiaotian Han, Ninghao Liu, Ruobing Wang, and Xin Wang. PokeMQA: Programmable knowledge editing for multi-hop question answering. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for C...

    2024

  36. [44]

    Retrieval-enhanced knowledge editing in language models for multi-hop question an- swering

    Yucheng Shi, Qiaoyu Tan, Xuansheng Wu, Shaochen Zhong, Kaixiong Zhou, and Ninghao Liu. Retrieval-enhanced knowledge editing in language models for multi-hop question an- swering. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, C...

    2024

  37. [45]

    subject is a

    Weixuan Wang, Barry Haddow, and Alexandra Birch. Retrieval-augmented multilingual knowledge editing. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages...

    2024 doi

  38. [46]

    baseline 99.08 93.48 75.52 620.53 [6,7,8] new multi layers 94.44 91.72 75.93 617.44 [6,7,8] old multi layers 99.31 93.23 76.78 616.00 GPT2-XL

  39. [47]

    The boxplots are generated from the mean and variance of weight scores, with the center line indicating the mean, boxes showing ±1 standard deviation, and whiskers ±1.5

    baseline 99.04 95.96 70.72 621.90 [15,16,17] new multi layers 94.81 92.68 70.26 618.51 [15,16,17] old multi layers 99.08 94.01 71.33 624.48 (a) Paraphrased facts (b) Neighborhood facts Figure 5: Visualization of weighted scores for paraphrased facts and neighborhood facts, usi...

  40. [48]

    These results confirm that, during inference, residuals unrelated to the edited facts remain inactive, resulting in near-zero weighted scores

    For neighborhood facts, where all components are considered negative, the scores are likewise consistently close to 0. These results confirm that, during inference, residuals unrelated to the edited facts remain inactive, resulting in near-zero weighted scores. 19 Table 7: Per...

    1925

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.