{"id":"f21ed9be-f4d5-4625-b545-12ce03513a82","arxiv_id":"2506.17864","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A queue-based self-correction framework that maps knowledge triples into transformer FFN parameters and realigns related earlier edits outperforms existing sequential model editing baselines.","lead":"QueueEDIT is a new method for sequentially editing facts inside large language models without wrecking the model's other abilities. It stores each edit's changed parameters in a queue and adjusts earlier, related edits whenever a new fact arrives.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The core self-correction update (Eq. 14–15) is not a well-defined tensor operation: W_proj is a matrix, ΔW is a vector sum, and the concatenation with σ and b′ leaves no reproducible shapes, so the reported gains rest on an unimplementable step as written.","rationale":"I read the paper in good faith: the empirical claim is supported by consistent tables across two backbones, ablations, and a larger-backbone check; the structural loss in Eq. 11 is clearly defined. The decisive weak point is the queue-based self-correction update, which is the only component that distinguishes QueueEDIT from a locate-then-edit baseline. The reader's CONDITIONAL verdict already identifies this under-specification, and I agree that it is the main blocker; my emphasis is that the dimensional inconsistency in Eq. 15 is the single most load-bearing issue because it makes the core mechanism unimplementable as stated. The Euclidean-distance assumption is relevant but secondary: even if the distance were a poor proxy for semantic relatedness, the empirical gains might still be explainable by another mechanism, but without a well-defined update there is no mechanism to test at all. The absence of code strengthens this concern. Because the issue is addressable through clarification or code release rather than being a fundamental contradiction in the experimental design, the verdict should remain CONDITIONAL rather than move to REJECT or ACCEPT.","tokens_in":22021,"tokens_out":3754,"duration_ms":41911,"concrete_test":"Ask the authors for the exact tensor shapes and one end-to-end forward trace of Eq. 14–15, or inspect the released code. Concretely, implement Step 3 in PyTorch with a LLaMA3-8B FFN shape for W_proj, with v* and h_i^r taken from the paper's definitions, and with K=1; if torch.cat or the intended operation raises a shape mismatch or requires undocumented reshaping, padding, or broadcasting, the central update is not reproducible. A minimal pass requires W_i' to have exactly W_i's shape and the corrected model to remain evaluable on the ZSRE edit set.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To support the central claim of significantly outperforming baselines, the queue-based self-correction must be a concrete operation. As written it is not. In §3.3 Step 3, ΔW = v_t^* ⊕ h_i^r (Eq. 14) is an element-wise sum of two vectors: v* is a hidden-state vector obtained from the optimization in Appendix C, and h_i^r is a relation embedding vector from Eq. 10. Then Eq. 15 sets W_i' = σ(W_i ∥ ΔW + b'), where W_i is the located parameter matrix W_proj, ∥ is said to denote concatenation, and σ and b' are not defined. A matrix cannot be concatenated with a vector to produce a matrix of the same shape as W_i without specifying which dimension is concatenated, how the vector is reshaped or broadcast, and what the operation means for the subsequent write-back into the LLM. Since W_i' is explicitly 'aligned back into the LLM,' this step is the entire mechanism responsible for the reported gains over DAFNet. The paper also does not specify how gradients from L_st interact with queue updates, and no code is provided (footnote 1). This is not an empirical disagreement; it is an internal specification gap. The Euclidean-distance similarity assumption in Eq. 12 is a separate concern, but it is secondary because it could be tested empirically; the update rule must be well-posed before that test is meaningful.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"QueueEDIT proposes a sequential model editing (SME) framework for LLMs. It first introduces a structural mapping editing loss that maps a knowledge triple (s, r, o) to designated positions in an FFN layer: the subject is mapped to a key vector k_s^*, the relation to a relation embedding h_r, and the object to a target vector v_*; a translation-style loss (Eq. 11) then updates the projection matrix W_proj^{l0}. It then maintains a FIFO queue of located parameter matrices W_proj for past edits. At each new edit, the method computes Euclidean distances between the current matrix and queued matrices (Eq. 12), selects the top-K closest ones below a threshold, and applies a 'self-correction' update (Eqs. 14-15) that is supposed to realign previously edited parameters to the current edit. The oldest parameter is dequeued based on a threshold. Experiments on ZSRE, CounterFact, and RIPE with GPT-J (6B), LLaMA3 (8B), and Qwen2.5-14B report that QueueEDIT outperforms baselines at 1000 edits and remains competitive in single-turn editing, while preserving general capabilities on CSQA, MMLU, ANLI, and SQuAD2.","tokens_in":22333,"tokens_out":4674,"duration_ms":49116,"significance":"The paper addresses a real and important problem: sequential model editing in LLMs, where parameter updates can interfere with earlier edits and degrade general capabilities. The empirical backbone is extensive: multiple backbones, three editing benchmarks, edit counts of 1, 10, 100, 1000, an ablation study, a queue-length memory analysis, and a larger-backbone experiment with Qwen2.5-14B. The reported gains over DAFNet, the strongest baseline, are substantial at 1000 edits (e.g., 69.1 vs 65.1 average on ZSRE, 61.8 vs 55.3 on CounterFact, 61.2 vs 55.6 on RIPE), and the queue-based self-correction idea is a sensible approach to modeling dependencies among sequential edits. If the method is properly specified and reproducible, these results would be a useful contribution to the SME literature. However, the central self-correction update is not well-defined as written, and the similarity assumption underlying the top-K selection is only anecdotally validated, so the significance is currently contingent on resolving these specification gaps.","major_comments":[{"comment":"The self-correction update is not a well-defined tensor operation as written. In Eq. (14), ΔW is defined as an element-wise sum of two vectors v_t^* and h_i^r, so it is a vector. In Eq. (15), this vector is concatenated with W_i (the located parameter matrix) via the symbol ∥, and then σ and b' are introduced without definition. A matrix cannot be concatenated with a vector without specifying the concatenation axis, the reshaping or broadcasting rule for the vector, and the shapes of σ and b'; moreover, the write-back of W_i' 'into the LLM' is not described. Since this step is the entire mechanism claimed to produce the performance gains, the paper must provide a precise, reproducible specification of Eqs. (14)-(15), including tensor shapes, or provide pseudocode and released code. As it stands, the reported results rest on an unimplementable step.","section":"§3.3, Eqs. (14)-(15)"},{"comment":"The Euclidean distance between located parameter matrices is assumed to be a valid proxy for semantic relatedness between the corresponding knowledge triples, and this assumption is load-bearing for the top-K selection and the threshold-based dequeue. The paper only provides qualitative case studies in Appendix B.2 and Figure 5; no quantitative evidence is given that parameter-space distance tracks fact similarity. A concrete validation would be to compare the distance-based top-K selection against an oracle that knows the true co-occurrence structure of the edit sequence, or to correlate the distances in Eq. (12) with an external semantic similarity measure (e.g., relation/entity embedding cosine similarity) on held-out edit sequences. Without such evidence, the apparent gains could be an artifact of the specific benchmark distributions.","section":"§3.3, Eq. (12)"},{"comment":"The relationship between the closed-form ROME update (Eq. 7), the structural editing loss (Eq. 11), and the total training loss L_total in §3.4 is unspecified. The paper says parameters are 'derived via a closed-form solution' (Eq. 7) but also says the gradient of L_st is used to optimize W_proj^{l0}, and L_total combines L_ed and L_st. It is unclear whether the closed-form update is applied first, whether L_total is optimized from the closed-form initialization, or whether the two are alternative procedures. The order of operations for each edit (closed-form update, gradient-based loss optimization, queue enqueue/top-K/self-correction/dequeue) must be stated precisely for the method to be reproducible.","section":"§3.2 and §3.4"},{"comment":"The main results depend on several hyperparameters that are only partially analyzed. Appendix A.3 fixes K=50, eta_que=0.5, eta_deq=0.5, and the loss coefficients alpha_1 and alpha_2, but the only sensitivity study reported is queue length (Table 2). No experiments vary K or the two thresholds, and the loss coefficients are not discussed at all. Because the performance advantage of QueueEDIT could hinge on these choices, the paper should report sensitivity of the main claims to K and the thresholds, or provide a principled justification for the chosen values beyond 'limitations of machine resources.'","section":"§4.2 and §A.3"}],"minor_comments":[{"comment":"The paper states that source code will be released upon acceptance but provides no code or pseudocode now; given the specification gaps in Eqs. (14)-(15), a detailed algorithm listing in the paper or an anonymous code repository would substantially aid reproducibility.","section":"Footnote 1"},{"comment":"The memory consumption comparison in Table 2 reports memory for the baselines but does not report QueueEDIT's own memory usage, even though the text states that QueueEDIT 'consumes slightly more memory'; please add the QueueEDIT memory numbers to the table.","section":"Table 2"},{"comment":"The sentence 'In general, it suffices to replace the object of the t-th sample with the subject of the i-th sample' is confusing and appears to describe the semantic relation between edits rather than a concrete operation; please rewrite it to state unambiguously what is replaced and how it relates to Eqs. (14)-(15).","section":"§3.3, Step 3"},{"comment":"The training procedure states 'When the number of sequential editing models reaches the maximum value, we perform an additional 20,000 iterations before stopping'; it is unclear whether these additional iterations are on the same edit sequence or on new synthetic facts, and this detail affects reproducibility.","section":"Appendix A.3"},{"comment":"Figure 3 reports average general-capability accuracy without error bars or statistical significance tests; given the 'no degradation' claim in the abstract, please add variance information or a significance statement.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is promising and the experimental comparison is unusually thorough, but the central update rule (Eqs. 14-15) is not a reproducible tensor operation as written, and the similarity assumption in Eq. (12) is not quantitatively validated. These are internal specification gaps rather than issues with the experimental results per se, so I believe they can be fixed within the manuscript's scope by adding precise mathematical definitions, an algorithmic pseudocode, and additional validation experiments. I would also encourage the editor to require a code release or a detailed implementation appendix before final acceptance, given that the reported gains rely entirely on the unspecified self-correction update."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper is worth reading and worth sending to reviewers, but the authors owe us a precise version of Eq. 14-15 before anyone can trust the gains.\n\nWhat's actually new: the structural mapping loss that treats subject, relation, and object separately inside the FFN editing (TransE-inspired) and the FIFO queue with top-K parameter realignment. That combination is not in the prior work I know. The experiments are also solid: 1000-edit results on ZSRE, CounterFact, and RIPE with GPT-J and LLaMA3, plus a 14B backbone, and the gains over DAFNet and other baselines are consistent and not tiny. The ablations show that removing the queue or the structural loss costs a lot, and random selection instead of top-K also hurts. So the two components matter.\n\nThe soft spot is real, and the stress-test note lands. Eq. 14-15 are dimensionally weird: ΔW is a vector sum, W_i is a matrix, and 'concatenation' of a matrix and a vector is not defined. σ and b' are introduced without being specified. Since this update is the whole self-correction mechanism, the paper as written cannot be reproduced. That's not a minor typo; it's the load-bearing step. The distance-as-similarity premise is also only supported by anecdote (Appendix B.2) and the ablation with random selection. The ablation shows selection matters, but not that Euclidean distance between parameter matrices is a faithful proxy for semantic relatedness. No code is provided either. These are addressable issues, but they need to be fixed before the results can be taken at face value.\n\nFor whom: people working on model editing, especially sequential editing, will want to see this. It proposes a practical mechanism for the 1000-edit scale, which is a real pain point. If the update rule gets clarified and code appears, this will be a useful contribution.\n\nRecommendation: send it to peer review. The empirical work is extensive enough and the idea is novel enough that referees should see it. Ask for a precise tensor operation, validation of the similarity assumption, and code.","headline":"QueueEDIT has a novel and empirically promising mechanism for sequential model editing, but the paper's central update rule is under-specified to the point of being unimplementable as written.","tokens_in":22892,"tokens_out":2516,"would_cite":true,"duration_ms":24780,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Sequential LLM editing stays accurate over 1,000 updates by queueing and realigning related knowledge parameters.","keywords":["sequential model editing","knowledge editing","large language models","queue-based self-correction","structural mapping loss","parameter alignment","FIFO weight queue","factual reliability generality locality"],"falsifier":"Run a controlled sequential-editing experiment on 1,000 facts in which parameter-space distance and semantic relatedness are deliberately decoupled — for example, unrelated facts whose located matrices happen to be close, and related facts on the same entity whose matrices are far apart. If the queue's distance-based top-K selection does not outperform random selection or selection by true relation overlap, the self-correction signal is not semantic and the reported gains would not transfer outside the benchmark distributions.","tokens_in":21807,"feed_emoji":"🔄","tokens_out":6922,"duration_ms":63318,"temperature":0.7,"pith_summary":"Sequential model editing aims to keep correcting a large language model's factual errors over many updates, but earlier edits tend to interfere with later ones and degrade general abilities. This paper claims that the interference can be managed by treating edited knowledge as a queue: each edit's located parameter matrix is enqueued, the most similar older matrices are realigned to the current edit, and irrelevant matrices are frozen until dequeued. On ZSRE, CounterFact, and RIPE at 1,000 edits on LLaMA3-8B, the method reports average scores of 69.1, 61.8, and 61.2, above the strongest baseline's 65.1, 55.3, and 55.6, while staying competitive in single-turn editing. The larger point is that a small structural bookkeeping mechanism can preserve both edit reliability and general language ability across long edit sequences.","feed_headline":"Self-correcting queue keeps LLM edits accurate for 1,000 updates","feed_subtitle":"By realigning only related earlier facts, the method also preserves general NLP performance.","key_machinery":"The carrying mechanism is a weight queue of located projection matrices $W_{\\text{proj}}$ plus a translation-based structural loss. The loss is $L_{\\text{st}} = \\|k_s^* W_{\\text{proj}} + h_r - v^*\\|^2$, where $k_s^*$ is the subject-token key vector, $h_r$ is a relation embedding, and $v^*$ is the target object value; it forces the update to behave like $(s + r) \\to o$. The queue then selects, by Euclidean distance in parameter space, the top-$K$ older matrices that are semantically close to the current edit and realigns them with $\\Delta W = v_t^* \\oplus h_r^i$ before concatenating and activating; freezing everything else preserves general capabilities.","core_discovery":"The paper's central claim is that sequential model editing degrades because each edit rewrites the same projection matrix independently, ignoring semantic dependencies among facts. QueueEDIT stores the projection matrix located for each edited triple in a FIFO weight queue. At each new edit it computes Euclidean distances between the current matrix and queued matrices, selects the top-K below a threshold, and applies a self-correction update that links the new object to the old relation; the queue head is dequeued when its distance falls below another threshold. A separate structural editing loss maps the triple's subject, relation, and object onto different parts of the FFN layer, and the paper reports that this combination beats modifying-parameter, extra-parameter, and retrieval baselines while keeping general NLP scores close to the unedited model.","pith_inferences":["Editorial inference: if Euclidean proximity among located projection matrices really tracks factual relatedness, then any locate-then-edit method could be retrofitted with the queue by storing its own located matrices, giving a generic stabilizer for sequential editing.","Editorial inference: the queue's distance threshold could be made adaptive per relation or per layer; the paper fixes one threshold for all edits, so a testable extension is to tune $\\eta_{\\text{que}}$ against relation diversity.","Editorial inference: the evaluation is built from knowledge-graph triples, so a stress test would be to run the same queue on edit sequences with deliberately entangled subjects where the relevant dependency is not nearest in parameter space; the paper's case studies only show same-entity or same-relation neighbors."],"forward_implications":["At 1,000 edits on LLaMA3-8B, average scores rise to 69.1, 61.8, and 61.2 on ZSRE, CounterFact, and RIPE, versus 65.1, 55.3, and 55.6 for the strongest baseline DAFNet.","The same queue mechanism remains competitive at single-turn editing, so adopting it does not sacrifice one-shot edit quality for long-sequence stability.","General NLP capability, measured on public QA benchmarks, stays near the unedited model even after 1,000 edits, while baseline methods decline.","Using a larger backbone, Qwen2.5-14B, raises performance further, consistent with the claim that bigger models hold more internal knowledge for the queue to align.","Queue length trades memory against accuracy: 30% of 1,000 edits works best, while longer queues add semantic noise."],"supporting_citations":[{"why":"Supplies the locate-then-edit recipe: causal tracing, closed-form projection update, and the $v^*$ objective that QueueEDIT builds on.","marker":"(Meng et al. 2022)"},{"why":"Extends editing to multiple layers and supplies the reliability/generality/locality loss formulation reused for training.","marker":"(Meng et al. 2023)"},{"why":"DAFNet is the strongest baseline and the prior sequential-editing method whose interference failure QueueEDIT targets.","marker":"(Zhang et al. 2024)"},{"why":"The translation-based embedding idea $(s+r)\\to o$ motivates the structural mapping loss.","marker":"(Bordes et al. 2013)"},{"why":"Documents gradual and catastrophic forgetting in sequential editing, the problem the queue is designed to mitigate.","marker":"(Gupta, Rao, and Anumanchipalli 2024)"},{"why":"Shows model collapse in sequential ROME-style edits, cited to justify FIFO ordering and the need for top-K alignment.","marker":"(Gupta and Anumanchipalli 2024)"},{"why":"Provides the RIPE evaluation protocol that separates generality and locality into finer components.","marker":"(Cohen et al. 2023)"},{"why":"Provides the ZSRE benchmark used in the main evaluations.","marker":"(Levy et al. 2017)"}],"fun_headline_variants":["QueueEDIT: self-correcting sequential edits without degrading LLMs","Weight queue fixes LLM edits in order, preserving skills","FIFO queue for LLM edits: realign only related facts","Self-correcting queue prevents LLM knowledge edit degradation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the Euclidean distance between two located projection matrices is a faithful stand-in for whether two edits concern related facts, so the top-K closest matrices are the ones needing realignment; the paper supports this with illustrative cases rather than quantitative correlation.","fun_headline_variants_meta":{"raw":{"variants":["QueueEDIT: self-correcting sequential edits without degrading LLMs","Weight queue fixes LLM edits in order, preserving skills","FIFO queue for LLM edits: realign only related facts","Self-correcting queue prevents LLM knowledge edit degradation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000888,"raw_usage":{"total_tokens":3831,"prompt_tokens":942,"completion_tokens":2889,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":558,"completion_tokens_details":{"reasoning_tokens":2817}},"tokens_in":558,"tokens_out":2889,"duration_ms":19648,"temperature":1.0,"reasoning_tokens":2817,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:58:54.683369+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a controlled sequential-editing experiment on 1,000 facts in which parameter-space distance and semantic relatedness are deliberately decoupled — for example, unrelated facts whose located matrices happen to be close, and related facts on the same entity whose matrices are far apart. If the queue's distance-based top-K selection does not outperform random selection or selection by true relation overlap, the self-correction signal is not semantic and the reported gains would not transfer outside the benchmark distributions.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the locate-then-edit recipe: causal tracing, closed-form projection update, and the $v^*$ objective that QueueEDIT builds on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DAFNet is the strongest baseline and the prior sequential-editing method whose interference failure QueueEDIT targets."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The translation-based embedding idea $(s+r)\\to o$ motivates the structural mapping loss."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the ZSRE benchmark used in the main evaluations."}],"review_version":1}