{"id":"406a03a3-d838-4d49-9b9d-9af5b2d09e91","arxiv_id":"2412.20412","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"MOLLM formulates LLM unlearning as a three-objective optimization problem and uses a bounded unlearning loss plus a common descent direction to forget target data while preserving model utility.","lead":"The paper introduces MOLLM, an algorithm that removes unwanted knowledge from large language models by treating unlearning as a multi-objective optimization problem. It combines a bounded unlearning loss with a multi-gradient descent step, aiming to forget target data while preserving the model's general abilities.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DS-MGDA can produce a zero update when gradients are linearly dependent, so the central convergence claim is not merely unproven but false in a simple case.","rationale":"The reader's weakest assumption correctly identified that the convergence guarantee for DS-MGDA is not established. My stress-test goes further: the proposed update direction is not just missing a proof, it is mathematically wrong in a concrete, low-dimensional case. Because the algorithm projects each gradient onto the orthogonal complement of the other two, it returns exactly zero whenever the three gradients are linearly dependent, even when a common descent direction exists. This is a direct counterexample to the paper's central claim that DS-MGDA computes a common descent direction and reaches Pareto stationarity. The counterexample is not exotic; it can arise whenever the objectives' gradients are linearly dependent, which can happen at non-stationary points, and no full-rank assumption is stated for the three-objective case. The appeal to Fliege–Svaiter is therefore invalid, since that theorem concerns the minimum-norm vector in the convex hull, not an average of individually projected gradients. The empirical results on PKU-SafeRLHF are suggestive and the UCE loss is a sensible bounded-loss device, but a single benchmark without seeds or error bars cannot compensate for a false central algorithm. The paper could potentially be repaired by restricting DS-MGDA to full-rank gradient sets with a valid convergence proof, or by modifying the direction to handle dependent gradients, but as written the main contribution is unsupported. I therefore move the verdict from CONDITIONAL to REJECT, while acknowledging that the empirical comparison may survive with substantial revision.","tokens_in":8302,"tokens_out":8071,"duration_ms":83453,"concrete_test":"Run DS-MGDA on the 3-objective toy example with gradients g_fgt=(1,0), g_KL=(0,1), g_rt=(1,1). Compute Eq. (8) for each objective and the resulting d_t; it returns 0. Then verify d=(-1,-1) satisfies d·g_i<0 for all three objectives, showing a common descent direction exists but DS-MGDA fails to find it. To assess relevance to the paper's setting, log the rank of [g_fgt, g_KL, g_rt] during MOLLM unlearning runs; if the rank ever drops to 2, d_t will be zero and no progress is made on at least one objective.","verdict_should_be":"REJECT","load_bearing_attack":"The paper's central theoretical claim is that the DS-MGDA direction d_t = -1/3(g*_fgt + g*_KL + g*_rt) is a common descent direction and that, by the Fliege–Svaiter theorem, iterating it reaches Pareto stationarity. This does not follow. Equation (8) projects each gradient onto the orthogonal complement of the span of the other two, so d_t·g_i = -(1/3)||g*_i||^2. Whenever g_i lies in the span of the other two gradients, g*_i = 0, and the update for that objective is zero. For example, in R^2 take g_fgt=(1,0), g_KL=(0,1), g_rt=(1,1). All three projections vanish, giving d_t=0, yet d=(-1,-1) strictly decreases all three objectives. Thus DS-MGDA is not a common-descent method in general and can stall away from Pareto stationarity. The cited MGDA theorem applies to the minimum-norm element of the convex hull of gradients, not to this averaged projection. Definition 2's 'dual space' is also internally inconsistent: no nonzero vector b can satisfy b·a<0 for every a in a linear subspace, because both a and -a lie in the subspace.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MOLLM, a multi-objective optimization approach to LLM unlearning. It replaces the unbounded cross-entropy loss in gradient ascent with a bounded Unlearning Cross-Entropy (UCE) loss, and introduces Dual Space Multiple Gradient Descent Algorithm (DS-MGDA) to compute a common descent direction that simultaneously decreases the forget loss, the KL-divergence to the original model, and the retain loss. Experiments on the PKU-SafeRLHF dataset with Llama 3-8B report that MOLLM achieves lower harmful rate, toxicity, and obscenity than GA-weighted-sum and GA-OGD baselines, while preserving (or improving) fluency. The authors also provide ablation studies and release source code.","tokens_in":8563,"tokens_out":5349,"duration_ms":51579,"significance":"If the theoretical and empirical claims held, the paper would contribute a useful, hyperparameter-light alternative to gradient-ascent-based LLM unlearning: the UCE loss is a simple and reasonable fix for gradient explosion, and a principled multi-objective update direction could mitigate catastrophic forgetting. The paper is clearly written and the code release is a strength. However, the central theoretical assertion—that DS-MGDA yields a common descent direction and inherits MGDA convergence guarantees—is not supported and is in fact false in simple cases, and the experimental comparison lacks error bars and multiple seeds. The contribution is therefore not yet established.","major_comments":[{"comment":"The DS-MGDA direction can be zero even when a strict common descent direction exists, so the claim that d_t is always a common descent direction is false. For example, take g_fgt = (1,0), g_KL = (0,1), and g_rt = (1,1) in R^2. The span of any two of these gradients is all of R^2, so the projection onto the null space of the other two yields g*_fgt = g*_KL = g*_rt = 0, giving d_t = 0. Yet the direction d = (-1,-1) has negative dot product with all three gradients, so it is a common descent direction and is not Pareto stationary. Hence DS-MGDA can stall at a non-stationary point.","section":"Section III-B, Eq. (8)"},{"comment":"Definition 2 is internally inconsistent. For a linear subspace S, if a ∈ S then -a ∈ S as well. No nonzero vector b can satisfy b·a < 0 for every a ∈ S, because b·a < 0 and b·(-a) < 0 would imply b·a > 0. The subsequent statement that convex combinations of g*_i 'form the dual space' is therefore ill-posed; the correct object would be the negative polar cone, with non-strict inequalities, not a vector space with strict inner products.","section":"Section III-B, Definition 2"},{"comment":"The claimed convergence to Pareto stationarity does not follow from the cited multiple gradient descent theorems [21]–[23]. Those theorems apply to the minimum-norm element of the convex hull of the gradients (or to the solution of a specific quadratic program), not to the averaged projection d_t = -(1/3)(g*_fgt + g*_KL + g*_rt). The paper provides no convergence proof for its specific direction, and the counterexample in the first major comment shows that convergence can fail completely.","section":"Section III-B, paragraph after Definition 1"},{"comment":"The experimental protocol 'take the best performance of each method' combined with no reported number of seeds or error bars makes the claimed superiority statistically unsupported; the reported gains over baselines may be due to selection bias in hyperparameter choice. In addition, the conflict probabilities PC_fgt, PC_KL, and PC_rt are zero for MOLLM by construction because d_t is explicitly built to have negative inner products with each gradient, so these columns do not provide independent evidence of conflict resolution.","section":"Section IV-A and Table I"}],"minor_comments":[{"comment":"The method name is inconsistently spelled as 'MoLLM' in the abstract and 'MOLLM' in the body; please standardize.","section":"Abstract and throughout"},{"comment":"The entry 'Re-fintuning' is a typo; it should be 'Re-finetuning'.","section":"Table I"},{"comment":"The sentence 'Since the dual vectors g*_fgt, g*_KL, g*_rt lie on the edge of S*, one of them is orthogonal to the other two' is unclear and not generally true; please state precisely the orthogonality relations that hold.","section":"Section III-B"},{"comment":"The KL divergence formula appears mis-typeset: the arguments of the KL term should be probability distributions, not the raw sequences; please correct the notation.","section":"Eq. (4)"},{"comment":"Reference [29] cites the original LLaMA paper, but the experiments use Llama 3-8B; please cite the correct Llama 3 model release.","section":"References"},{"comment":"The computation requires A A^T to be invertible, i.e., the gradients g_KL and g_rt must be linearly independent; the paper only notes this for more than three objectives, but it also applies in the three-objective case and is not guaranteed in high-dimensional parameter spaces.","section":"Section III-B, Eq. (8)"}],"recommendation":"major_revision","confidential_remarks":"The central mathematical contribution (DS-MGDA) is not correct as stated: Definition 2 is mathematically impossible, and the algorithm can return a zero update in a simple three-gradient example, so the convergence claim is unsupported. The authors would need to either replace DS-MGDA with a method that provably yields a common descent direction (e.g., the standard MGDA direction) or substantially weaken the theoretical claims to 'empirically effective'. The experimental section also needs error bars and a transparent model-selection protocol before the empirical claims can be evaluated. These are substantial but potentially addressable within the scope of a revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take on arXiv:2412.20412 (MOLLM). The reader's conditional verdict is about right on the experimental side, but the stress-test note is correct and lands hard: the central convergence claim is false, and Definition 2 is mathematically broken.\n\nWhat's actually new: the UCE loss (bounded unlearning surrogate) is a simple, sensible fix for gradient explosion and avoids the clipping hyperparameter. That part is worth crediting. The experimental results on SafeRLHF look good—MOLLM gets the lowest harmful rate and best fluency in Table I—though there are no error bars and the \"best performance\" learning-rate selection weakens the comparison. Still, the empirical direction is reasonable.\n\nWhere it falls apart: DS-MGDA is not the multiple gradient descent of Fliege–Svaiter. Equation (8) projects each gradient onto the orthogonal complement of the other two; when the three gradients are linearly dependent, all projections vanish and d=0, even when a common descent direction exists. In any parameter space of dimension less than three this happens immediately; in high dimension it is a near-unmeasureable edge case, but the theorem as stated is simply not true. Also, Definition 2 asserts a vector b with b·a<0 for every a in a linear subspace, which is impossible because a and -a both live in the subspace. That definition needs to be replaced with a polar cone or something similar. These are not cosmetic issues; they are the paper's justification for calling MOLLM a principled multi-objective method.\n\nThe good news: the empirical claim that MOLLM outperforms GA-weighted-sum and GA-OGD on this benchmark is independently measurable and not derived from the broken theory. The UCE loss and the overall recipe might survive a rewrite that uses a real MGDA direction (or that drops the convergence claim entirely and presents DS-MGDA as a heuristic). The authors should also add multiple seeds and report variance.\n\nWho this is for: people working on LLM unlearning who want a bounded loss and a gradient-surgery update. It deserves a serious referee because the empirical idea is useful and the flaws are correctable, but it should not be accepted as-is. My recommendation: send it to review, with a clear request to fix the math or remove the convergence claims, and to tighten the experimental reporting.\n\nThat's my read. — [Name]","headline":"The empirical story is plausible, but the theoretical core of DS-MGDA does not survive contact with linear algebra; the paper needs a major revision before it can be trusted.","tokens_in":9092,"tokens_out":2910,"would_cite":false,"duration_ms":27744,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MOLLM reformulates LLM unlearning as three competing objectives and finds a single descent direction that improves all three at once.","keywords":["large language model unlearning","gradient ascent","multi-objective optimization","catastrophic forgetting","gradient explosion","common descent direction","unlearning cross-entropy loss","Pareto stationarity"],"falsifier":"Recompute MOLLM's update direction on a small model and check whether the inner product with each of the three gradients is negative at every step; any non-negative inner product shows the direction is not a common descent direction.","tokens_in":8130,"feed_emoji":"🛡️","tokens_out":8046,"duration_ms":65905,"temperature":0.7,"pith_summary":"The paper claims that gradient-ascent-based LLM unlearning can be made stable and utility-preserving by reformulating it as a multi-objective optimization problem. It introduces an Unlearning Cross-Entropy (UCE) loss that replaces the unbounded inverse cross-entropy, preventing gradient explosion without manual clipping. To handle the conflict between forgetting and retaining knowledge, it proposes DS-MGDA, which projects the three loss gradients into the dual space and averages the dual vectors to get a common descent direction that reduces all three objectives at once. On the PKU-SafeRLHF benchmark with Llama 3-8B, MOLLM reports the lowest harmful rate, toxicity, and obscenity among gradient-ascent baselines while keeping fluency at 1.689.","feed_headline":"Dual-space gradient method unlearns harmful LLM data, keeps fluency","feed_subtitle":"MOLLM cuts harmful rate to 3.45% on Llama-3-8B with fluency 1.689.","key_machinery":"The central object is the Dual Space Multiple Gradient Descent Algorithm (DS-MGDA). It takes the gradients of the unlearning loss, KL preservation loss, and retain loss; computes each gradient's projection onto the null space of the other two (so that each dual vector is orthogonal to the other two gradients); and averages the three dual vectors (with negative sign) to obtain the update direction. This direction is claimed to lie in the dual cone where every sub-objective decreases. The UCE loss, defined as $L_{UCE} = -\\frac{1}{K}\\sum_i\\sum_c y_{i,c}\\log(1-(1-\\epsilon)p_{i,c})$, is bounded below, which prevents the gradient explosion of standard gradient ascent on cross-entropy.","core_discovery":"The central claim is that LLM unlearning should be treated as a multi-objective problem with three objectives: increasing the unlearning loss on the forget set, minimizing KL divergence from the original model on retain data, and minimizing cross-entropy on retain data. The paper's contribution is a concrete algorithm, MOLLM, that computes a single update direction satisfying all three objectives simultaneously. The key innovation is DS-MGDA, which constructs the dual space of the span of the three gradients, computes each gradient's projection onto the null space of the other two, and averages the three dual vectors to produce a common descent direction. The paper argues this direction reaches Pareto stationarity by invoking multiple gradient descent theory, and empirically it reports that MOLLM outperforms GA-weighted-sum and GA-OGD on unlearning effect and utility preservation.","pith_inferences":["This reading suggests that the averaged-dual-vector direction may not be guaranteed to lie in the dual cone unless the gradients are scaled comparably; a normalization step could be needed for other models.","The convergence argument would be stronger if tested on a simple two-objective quadratic where the Pareto front is known, since the paper does not prove DS-MGDA reaches Pareto stationarity for its specific direction.","The dual-space construction is generic and could be applied to other multi-task fine-tuning problems, such as aligning a model while retaining reasoning ability, though the paper does not make that claim."],"forward_implications":["UCE loss removes the need for gradient clipping hyperparameter tuning in GA-based unlearning.","MOLLM's multi-objective step is linear-algebra-based and can be extended to more than three objectives whenever the gradient matrix remains full-rank.","The reported fluency of 1.689 (lower than the original 1.754) suggests unlearning with a common descent direction can, in principle, leave general language ability intact.","If the Pareto-stationarity claim holds, MOLLM offers a stable alternative to retraining-based unlearning in resource-constrained settings."],"supporting_citations":[{"why":"defines the GA-based unlearning loss and KL preservation loss that MOLLM builds on","marker":"[6]"},{"why":"introduces the retain-set cross-entropy loss used as a third objective","marker":"[10]"},{"why":"provides the multiple gradient descent theorem that underpins the common-descent-direction claim","marker":"[21]"},{"why":"supplies the PKU-SafeRLHF dataset on which all methods are evaluated","marker":"[26]"},{"why":"is the GA-OGD baseline that MOLLM is compared against","marker":"[27]"},{"why":"is the Llama 3-8B model used in experiments","marker":"[29]"},{"why":"is the Detoxify tool used for toxicity and obscenity scores","marker":"[30]"},{"why":"is the contrastive framework used for fluency evaluation","marker":"[31]"}],"fun_headline_variants":["Multi-objective unlearning trims harmful LLM data without retraining","MOLLM balances three goals to erase LLM data and retain utility","Gradient-ascent fix: multi-objective LLM unlearning beats existing","New algorithm solves LLM unlearning's explosion and forgetting","Unlearn harmful data, keep fluency with MoLLM's dual space"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire method depends on the unproven premise that the average of the three dual vectors always lands in the dual cone (so it decreases all three objectives) and that the reported gains are not just an artifact of tuning each baseline's learning rate separately.","fun_headline_variants_meta":{"raw":{"variants":["Multi-objective unlearning trims harmful LLM data without retraining","MOLLM balances three goals to erase LLM data and retain utility","Gradient-ascent fix: multi-objective LLM unlearning beats existing","New algorithm solves LLM unlearning's explosion and forgetting","Unlearn harmful data, keep fluency with MoLLM's dual space"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000195,"raw_usage":{"total_tokens":1342,"prompt_tokens":914,"completion_tokens":428,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":530,"completion_tokens_details":{"reasoning_tokens":334}},"tokens_in":530,"tokens_out":428,"duration_ms":4547,"temperature":1.0,"reasoning_tokens":334,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:22:28.425876+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute MOLLM's update direction on a small model and check whether the inner product with each of the three gradients is negative at every step; any non-negative inner product shows the direction is not a common descent direction.","supporting_citations":[{"cited_title":"Steepest descent methods for multicriteria optimization,","cited_arxiv_id":null,"evidence_quote":"provides the multiple gradient descent theorem that underpins the common-descent-direction claim"},{"cited_title":"Beavertails: Towards improved safety alignment of llm via a human-preference dataset,","cited_arxiv_id":null,"evidence_quote":"supplies the PKU-SafeRLHF dataset on which all methods are evaluated"},{"cited_title":"A contrastive framework for neural text generation,","cited_arxiv_id":null,"evidence_quote":"is the contrastive framework used for fluency evaluation"}],"review_version":1}