{"id":"18d3f120-adc5-4340-942e-16e7b2657174","arxiv_id":"2412.16451","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"LANCET uses influence functions to find training examples that drive an LLM's harmful outputs and then fine-tunes the model with a pairwise ranking loss to suppress those outputs without human-labeled corrections.","lead":"This paper introduces LANCET, a two-stage method that identifies which training examples push a large language model toward harmful outputs and then fine-tunes the model to reduce those outputs without human-provided corrections. It reports that this automatic correction matches or beats approaches that rely on human preference data and generalizes to unseen harmful prompts.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"LinFAC influence scores lack validation against true causal effects; if the modular Hessian approximation misranks samples, the IBO correction and interpretability claims rest on unverified scores.","rationale":"The paper's headline contribution is a two-stage pipeline whose first stage is supposed to find the training data responsible for an undesirable output. Every downstream claim—superior generalization, interpretability, 'no human involvement'—depends on those influence scores being at least approximately correct. The reader's weakest_assumption identifies this, and I agree. The experiments provide indirect support: PR curves show LinFAC retrieves injected unsafe samples better than EK-FAC, and the compatibility analysis shows LinFAC+IBO beats EK-FAC+IBO. But those comparisons validate a classification task (unsafe vs safe), not the causal effect sizes that the IBO loss Eq. 15 uses as weights. A method could rank harmful data highly for spurious reasons (e.g., lexical similarity via TF-IDF) while the score magnitudes are miscalibrated. The paper gives no leave-one-out or retraining-based validation, and no error bound on the modular GNH approximation. Because the approximation in Eq. 20 is a strong structural assumption, it is plausible that the rankings are noisy; the fact that LANCET still improves safety could then be attributed to the post-training objective acting on a coarse set of harmful examples rather than to accurate influence attribution. This would not refute the empirical safety gains, but it would invalidate the causal-interpretability framing and weaken the claim that the method outperforms baselines because of better influence estimation. I therefore keep the reader's CONDITIONAL verdict: the paper should be accepted only if the influence scores are validated against ground-truth causal effects or the claims are weakened. My proposed test would settle this directly.","tokens_in":18276,"tokens_out":12044,"duration_ms":109783,"concrete_test":"On a small subset of the Safe RLHF setup (e.g., 50 training examples), compute LinFAC influence scores for a fixed influence query. Then perform leave-one-out validation: for each example, fine-tune the impure model with that example upweighted by a small epsilon (or removed), measure the change in log-probability of the query response, and compare the resulting ranking with LinFAC scores using Spearman correlation and top-10 overlap. If correlation is weak or top-10 overlap is below chance, LinFAC scores are not reliable causal estimates, and the paper's central mechanism is unverified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that LANCET corrects behavior by tracing undesirable outputs back to the training examples that cause them depends on the accuracy of the LinFAC influence scores. The derivation in Eq. 20 approximates the per-sequence gradient sum \\Sigma_t a_t \\otimes D_s_t by a single Kronecker product (1/T \\Sigma a_t) \\otimes (\\Sigma D_s_t) and then factors E[X\\otimes Y] \\approx E[X]\\otimes E[Y], a strong independence assumption with no error bound. The only validation is PR curves for retrieving injected unsafe samples against EK-FAC; this is a coarse classification proxy, not a test of whether the scores reflect true causal effects on the query distribution. If LinFAC misranks, the sets DIF_+ and DIF_- in Eq. 15 are contaminated, so the pairwise loss may reinforce the wrong examples, and the claimed interpretability of 'learning human preferences' is unsupported. The method might still work as a heuristic fine-tune on recalled harmful samples, but then the influence-function mechanism is not load-bearing for the empirical gains.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LANCET, a two-stage method to correct undesirable LLM behavior without human annotation. In the first stage, the new LinFAC influence-function estimator recalls training examples that increase or decrease the probability of undesirable outputs (influence queries). In the second stage, Influence-driven Bregman Optimization (IBO) trains the model with a pairwise loss over positively and negatively influential samples, plus a Bregman divergence term on non-influential samples to preserve utility. The authors evaluate on Safe RLHF, BeaverTails, and Anthropic-HH with Llama 3.1, Llama 2, and OPT backbones, reporting reduced harmfulness on seen and unseen prompts while maintaining helpfulness, and they provide efficiency, compatibility, and case-study analyses.","tokens_in":18569,"tokens_out":10900,"duration_ms":86521,"significance":"If the influence scores are reliable, LANCET offers a plausible human-free alternative to preference-data collection and model unlearning, with the additional appeal of making the recalled training samples interpretable. The paper's strengths include the modular two-stage design, evaluation across three datasets and three backbones, a compatibility matrix (Table 7), explicit wall-clock comparisons (Figure 4e), and a concrete case study (Table 8). The central risk is that both the recall stage and the correction signal depend on an unvalidated Kronecker-factor approximation, and the evaluation pipeline reuses the same reward/cost models for query selection, checkpoint selection, and reporting. These issues are addressable with additional validation and ablations.","major_comments":[{"comment":"The load-bearing premise of the method is that LinFAC influence scores rank training samples by their causal effect on the query output distribution. This premise is not validated. Eq. (20) replaces the sequence-level sum of Kronecker products \\sum_t a_t \\otimes Ds_t with a single Kronecker product (1/T \\sum_t a_t) \\otimes (\\sum_t Ds_t), and Eq. (21) then replaces an expectation of an outer product by a product of expectations. Neither step is an equality, and no error bound or small-scale comparison against leave-one-out retraining is provided. The PR curves in Figure 4(a-c) only measure recovery of injected unsafe samples under a coarse classification metric, so they do not establish that the scores reflect the true effect on p(z_r|z_p). Because DIF_+ and DIF_- in Eq. (15) are built directly from these scores, contaminated rankings can propagate into the IBO loss and the interpretability claims. I would like to see (i) a comparison of LinFAC rankings with exact leave-one-out influence or full-Hessian influence on a small model, reported by rank correlation or NDCG, and (ii) an ablation that replaces LinFAC with random or TF-IDF recall inside IBO to quantify how much of the behavioral improvement comes from the influence ranking.","section":"Section 3.1, Appendix A.2, Eqs. (9)-(13), (20)-(22)"},{"comment":"The evaluation protocol shares the same reward/cost model signal across query selection, checkpoint selection, and final evaluation. Section 4.1 selects influence queries by the largest increase in harmfulness according to the cost model; Appendix D.2 selects the best post-training checkpoint by \"the maximal RM-CM score on a valid set\"; and Appendix D.3 computes the reported Helpful/Harmless scores with the same family of RM/CM models. This creates a selection-on-the-evaluation-metric risk: the reported gains, especially on unseen prompts, may be inflated by optimizing the same scores used for measurement. Please evaluate with a held-out set of human ratings on unseen prompts, or with a different reward/cost model that was not used for selection, and state explicitly that the validation set used for checkpointing is disjoint from the test set.","section":"Section 4.1, Appendix D.2, Appendix D.3"},{"comment":"Several hyperparameters that directly control the correction strength are not reported. The final objective in Eq. (17) contains the proximity regularizer weight \\lambda; Eq. (15) contains \\epsilon and the Pareto threshold \\alpha; Eq. (13) has the batch/cluster size K; and the recall pipeline has a TF-IDF recall size. The text states only that \"we set \\epsilon = -1\" and that the model is trained for 4000 steps. Without the values of \\lambda, \\alpha, and K, and without a sensitivity analysis for \\lambda and \\alpha, the method cannot be reproduced and the robustness of the reported gains cannot be assessed. Please report all hyperparameter values and include a sensitivity study for at least \\lambda and \\alpha.","section":"Section 3.2, Appendix D.2"},{"comment":"The motivating problem is correcting behaviors caused by outdated or erroneous human preferences, but the experiments operationalize \"inappropriate\" exclusively as harmful or unsafe content. The paper acknowledges this in Section 4.1 (\"no existing data is available to simulate this scenario\"), yet the abstract and introduction claim corrections that make LLMs deviate from contemporary human preferences and societal norms. Harmfulness is a reasonable proxy, but it is not the same as preference drift: a preference shift need not involve safety, and the mechanism of learning new preferences differs from suppressing unsafe responses. I recommend either adding a preference-drift experiment (for example, flipping chosen/rejected pairs in a preference dataset over time) or explicitly narrowing the claims in the abstract and conclusion to unsafe-behavior correction until such evidence exists.","section":"Section 1, Section 4.1"},{"comment":"On the seen influence queries, the correction is partly by construction. Eq. (15) optimizes a pairwise loss constructed from the influence of each recalled training sample on the query log-probability, so reducing the probability of those exact queries is the designed effect and should not be reported as an independent discovery. The paper does evaluate unseen prompts, which is the right test, but the current ablations do not isolate the contribution of the influence ranking. Table 7 varies the recall strategy (LinFAC vs. EK-FAC) and the post-training objective (IBO vs. PBO), but it does not include a random-recall or TF-IDF-only baseline with IBO. Please add such an ablation, and report the human Likert or a held-out metric on unseen prompts for it; this is necessary to support the claim that influence-function recall, rather than fine-tuning on recalled harmful data, drives the generalization gains.","section":"Section 3.2 Eq. (15), Table 7"}],"minor_comments":[{"comment":"There are typos and naming inconsistencies, including \"an novel\" in the abstract, \"Bregman Diversity\" in Appendix A.1, \"deviding\" in Section 3.2, and inconsistent \"Linear-FAC\"/\"LinFAC\" usage; please proofread the manuscript.","section":"Throughout"},{"comment":"The normalization in Eq. (21) seems inconsistent: the left factor carries an extra 1/N while Eq. (22) assigns 1/N to \\hat S; please make the factor derivation internally consistent.","section":"Appendix A.2, Eq. (21)"},{"comment":"The plot lacks axis labels, and the text describes \"positive intervention (epsilon=+1)\" and \"negative intervention (epsilon=-1)\", but Eq. (15) sets epsilon=-1 in the loss; the sign convention should be clarified.","section":"Section 4.4, Figure 4(f)"},{"comment":"The PR curves do not state the retrieval cutoff or the number of candidate samples used to compute precision and recall; please specify these details so the comparison with EK-FAC is reproducible.","section":"Figure 4(a-c)"},{"comment":"Anthropic-HH is cited as (Ji et al. 2024); the correct source appears to be Bai et al. (2022a); please correct the citation.","section":"Appendix D.1"},{"comment":"The statement that batch query improves performance is not uniformly supported by the table, since the IF-Query Harmless score is higher (worse) for LANCET_B (0.616) than for LANCET_S (0.581); please clarify which metrics support the claim.","section":"Table 6"},{"comment":"No significance tests are reported; given the overlapping standard deviations in several rows, please add statistical tests or confidence intervals for the headline comparisons.","section":"Tables 4 and 5"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within scope for a machine-learning venue and the empirical effort is substantial, but the load-bearing LinFAC approximation and the shared RM/CM evaluation signal need direct validation before the central claims can be accepted. I do not see evidence of fabrication; the main concerns are methodological and reproducibility-related."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a real attempt at automating LLM behavior correction via influence functions, and the experiments mostly back it up. It deserves a serious referee, but it also needs another round of hard work before I would trust the mechanism claims.\n\nWhat is new: LinFAC computes influence over whole sequences and uses the transformer sublayer as the computational unit, which is a sensible adaptation of EK-FAC to modern architectures. IBO turns influence rankings into a pairwise loss, which is a clean way to avoid gradient ascent. The batch-query trick to stabilize negative influence scores is a nice empirical fix. Across Safe RLHF, BeaverTails, and Anthropic-HH, on three backbones, the method consistently lowers harmfulness while keeping helpfulness close to the impure model, and it beats both human-correction SFT/DPO baselines and unlearning baselines on unseen prompts. Those are real measurements.\n\nSoft spots: the load-bearing part of the method, the LinFAC influence scores, is validated only by PR curves for retrieving injected unsafe samples. That is a retrieval test, not a test of whether the scores track true causal effects on the query distribution. The approximation in Eq. 20-21 moves from a sum of Kronecker products to a Kronecker product of sums, then factors an expectation of a Kronecker product as a Kronecker product of expectations. That is a strong independence assumption, and there is no error bound or leave-one-out validation. If the scores misrank, both the recall stage and the IBO preference pairs are contaminated.\n\nRelatedly, for seen influence queries the correction is partly by construction: the IBO loss is built from the very scores that say these samples raise the query probability. The generalization results on unseen prompts are the independent evidence, and they are encouraging, so this is not fatal, but it deserves a clearer statement in the paper.\n\nThe paper also leaves out several hyperparameters (lambda, alpha, the Pareto threshold, K for clustering, checkpoint selection details), and it does not ship code. Checkpoint selection on the same RM/CM signal used for evaluation is a selection-bias worry, though not disqualifying.\n\nThe “no human involvement” claim is too strong. The pipeline replaces human labels with cost-model scores and a human-chosen notion of unsafe data. That is cheaper, but it is not “no human involvement.”\n\nBottom line: the method is plausible and the experiments are broad enough that this deserves refereeing. I would want code and a clearer validation of the influence scores before endorsing the causal story, but the empirical case is solid enough to engage with.","headline":"LANCET is a solid empirical contribution to automatic LLM behavior correction, but the influence-score approximation needs direct validation before the causal mechanism is credible.","tokens_in":19048,"tokens_out":2550,"would_cite":false,"duration_ms":22169,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that LANCET, a two-phase influence-function method, can correct undesirable LLM behavior without human intervention by tracing bad outputs to influential training samples and re-optimizing on their influence ranking.","keywords":["influence functions","LLM behavior correction","model unlearning","direct preference optimization","Gauss-Newton Hessian","Kronecker-factored approximation","safe alignment","out-of-distribution generalization"],"falsifier":"Train a small transformer on contaminated data, delete each of the top-100 recalled training examples one at a time, retrain, and measure the change in the harmful query's probability; if the rank correlation between true retraining effects and LinFAC scores is near zero, the recall stage supplies wrong data and IBO's correction signal is built on noise.","tokens_in":18110,"feed_emoji":"🎯","tokens_out":6316,"duration_ms":51837,"temperature":0.7,"pith_summary":"The paper sets out to correct behaviors a large language model learned from outdated or inappropriate training data without collecting fresh human feedback. It claims that influence functions can locate the training examples most responsible for a given undesired output, and that a new post-training objective can then reshape the model so those outputs become unlikely while its general utility is preserved. If true, model behavior could be kept in step with changing social norms at the cost of computing influence scores rather than paying annotators. The paper reports that its pipeline outperforms both human-correction alignment baselines and gradient-ascent unlearning methods, especially on out-of-distribution harmful prompts.","feed_headline":"No-human pipeline finds and fixes harmful LLM training data","feed_subtitle":"Influence scores trace bad outputs back to the training samples behind them, then re-rank them safely.","key_machinery":"The central object is the influence score of a training sample on an influence query, computed through the Proximal Bregman Response Function. LinFAC makes it tractable for LLMs by treating each Transformer sublayer as one linearized module, summing over token activations and pre-activation pseudo-gradients so token dependencies are kept, and approximating the modular Gauss-Newton Hessian as a Kronecker product of sequence-summed factors $\\hat{A}\\otimes\\hat{S}$. The forward pass is carried by the inverse-Hessian-vector product $\\hat{G}^{-1}(\\sum_t a_t^{z_q}\\otimes D_s^{z_q}_t)$, which reduces to small matrix solves. IBO then converts the ranked scores into a pairwise correction loss, a Bregman-divergence forgetfulness penalty, and a proximity regularizer.","core_discovery":"LANCET treats behavior correction as two linked tasks. First, LinFAC scores every recalled training sample by how much it increases or decreases the probability of the unwanted response, using a linearized Transformer sublayer and a Kronecker-factored approximation of the Gauss-Newton Hessian. Second, Influence-driven Bregman Optimization (IBO) turns those scores into a pairwise ranking objective, much like direct preference optimization but with influence rankings replacing human preference labels, and adds a Bregman-divergence regularizer on non-influential samples so the model does not forget its other knowledge. The paper reports that this pipeline lowers harmful responses on seen and unseen prompts across Safe RLHF, BeaverTails, and Anthropic-HH, with smaller utility loss than gradient-ascent unlearning and stronger out-of-distribution generalization than SFT or DPO retrained on human-corrected data.","pith_inferences":["If the ranking assumption holds, the same machinery could serve as a training-data audit: influence scores would flag stale, poisoned, or otherwise harmful examples even before a specific bad output is observed.","The case study's pairing of top-1 and last-1 samples suggests correction may be driven by semantic opposition of instructions; a direct test would compare IBO against a variant that pairs samples with opposite labels but shuffles influence magnitudes.","One could test a cheaper deployment: use LinFAC only for recall and replace IBO with standard DPO on synthesized pairs; if correction holds, the expensive Hessian machinery is only needed for data selection, not for training.","The paper's safety-focused benchmarks leave open whether the same method corrects non-safety preference drift, such as style or factual recency, which is a natural next experiment."],"forward_implications":["If LinFAC rankings are faithful, harmful behavior can be traced to specific training examples without manual annotation, and corrected by editing model behavior rather than retraining from scratch.","IBO's pairwise influence-ranking objective can be trained with samples from different prompts, so correction signals are not limited to same-prompt preference pairs.","The approach should plug into any LLM whose raw training set is accessible, including continual-alignment pipelines where old preference data becomes outdated.","Batch-query IHVP with clustered influence queries reduces computation and improves correction by averaging out cross-harmful-behavior noise, suggesting influence estimation should be done over sets of queries rather than single prompts.","According to the paper, LANCET lowers harmfulness on out-of-distribution prompts more than methods that collect human-corrected data, implying influence-based correction transfers beyond the observed bad outputs."],"supporting_citations":[{"why":"Supplies the original influence-function definition that LANCET generalizes to language model behavior.","marker":"Hampel 1974"},{"why":"Provides the Proximal Bregman Response Function that corrects influence-function bias and gives the formula LANCET computes.","marker":"Bae et al. 2022"},{"why":"Introduces EK-FAC, the Kronecker-factored curvature approximation that LinFAC extends and compares against.","marker":"George et al. 2018"},{"why":"Scales influence functions to LLMs and supplies the TF-IDF recall procedure and heavy-tail observation used in LANCET.","marker":"Grosse et al. 2023"},{"why":"Provides the basis for Kronecker-factored approximate curvature on modern architectures, supporting LinFAC's modular Gauss-Newton Hessian.","marker":"Eschenhagen et al. 2023"},{"why":"Direct Preference Optimization is the pairwise preference-learning template that IBO adapts to influence rankings.","marker":"Rafailov et al. 2023"},{"why":"Safe RLHF supplies the main experimental dataset and the reward and cost models used to measure helpfulness and harmfulness.","marker":"Dai et al. 2023"},{"why":"Large Language Model Unlearning is the primary unlearning baseline whose gradient-ascent limitations LANCET aims to overcome.","marker":"Yao, Xu, and Liu 2024"}],"fun_headline_variants":["Influence functions trace bad outputs to fix LLM behavior","No-human pipeline: influence-based rankings correct LLMs","LANCET realigns LLMs by ranking problematic training samples","Ranking training influence replaces human labels for LLM fixes","Automatic LLM correction via influence-driven Bregman optimization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything depends on the assumption that LinFAC's approximate influence scores rank training examples in the same order as actually deleting each example and retraining the model would; the paper gives precision-recall curves against known unsafe labels but never directly checks this ranking against real retraining.","fun_headline_variants_meta":{"raw":{"variants":["Influence functions trace bad outputs to fix LLM behavior","No-human pipeline: influence-based rankings correct LLMs","LANCET realigns LLMs by ranking problematic training samples","Ranking training influence replaces human labels for LLM fixes","Automatic LLM correction via influence-driven Bregman optimization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000232,"raw_usage":{"total_tokens":1474,"prompt_tokens":911,"completion_tokens":563,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":527,"completion_tokens_details":{"reasoning_tokens":480}},"tokens_in":527,"tokens_out":563,"duration_ms":6080,"temperature":1.0,"reasoning_tokens":480,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:35:11.672798+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a small transformer on contaminated data, delete each of the top-100 recalled training examples one at a time, retrain, and measure the change in the harmful query's probability; if the rank correlation between true retraining effects and LinFAC scores is near zero, the recall stage supplies wrong data and IBO's correction signal is built on noise.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the original influence-function definition that LANCET generalizes to language model behavior."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Proximal Bregman Response Function that corrects influence-function bias and gives the formula LANCET computes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces EK-FAC, the Kronecker-factored curvature approximation that LinFAC extends and compares against."},{"cited_title":"E.; Schneider, F.; and Hennig, P","cited_arxiv_id":null,"evidence_quote":"Provides the basis for Kronecker-factored approximate curvature on modern architectures, supporting LinFAC's modular Gauss-Newton Hessian."},{"cited_title":"D.; Ermon, S.; and Finn, C","cited_arxiv_id":null,"evidence_quote":"Direct Preference Optimization is the pairwise preference-learning template that IBO adapts to influence rankings."}],"review_version":1}