{"id":"c7329ffe-c877-40ef-83ca-465ea465b6b7","arxiv_id":"2507.18631","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"LARF ranks fine-tuning samples by how close their hidden representations lie to unsafe versus safe reference responses, and removing the top-ranked samples preserves safety alignment.","lead":"The authors present a layer-aware method to filter fine-tuning data that can silently weaken an LLM's safety guardrails. The result is a cheap pre-filter that could make custom fine-tuning of aligned models safer.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"LARF's score may be a proxy for response length/format; no length-only baseline in Tables 1/5 leaves the safety-specific mechanism unproven.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing concern: the representation similarity score may be proxying response length and point-by-point formatting rather than safety-degrading semantics. The paper's own Section 4.5 and Appendix E.1 show that top-ranked samples are consistently long and structured, yet no control baseline selects by these surface features. This is not an external disagreement but an internal underdetermination: the evidence presented cannot distinguish the proposed safety-sensitive mechanism from a simple length filter. The concern is load-bearing because the abstract and introduction claim that safety-degrading features are 'not easily identifiable on the surface,' while the paper's own analysis shows they are. If the proposed concrete test finds that length-only filtering reproduces LARF's ASR effects, the central claim would be unsupported and the method reduces to a length heuristic. Until such a control is run, a conditional verdict is appropriate. The paper does have real strengths: the method is efficient, reproducible, and the experiments are broad; but the missing length/format baseline leaves the mechanistic claim unproven. I agree with the reader's assessment and recommend keeping the verdict conditional, pending the control experiments described above.","tokens_in":26489,"tokens_out":4765,"duration_ms":49918,"concrete_test":"Add two baselines to Table 1, Table 5, and Table 2: (i) rank samples by response token length only; (ii) rank by point-style format only (e.g., presence of numbered/bulleted lists). Fine-tune on the same top/bottom 1,000 (or remove top 2,000) and compare ASR/GPT Score. Also compute the Spearman correlation between LARF's score and response length across each dataset; report it. If the length-only baseline achieves ASR within the run-to-run variance of LARF (e.g., ±3 points) or the Spearman rho exceeds ~0.8, the score is a length/format proxy and the central mechanistic claim is unsupported. Additionally, repeat the downstream experiments removing the 2,000 longest responses instead of LARF's top-ranked samples; if safety and utility outcomes match Table 2, LARF's layer-aware component is not needed.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that LARF identifies benign samples with safety-degrading features via safety-sensitive layer representations, and removing them mitigates alignment loss. The load-bearing assumption is that the score in Eq. 13 (sim(r_i, r_unsafe) − sim(r_i, r_safe)) isolates safety-degrading content rather than surface style. The reference representations r_safe and r_unsafe are means over final-<eos> hidden states of safe refusals and unsafe completions to the same harmful instructions (Sec. 3.3, App. C.2). Safe refusals are typically short templated rejections; unsafe completions from an uncensored model are long, detailed, point-by-point answers. A test sample's final-token representation encodes the style and length of its response. Hence the score will rank long, structured, point-by-point responses high, and short refusal-like responses low. This is precisely what the paper finds in Sec. 4.5/Table 4: top-1000 samples have 2-3x the average token length and >50% point-style responses across models/datasets (App. E.1). The paper provides no baseline that selects by length or point-style format alone. Therefore Tables 1 and 5 cannot distinguish 'safety-degrading semantics' from 'long/structured responses that, when fine-tuned, degrade safety.' If the latter holds, the representation mechanism and the safety-sensitive-layer interpretation add nothing over a trivial length filter, contradicting the abstract's claim that these features are 'not easily identifiable on the surface.' The downstream utility results (Table 2) also lack a length-only removal baseline, so the mitigation benefit of LARF is not isolated from removing long verbose samples. This is a correctness risk for the central claim, not merely a missing robustness check.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LARF, a two-stage method to filter safety-degrading samples from benign fine-tuning datasets. In the first stage, LARF identifies a 'safety-sensitive layer' by scaling each layer's attention and feedforward parameters by 1±α and selecting the layer whose scaling produces the largest change in refusal counts on a small overrejection dataset (Section 3.2). In the second stage, LARF computes the final-<eos>-token hidden representations at that layer for a test sample and scores it as sim(r_i, r_unsafe) − sim(r_i, r_safe), where r_unsafe and r_safe are mean representations of harmful completions and safe refusals to the same harmful instructions (Eq. 13). The authors demonstrate on Llama3, Llama3.1, and Qwen2.5 that fine-tuning on the 1,000 highest-scoring Alpaca or Dolly samples raises Attack Success Rate (ASR) on HarmBench, HEx-PHI, and DirectHarm4 much more than random sampling, while fine-tuning on the 1,000 lowest-scoring samples keeps ASR near zero (Tables 1 and 5). They also show that removing the top 2,000 ranked samples from 10,000-sample downstream corpora reduces safety degradation without hurting task performance (Table 2), and they report computational efficiency gains over SEAL, GradSafe, and Bi-Anchoring (Table 3). The paper concludes that LARF effectively identifies stealthy safety-degrading data and mitigates alignment loss when such data are removed.","tokens_in":26981,"tokens_out":4147,"duration_ms":43646,"significance":"If the central claim holds, LARF provides a practical, training-free, gradient-free data filter that could be used before fine-tuning aligned LLMs, and it would complement existing alignment-preservation methods. The paper has several strengths: it evaluates across three base models and five datasets, includes a downstream-utility analysis, reports wall-clock and memory costs, shows transferability to 32B/70B models, and releases code. The representation-similarity idea is simple and cheap, and the bidirectional scoring (safe minus unsafe similarity) is a sensible way to approximate a refusal direction. However, the significance critically depends on whether the score isolates safety-degrading semantics rather than surface response properties such as length or point-by-point formatting; the paper's own analysis (Section 4.5, Appendix E.1) shows the top-ranked samples are disproportionately long and point-style, and the absence of a length/format control makes the core mechanistic claim unproven. The downstream tables also contain a dataset-size confound.","major_comments":[{"comment":"The paper's own data characterization shows that the top-ranked samples are predominantly long, point-by-point responses: Table 4 reports point-style counts of 516-872 and average output lengths of 333-354 tokens versus dataset averages of 276 and 138 tokens on Alpaca, and Appendix E.1 confirms this across all five datasets. The reference representations r_safe and r_unsafe are means over final-<eos> hidden states of short templated refusals and long detailed completions to the same instructions (Section 3.3, Appendix C.2), so the score in Eq. 13 is likely to rank any long, structured response highly regardless of its safety-relevant semantics. The manuscript provides no baseline that selects samples by output length or point-style format alone, so Tables 1 and 5 cannot distinguish the claim that LARF identifies safety-degrading features from the alternative that it simply identifies long structured responses whose inclusion degrades safety. I request the addition of a length-only (or point-style-only) ranking baseline across the same models, datasets, and benchmarks, and a re-analysis showing that LARF outperforms such a baseline or that the representation score adds signal beyond these surface features.","section":"4.5, Table 4; Appendix E.1; Eq. 13"},{"comment":"The downstream-task comparison is confounded by training-set size. The random baseline is fine-tuned on all 10,000 sampled data points, while each filtering method (LARF, SEAL, Bi-Anchoring) removes the top 2,000 ranked samples and is therefore fine-tuned on only 8,000 examples. The safety improvements reported for LARF in Table 2 (e.g., DirectHarm4 GPT Score 1.46 vs 1.68 for Llama3.1 Magicoder) could simply reflect the effect of training on fewer examples, which is a known factor in fine-tuning-related safety degradation. A control that removes 2,000 randomly selected samples, or a comparison in which all methods train on the same number of examples (e.g., 8,000) while only varying which samples are kept, is needed to support the conclusion that LARF's filtering, rather than the reduced dataset size, mitigates safety degradation.","section":"Table 2; Section 4.4"},{"comment":"Except for the Random baseline, which is averaged over three runs, all reported ASR values for LARF, SEAL, GradSafe, and Bi-Anchoring come from a single fine-tuning run per condition. Since fine-tuning is stochastic and the differences between methods are sometimes modest (e.g., Table 1, Llama3.1 Alpaca HEx-PHI: SEAL 3.45% vs Bi-Anchoring 3.10%; Table 5, Llama3.1 Dolly DirectHarm4: LARF 71.75% vs SEAL 71.75%), the absence of variance estimates or significance testing weakens the paper's 'State-of-the-art' claim. I request at least 3 seeds for the headline comparisons (or an explanation of why single runs are sufficient), and error bars or confidence intervals for the main tables.","section":"4.3.1; Tables 1 and 5"},{"comment":"The safety-sensitive layer is selected using an overrejection dataset of only 110 instructions (Appendix C.1), and the layer's suitability is validated post hoc by showing that the selected layer yields the highest ASR when fine-tuning on its top-ranked samples (Figure 5). Because the same overrejection dataset and layer-selection criterion are used to pick the layer that is then deployed in all main experiments, there is a risk of selection overfitting to this small calibration set. The paper does not report how stable the identified layer is to resampling or varying the overrejection dataset, nor does it show results for a few neighboring layers (e.g., 12, 14, 17, 19 for the respective models) in Tables 1/5. A robustness analysis would considerably strengthen the claim that the identified layer is genuinely safety-sensitive rather than an artifact of a 110-example set.","section":"3.2; Figures 4-5; Appendix C.1"}],"minor_comments":[{"comment":"There are pervasive typos and inconsistent labels: 'overrecjtion' (Appendix C.1), 'GardSafe' (Table 3), 'XHE-PHI' in Figures 3 and 13-15 vs 'HEx-PHI' elsewhere, 'insturct' in Figure 17, 'XEx-PHI' in Figure 19, and 'ths loss' in Section 4.3.2. I recommend a careful proofreading pass.","section":"Throughout"},{"comment":"The presentation of Table 4 is confusing: the row labeled 'Avg' contains two numbers (276 and 138), which are presumably the point-style count and output-token average for the full dataset, but the table structure does not make this clear. Please reformat the table with explicit column headers and separate rows for 'Dataset average' and each model.","section":"Table 4"},{"comment":"For Bi-Anchoring, the paper states it uses the same D_unsafe and D_safe reference datasets as LARF (Appendix C.2), but the method description says it computes the loss gradient over all model parameters using the first 10 tokens of the response; this is a reasonable setup, but a precise reference to the Bi-Anchoring hyperparameters (number of tokens, gradient projector, etc.) would aid reproducibility.","section":"Section 4.3.1 and Appendix C.2"},{"comment":"The comparison between bidirectional (Orig) and unidirectional (Unsafe) scoring in Figure 3 is presented only for Llama3 in the main text, with other models deferred to Appendix D.1. Since this ablation supports the design choice in Eq. 13, at least one additional model should be shown in the main text, and the appendix figures should use consistent label ordering ('Top (Orig)', 'Bottom (Orig)', etc.) to avoid confusion.","section":"Section 3.3 and Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The central concern is whether LARF is essentially a length/style filter in disguise. The paper's own Table 4 makes this worry concrete, and the missing length-only baseline is a standard control that the authors can run at low cost. If that baseline reproduces the ASR patterns, the paper's contribution shrinks to a simple observation about long structured responses, which would be a much weaker result than claimed. I would also flag the dataset-size confound in Table 2, which could undermine the practical-filtering claim even if the ranking results in Table 1 survive. The authors have the tools to address both issues in a revision, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: this paper proposes LARF, a cheap, training-free filter for removing benign-looking fine-tuning data that degrades safety alignment, and the experiments suggest it does that job. But the paper's own analysis undercuts its headline claim. The top-ranked samples turn out to be long, point-by-point responses, and there is no baseline that filters by length or format alone. So the safety-specific mechanism remains unproven.\n\nWhat is new: the bidirectional similarity scoring over representations at a safety-sensitive layer, applied to data filtering, is a sensible combination. It avoids the expensive ranker training of SEAL and the noisy full-model gradients of Bi-Anchoring. The runtime table is honest: 0.5 hours on one GPU versus 3-6 hours on multiple GPUs for baselines. The evaluation is reasonably broad with three models, three safety benchmarks, and three downstream tasks, and the code is released. That is real work and deserves credit.\n\nWhere it wobbles: the stress-test concern is correct. The reference sets are short refusals versus long unsafe completions, so a sample's final-token representation plausibly scores high just because it is long and structured. The paper demonstrates exactly that pattern in Section 4.5, then calls it a hypothesis, but never runs the obvious control: filter by length, or by point-style formatting, and see if the ASR effect disappears. Without that, Tables 1 and 5 cannot distinguish semantics from surface style. The downstream utility results have the same gap. Also, most conditions are single runs; only Random is averaged, so there is no sense of variance. The overrejection dataset used to pick the safety-sensitive layer is only 110 instructions, which is small but acceptable because the layer choice is validated independently on Alpaca. That is a moderate concern, not a fatal one.\n\nBottom line: LARF may well work as a practical filter, and the paper is worth engaging with. But the abstract's claim that the flagged features are 'not easily identifiable on the surface' is contradicted by the paper's own data. This deserves a serious referee: send it to peer review with the demand for a length/format baseline, repeated runs with confidence intervals, and a softened mechanism claim if the length control shows a trivial filter does the same.","headline":"A useful, cheap data filter whose mechanism is unproven: the top-ranked samples are long point-by-point responses, and no length/format baseline exists to rule out a trivial explanation.","tokens_in":27399,"tokens_out":3572,"would_cite":true,"duration_ms":36448,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LARF claims that filtering fine-tuning data by representation similarity at a safety-sensitive layer removes benign-looking samples that degrade LLM safety alignment.","keywords":["safety alignment","fine-tuning","data filtering","representation similarity","safety-sensitive layers","attack success rate","LLM safety","LARF"],"falsifier":"Construct a control set of 1,000 fine-tuning samples that match LARF's top-ranked samples in output length and point-by-point formatting but are drawn from the lowest-scoring region of the representation score; if fine-tuning on this control reproduces the ASR jump seen with LARF's top set, then the representation score adds nothing beyond a length/format filter.","tokens_in":26321,"feed_emoji":"🛡️","tokens_out":8001,"duration_ms":67271,"temperature":0.7,"pith_summary":"Fine-tuning an aligned LLM on seemingly benign instruction data can still erode its safety guardrails. This paper claims that such datasets contain 'safety-degrading' samples—benign-looking instruction–response pairs that push the model toward complying with harmful prompts—and that these samples can be identified cheaply before training. The proposed method, LARF, locates the model layer whose parameter scaling most changes refusal behavior, then ranks each fine-tuning sample by how close its hidden representation at that layer sits to unsafe versus safe reference examples. Removing the top-ranked samples, the paper reports, largely restores the refusal behavior that fine-tuning would otherwise destroy, with no measurable loss on downstream task quality. If true, this gives practitioners a training-free pre-deployment audit tool for protecting alignment.","feed_headline":"LARF finds hidden data that silently breaks LLM safety","feed_subtitle":"Deleting 1,000 hard-to-spot samples from fine-tuning data keeps LLM refusal behavior intact.","key_machinery":"The two load-bearing mechanisms are (1) safety-sensitive layer identification and (2) bidirectional representation scoring. Layer identification: for each candidate layer, parameters are scaled by (1±α) and the normalized change in refusal count on an overrejection dataset is computed; the layer with maximal normalized change rate k is deemed safety-sensitive. Scoring: at that layer, the final-token hidden representation of each data sample is compared by cosine similarity against the mean representation of unsafe reference examples (harmful instruction with compliant answer) and safe reference examples (same instructions with refusal); the safety-degrading score is the difference sim(unsafe) − sim(safe). The argument is that the refusal direction lives in this layer's representation space, so samples that sit closer to unsafe completions and farther from refusals are exactly the ones that, when used for fine-tuning, erode the model's rejection capability.","core_discovery":"The paper's central claim is that safety alignment in LLMs is concentrated in specific layers, and that the hidden representations at those layers encode whether a data sample will weaken refusal behavior during fine-tuning. LARF operationalizes this: it scales each layer's attention and feedforward parameters up and down, measures the resulting change in refusal count on an overrejection dataset, and picks the layer with the largest normalized change rate (for Llama3 and Llama3.1 this is layer 13; for Qwen2.5 it is layer 18). At that layer it computes mean representations of a small set of harmful-instruction-plus-compliant-response examples and of the same instructions paired with refusals, then scores each test example by cosine similarity to the unsafe mean minus similarity to the safe mean. Fine-tuning on the 1,000 highest-scoring Alpaca samples raises Attack Success Rate on HarmBench from 3.5% to 39% for Llama3.1, while fine-tuning on the 1,000 lowest-scoring samples drives ASR to 0%; removing the top-ranked samples from downstream datasets lowers harmfulness scores on DirectHarm4 without hurting performance on HumanEval, PubMedQA, or MATH.","pith_inferences":["An editorially added testable extension: ablate the score by replacing it with a pure length/point-by-point proxy; if the proxy matches LARF's ASR results, the representation layer choice is redundant for this failure mode.","The reference sets are drawn from the Circuit Breaker taxonomy; in principle the score could be re-pointed at other alignment targets (honesty, helpfulness, privacy) by changing the reference corpora, a direction the paper leaves open.","The paper's layer-scaling procedure is model-specific and requires an overrejection dataset; the same sensitivity analysis might be used to monitor how safety layers shift after each fine-tuning run, enabling iterative re-filtering."],"forward_implications":["If the central claim holds, practitioners can filter fine-tuning corpora with a single forward pass through the model, needing no gradient computation and no dedicated safety ranker.","Removing LARF's top-ranked 1,000 samples from a 10,000-sample downstream dataset (Magicoder, PubMedQA, MetaMath) keeps task performance within 1% of random sampling while consistently lowering harmfulness scores.","Layer selection transfers across datasets: the same safety-sensitive layer (e.g., layer 13 for Llama3.1) ranks safety-degrading data effectively on Alpaca and Dolly, and the method works on Llama3.1-70B, Qwen2.5-32B and Qwen2.5-72B.","Fine-tuning on the bottom-ranked 1,000 samples can actually bring ASR below the instruct baseline, suggesting that LARF's score identifies not only harmful data but also data that actively reinforces refusal behavior."],"supporting_citations":[{"why":"Supplies the overrejection dataset construction and layer-scaling procedure used to identify safety-sensitive layers.","marker":"(Li et al., 2025c)"},{"why":"Provides the Circuit Breaker training dataset whose 20 harm categories define the Dsafe and Dunsafe reference examples.","marker":"(Zou et al., 2024)"},{"why":"Establishes that fine-tuning on benign data compromises safety alignment, the problem LARF is built to solve.","marker":"(Qi et al., 2024)"},{"why":"Introduces Bi-Anchoring, the gradient-based baseline LARF is compared against, and notes the point-by-point response pattern.","marker":"(He et al., 2024)"},{"why":"Provides SEAL, the trained-ranker baseline that LARF outperforms without requiring extra training.","marker":"(Shen et al., 2025)"},{"why":"Provides GradSafe, a gradient-based safety detection baseline using only instruction gradients.","marker":"(Xie et al., 2024)"},{"why":"Motivates the representation-based approach by showing refusal behavior is mediated by a single direction in representation space.","marker":"(Arditi et al., 2024)"}],"fun_headline_variants":["Layer-aware filtering spots unsafe fine-tuning samples","How to keep LLMs safe while fine-tuning on benign data","LARF: Remove hidden samples that wreck LLM safety","Data purification preserves LLM refusal behavior","Hidden safety-breakers in fine-tuning data exposed"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the score's ranking reflects safety-critical content rather than just long, point-by-point responses, since the paper's own analysis shows top-ranked samples are overwhelmingly long and structured.","fun_headline_variants_meta":{"raw":{"variants":["Layer-aware filtering spots unsafe fine-tuning samples","How to keep LLMs safe while fine-tuning on benign data","LARF: Remove hidden samples that wreck LLM safety","Data purification preserves LLM refusal behavior","Hidden safety-breakers in fine-tuning data exposed"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000552,"raw_usage":{"total_tokens":2660,"prompt_tokens":999,"completion_tokens":1661,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":615,"completion_tokens_details":{"reasoning_tokens":1587}},"tokens_in":615,"tokens_out":1661,"duration_ms":11138,"temperature":1.0,"reasoning_tokens":1587,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:09:08.414796+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a control set of 1,000 fine-tuning samples that match LARF's top-ranked samples in output length and point-by-point formatting but are drawn from the lowest-scoring region of the representation score; if fine-tuning on this control reproduces the ASR jump seen with LARF's top set, then the representation score adds nothing beyond a length/format filter.","supporting_citations":[],"review_version":2}