{"id":"ea46de8e-aa9c-47e2-9bc4-b71db6ac247c","arxiv_id":"2504.16120","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"A BART-based post-generation corrector lowers toxicity and jailbreaking scores, but the reported gains are partly in-sample because thresholds are optimized on the evaluation data.","lead":"This paper proposes a post-generation safety filter that uses a fine-tuned BART model to rewrite toxic or jailbroken LLM responses. It reports large reductions in toxicity and jailbreak scores across GPT-4, PaLM2, Mistral-7B, and Gemma-2b-it, but the evaluation is weakened by thresholds fitted on the same test data.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline reductions are in-sample threshold optima: Eq. 1 chooses tau on the same test responses used for Table 3, so the 15%/21% numbers are maxima over thresholds, not expected deployment gains.","rationale":"The reader's weakest assumption, that the LangKit similarity score is a valid and comprehensive toxicity/jailbreak measure, is a real concern, but the more load-bearing flaw is internal validity: the threshold tau* is selected on the test data that later generates the headline percentages. Because Eq. 1 optimizes the exact quantity reported, Table 3 is an in-sample fit, not a predictive evaluation. Any threshold-based system will show large reductions on the data used to choose the threshold, and Section 4.1.2 admits this is the same dataset as Section 4.3. This problem would remain even if the LangKit score were a perfect toxicity measure, so I consider the reader's analysis only partially aligned with mine: the metric concern matters, but threshold overfitting alone is sufficient to withhold the strong causal claim. The internal inconsistencies between the abstract and the contributions section (GPT-4 15%/21% vs. 10%/12.5%; Gemma jailbreak 19% vs. 6.5%) further show the numbers are not robust. A held-out evaluation with a second, independent safety metric would settle whether the method generalizes. Since this analysis supports the existing REJECT verdict, no verdict adjustment is needed.","tokens_in":12708,"tokens_out":6152,"duration_ms":54437,"concrete_test":"Split each of the three datasets (Vigil-jail-break, ChatGPT-jail-break, Prompt-Injection) into two disjoint folds. Run the Section 4.1 threshold search over tau in [0,1] in 0.01 steps on the selection fold only; freeze the optimal tau; run the full BART-Corrective pipeline on the held-out evaluation fold; recompute the mean toxicity and jail-breaking reductions and correction rates. If the held-out reductions are substantially smaller than Table 3, or if a fixed default threshold (e.g., 0.5) achieves comparable reductions, the headline gains are threshold overfitting rather than a property of the method. Also report bootstrap confidence intervals and use a second, independent safety evaluation (e.g., Perspective API or human ratings) to avoid same-metric circularity.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on Table 3, but the pipeline's triggering threshold tau* is chosen by Eq. 1 to maximize the mean reduction on exactly the test responses whose post-correction scores are then reported. Section 4.1.2 says the threshold variation is done 'on the same dataset we used to test the performance of the BART model in Section 4.3,' and Section 4.3.1 uses thresholds 'grounded by the ablation study.' This is a selection-on-test loop: with tau swept from 0 to 1 in steps of 0.01, the reported gains are the best of about 100 thresholds for this particular set of responses. They are not estimates of what a fixed system would do on new prompts. The evaluation is also self-referential: LangKit's cosine-similarity score is both the trigger for correction and the outcome measure, so the system is judged on the same metric it is gated and optimized over. In addition, the reported numbers are internally inconsistent: the abstract states 15%/21% for GPT-4 while the contributions section states 10%/12.5%, and Gemma's jailbreaking reduction is 19% in the abstract versus 6.5% in the contributions. These inconsistencies reinforce that the numerical claims are not stable enough to support the conclusion of significant safety improvement.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a post-generation correction mechanism, the BART-Corrective Model, that detects toxic or jail-breaking LLM outputs with LangKit's injections module and rewrites them using a BART model fine-tuned on Anthropic/hh-rlhf. The framework is designed to be API-compatible and model-agnostic. The authors report mean toxicity and jail-breaking reductions for GPT-4, PaLM2, Mistral-7B, and Gemma-2b-it, comparing their method with an approximated version of ProFS. The central claim is that a text-level safety filter can be bolted onto any LLM without modifying the model.","tokens_in":12965,"tokens_out":4447,"duration_ms":42422,"significance":"If the claims were supported, the contribution would be practically useful: a black-box-compatible, post-hoc safety filter that does not require parameter access would be an attractive deployment option. The paper has several strengths: the modular design is clearly described, the threshold-selection objective is stated explicitly in Eq. (1), and the authors include a comparison against ProFS. However, the evaluation protocol is the load-bearing weakness. The thresholds in Eq. (1) are selected on the same test responses whose post-correction scores are then reported in Table 3, making the headline reductions in-sample optima rather than predictive estimates. The evaluation is also self-referential because LangKit's cosine-similarity score is both the correction trigger and the outcome metric. The abstract and the contributions section report inconsistent numbers for GPT-4 and Gemma-2b-it. These issues directly undermine the paper's empirical claims, although they appear fixable with a properly split evaluation and an independent outcome measure.","major_comments":[{"comment":"The reported reductions are in-sample threshold optima. Section 4.1.2 states that the threshold variation is performed 'on the same dataset we used to test the performance of the BART model in Section 4.3,' and Eq. (1) chooses the threshold that maximizes the mean reduction on exactly those responses. Sweeping thresholds from 0.0 to 1.0 in steps of 0.01 means the gains in Table 3 are the best of roughly 100 candidates for that particular test set. They are therefore not estimates of what a fixed deployment would achieve on new prompts. The paper must use a train/validation/test split (select thresholds on validation, report only held-out test results) or otherwise report threshold-independent measures such as area under the gain-vs-threshold curve. Additionally, Eq. (1) does not encode the 'correction rate below a practical constraint' mentioned in the text, so the claimed 'optimal' threshold is not fully specified.","section":"§4.1.2, Eq. (1), §4.3.1"},{"comment":"The evaluation is self-referential: LangKit's injections module score, defined as the highest cosine similarity between the response embedding and known jailbreak examples, is used both to decide when to invoke the corrective model and as the jail-breaking outcome measure in Table 3. Thus the system is gated and optimized on exactly the same metric on which it is judged. Reductions in this score do not by themselves establish that outputs are safer or more secure under an independent definition of toxicity or jail-breaking. The authors should supplement the evaluation with independent toxicity classifiers, human annotation, or attack-success-rate measures.","section":"§3.2 and Table 3"},{"comment":"The numerical claims are internally inconsistent. The abstract reports a 15% and 21% reduction for GPT-4 and an 11.1% and 19% reduction for Gemma-2b-it, while the contributions section reports 10% and 12.5% for GPT-4 and 11.1% and 6.5% for Gemma-2b-it. The body text in Section 4.3.1 matches the abstract, not the contributions. This inconsistency suggests the reported numbers are not stable enough to support the paper's conclusion, and it must be resolved before the empirical claims can be assessed.","section":"Abstract vs. Contributions (Section 1)"}],"minor_comments":[{"comment":"The quality-preservation evaluation uses MRPC sentence pairs and a DeBERTa paraphrase classifier, but the detoxification process is applied at sentence level rather than to actual LLM responses. This is a weak proxy for the response-level semantic drift that matters in the proposed framework; consider reporting quality metrics on the actual evaluated responses.","section":"§4.2"},{"comment":"In the 'After the integration of the BART-Corrective model' block, the PaLM2 row lacks a '+ BART' label while the text in Section 4.3.1 reports PaLM2 reductions; the table should be labeled consistently so the reader can tell which rows are corrected and which are not.","section":"Table 3"},{"comment":"The text says the experiment uses three datasets, but Table 3 appears to aggregate all responses into a single mean/SD/min/max. Reporting per-dataset results would make the evaluation more transparent and would help assess cross-dataset variability.","section":"§4.3.1"},{"comment":"The 'data-centric version of ProFS' is a substantial approximation of the original method, because it extracts embeddings externally rather than modifying internal representations. The comparison should be framed as approximate, and the discussion of ProFS limitations should not be stated as if they apply to the original method without qualification.","section":"§4.3.2"},{"comment":"The claim that 'no data-centric approach has been proposed so far' is too strong given that the paper itself later describes a data-centric variant of ProFS and related data-filtering work exists; please qualify the claim.","section":"§1 and §2"},{"comment":"Please correct typographical errors and spacing issues (e.g., 'theAnthropic', 'conterparts', 'dentence', 'Suspaces' in the ProFS description).","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's core idea is reasonable and the writing is generally clear, but the evaluation methodology is not yet acceptable. The selection-on-test threshold optimization is a fixable flaw; re-running the evaluation with a proper split and an independent outcome measure would largely address it. I also recommend that the editor require the authors to reconcile the inconsistent headline numbers in the abstract and contributions section. The contribution is incremental relative to existing safety filters, but it may be suitable for a venue that values deployment-oriented, API-compatible safeguards. I would not recommend acceptance in the current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: this paper describes a plausible post-generation safety filter—BART fine-tuned on hh-rlhf, triggered by LangKit similarity scores—but its headline numbers are selected on the same test data used to report them. That makes the 15%/21% reductions in-sample maxima, not expected deployment gains. The paper's own Eq. 1 and Section 4.1.2 admit this. I'd treat the specific numbers as unsubstantiated, though the system itself is a reasonable engineering idea.\n\nWhat's new here is the integration: a text-level corrector that works with black-box LLM APIs, evaluated on four LLMs and three jailbreak datasets. The authors also attempt a ProFS baseline, though they approximate it since they lack access to internal embeddings. The MRPC-based semantic drift check (0.007 drop) is a nice touch. So there is a kernel of a useful applied contribution.\n\nThe soft spots are serious. First, threshold selection on the test set invalidates the headline reductions. To claim predictive improvement, thresholds need to be set on a validation split, with the test set held out. Second, the abstract and contributions give different numbers for the same models (GPT-4 15%/21% vs 10%/12.5%; Gemma jailbreak 19% vs 6.5%). A reader can't tell which is correct. Third, the trigger metric (LangKit cosine similarity) and the outcome metric are identical, so the system is being optimized and judged on the same score; that doesn't show real-world safety, just sensitivity to that particular embedding similarity. Fourth, no code, no statistical tests, and the ProFS baseline is a rough approximation, not the real thing.\n\nIf the authors redo the evaluation with proper train/validation/test separation and reconcile the numbers, the paper could become a solid applied contribution. As it stands, the central evidence doesn't support the claim of significant safety improvement. I'd recommend sending it for review only if the editor expects major revision, not as-is.\n\nWho's this for? People building safety tooling for API-based LLMs might find the pipeline idea worth a look, but they should not rely on the reported effect sizes.\n\nMy recommendation: if you're the editor, send to reviewers with the threshold issue flagged, or reject if the journal doesn't want heavy revision.","headline":"Plausible post-generation BART filter, but the headline reductions are in-sample threshold optima, not predictions; the evaluation needs a validation split and reconciled numbers.","tokens_in":13477,"tokens_out":2679,"would_cite":false,"duration_ms":23607,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a fine-tuned BART model, applied after generation as an external rewrite step, lowers mean toxicity and jail-breaking scores across four LLM APIs, including a 15% toxicity reduction and 21% jail-breaking reduction…","keywords":["large language models","toxicity mitigation","jailbreak defense","post-generation correction","BART fine-tuning","data-centric AI","semantic similarity scoring","safety filter"],"falsifier":"Run the corrected and uncorrected outputs through an independently trained toxicity classifier or a panel of human raters; if those independent judgments show little or no drop in harmful content while the injection-similarity score drops sharply, then the reported reductions are an artifact of the chosen scoring metric rather than genuine safety improvement.","tokens_in":12516,"feed_emoji":"🛡️","tokens_out":7706,"duration_ms":64159,"temperature":0.7,"pith_summary":"This paper is trying to show that LLM safety does not have to be built into the model. Its proposal is a post-generation correction step: a fine-tuned BART model that detects toxic or jail-broken responses and rewrites them into safe alternatives before the user sees them. The authors report that this bolt-on filter reduces mean toxicity and jail-breaking scores across four very different LLMs, with the largest effects being a 28% toxicity cut for PaLM2 and a roughly 26% toxicity cut for Mistral-7B. A sympathetic reader would care because the approach works at the text level, so it can be wrapped around any LLM API without retraining or accessing internal weights.","feed_headline":"A text-level safety filter cuts LLM toxicity and jailbreaks","feed_subtitle":"A fine-tuned BART model rewrites dangerous replies after generation, working with black-box LLM APIs.","key_machinery":"The load-bearing object is the BART-Corrective Model: a BART sequence-to-sequence transformer fine-tuned so that, given a rejected (toxic) response, it predicts the corresponding chosen (non-toxic) response. It is embedded in a two-stage pipeline: generation, then a safety check using a semantic-similarity score computed as the highest cosine similarity between the response embedding and known jailbreak examples; when the score exceeds a threshold, the corrective model rewrites the response. The threshold is not fixed but chosen per model by maximizing the average toxicity and jail-breaking reduction while keeping the correction rate within limits, using the formula $$\\tau^* = \\arg\\max_\\tau \\frac{\\sum_{i=1}^N ($s_i^{{\\mathrm{LLM}}$} - $s_i^{{\\mathrm{BART}}$}) \\cdot \\mathbb{I}($s_i^{{\\mathrm{LLM}}$} > \\tau)}{\\sum_{i=1}^N \\mathbb{I}($s_i^{{\\mathrm{LLM}}$} > \\tau)}.$$ This mechanism is what makes the approach data-centric: it changes the output text rather than the model's parameters or embeddings.","core_discovery":"The central discovery is that a sequence-to-sequence BART model, fine-tuned on pairs of rejected (toxic) and chosen (non-toxic) assistant responses, can serve as a general-purpose safety corrector. In the proposed pipeline, an LLM generates a response, a semantic-similarity scorer compares that response against known jailbreak and injection examples, and if the score exceeds a tuned threshold the BART model rewrites the response. The paper reports that this correction reduces mean toxicity and jail-breaking scores for GPT-4 (15% and 21%), PaLM2 (28% and 5%), Mistral-7B (26% and 23%), and Gemma-2b-it (11.1% and 19%), while a paraphrase-detection benchmark shows only a small semantic drift (an accuracy drop of 0.007). Compared with a projection-filter baseline that edits embeddings, the BART approach is claimed to be more adaptable to API-based LLMs because it only touches generated text.","pith_inferences":["If the similarity-based safety score is an imperfect proxy, the same pipeline could be driven by an ensemble of detectors, such as classifier-based toxicity models, human feedback, or model self-assessment, and still gain the text-level correction benefit; the paper does not test this.","The threshold-optimization formula could be run continuously in production, letting the filter adapt as jailbreak tactics evolve; the paper presents it as a static per-model choice.","The approach may transfer to other safety properties, such as hallucination or privacy leakage, wherever a rewrite model can be trained on bad-versus-good output pairs; the paper only tests toxicity and jailbreaking."],"forward_implications":["Any LLM accessible through an API, including proprietary black-box systems, can be made safer by wrapping it with this filter, with no fine-tuning or parameter access.","The same corrected BART model can serve multiple target LLMs, since it operates on text rather than on model internals.","Deployers can tune the correction threshold per model, trading off how often outputs are rewritten against how much toxicity reduction is achieved.","Correction preserves task semantics closely enough that paraphrase-classification accuracy drops by only 0.007, so the filter can be inserted into production pipelines without visibly degrading response quality.","The framework is designed so it can be extended in future work to retrieval-augmented-generation pipelines and additional baseline models."],"supporting_citations":[{"why":"Supplies the ProFS baseline, a model-editing method that reduces toxicity by projecting embeddings into a non-toxic subspace; the paper compares its toxicity and jail-breaking scores directly against the BART-Corrective Model.","marker":"[24]"},{"why":"Defines GPT-4, one of the four API-based LLMs whose outputs are corrected and scored, and for which a 15% toxicity and 21% jail-breaking reduction is reported.","marker":"[25]"},{"why":"Defines PaLM2, the target LLM with the largest reported mean toxicity reduction of 28% after BART correction.","marker":"[26]"},{"why":"Defines Mistral-7B, an open-weights LLM whose outputs are corrected, showing roughly 26% toxicity and 23% jail-breaking reductions.","marker":"[27]"},{"why":"Defines Gemma-2b-it, the fourth target LLM, with reported reductions of 11.1% in toxicity and 19% in jail-breaking score.","marker":"[28]"}],"fun_headline_variants":["BART rewrite model trims LLM toxicity and jailbreaks","Post-generation BART corrector cuts toxic LLM replies","Safety filter rewrites LLM outputs to reduce toxicity","Text-level corrector lowers LLM toxicity and jailbreaks","Fine-tuned BART cleans up harmful LLM responses"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central assumption is that the similarity score a response shows to known jailbreak examples correctly measures whether that response is toxic or unsafe, both when deciding to correct it and when judging whether the correction worked.","fun_headline_variants_meta":{"raw":{"variants":["BART rewrite model trims LLM toxicity and jailbreaks","Post-generation BART corrector cuts toxic LLM replies","Safety filter rewrites LLM outputs to reduce toxicity","Text-level corrector lowers LLM toxicity and jailbreaks","Fine-tuned BART cleans up harmful LLM responses"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000218,"raw_usage":{"total_tokens":1446,"prompt_tokens":955,"completion_tokens":491,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":571,"completion_tokens_details":{"reasoning_tokens":410}},"tokens_in":571,"tokens_out":491,"duration_ms":4942,"temperature":1.0,"reasoning_tokens":410,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:53:53.486819+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the corrected and uncorrected outputs through an independently trained toxicity classifier or a panel of human raters; if those independent judgments show little or no drop in harmful content while the injection-similarity score drops sharply, then the reported reductions are an artifact of the chosen scoring metric rather than genuine safety improvement.","supporting_citations":[{"cited_title":"Model editing as a robust and denoised variant of dpo: A case study on toxicity","cited_arxiv_id":null,"evidence_quote":"Supplies the ProFS baseline, a model-editing method that reduces toxicity by projecting embeddings into a non-toxic subspace; the paper compares its toxicity and jail-breaking scores directly against the BART-Corrective Model."},{"cited_title":"Gpt-4 technical report, 2024","cited_arxiv_id":null,"evidence_quote":"Defines GPT-4, one of the four API-based LLMs whose outputs are corrected and scored, and for which a 15% toxicity and 21% jail-breaking reduction is reported."},{"cited_title":"Dai, and Orhan Firat et al","cited_arxiv_id":null,"evidence_quote":"Defines PaLM2, the target LLM with the largest reported mean toxicity reduction of 28% after BART correction."}],"review_version":1}