IntentionReasoner: Facilitating Adaptive LLM Safeguards through Intent Reasoning and Selective Query Refinement

Guanxu Chen; Ruicheng Yin; Xiaoqing Zheng; Xuanjing Huang; Yide Liu; Yuanzhe Shen; Zhengkang Guo; Zisu Huang

REVIEW 3 major objections 4 minor 13 references

IntentionReasoner: Facilitating Adaptive LLM Safeguards through Intent Reasoning and Selective Query Refinement

T0 review · 3 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read IntentionReasoner claims a guard model can block harmful prompts, rewrite borderline ones, and nearly eliminate over-refusal.

desk verdict Well-engineered guard model with a genuinely new idea, but the near-perfect safety numbers are inflated by training on four of the six evaluation benchmarks; the WildGuard circularity is real but secondary. read the letter →

arxiv 2508.20151 v1 pith:ZBIE6HZN submitted 2025-08-27 cs.AI

Yuanzhe Shen , Zisu Huang , Zhengkang Guo , Yide Liu , Guanxu Chen , Ruicheng Yin , Xiaoqing Zheng , Xuanjing Huang This is my paper

classification cs.AI

keywords intentreasoningguardmodelLLMsafetyqueryrewritingover-refusaljailbreakdefensemulti-levelclassificationreinforcementlearning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

IntentionReasoner is a guard model that sits in front of a large language model and decides what to do with each user query: refuse it, pass it through, or rewrite it first. The paper's central claim is that making that decision through explicit intent reasoning and a four-level safety label—rather than a binary safe/unsafe choice—lets a small model (1.5B–7B parameters) block harmful and jailbreak prompts while almost never refusing harmless ones. On six harmfulness benchmarks, the 7B version reaches 99.4 average F1 and 1.2% average attack success rate, with a 0.0% over-refusal rate. If this works in deployment, LLM services could add strong, cheap, task-agnostic safeguards without retraining the base model or paying the usual safety-alignment tax in utility.

What carries the argument

The key mechanism is the guard's structured reasoning-and-rewrite output: a <thinking> block analyzing both good and harmful intent, a <label> chosen from four ordered classes (Completely Unharmful, Borderline Unharmful, Borderline Harmful, Completely Harmful), and a <refined query> that is mandatory for borderline-harmful queries and optional for harmless ones. The four-level taxonomy is what separates 'should refuse' from 'should rewrite,' and the selective rewriting is what removes over-refusal. The RL stage then tunes this behavior with a sum of five rewards, including a safety score for rewritten queries supplied by a separate guard model, a utility score from a preference reward model,

What would settle it

Have independent human annotators (or a second, independently trained judge) score 1,000 rewritten borderline queries from the benchmark set and check whether the rewritten prompts can still make a target LLM produce harmful content. If the independently confirmed harmful rate is substantially above the reported 1.2%, the safety claim is an artifact of using the same judge for training and evaluation.

Watch

Extended reading notes

Core claim

The paper claims that a single lightweight guard can replace the usual binary 'safe/unsafe' gate with a three-way action: completely harmful queries are rejected, borderline harmful queries are rewritten to remove latent harmful intent while keeping the benign core, and harmless queries may be refined to elicit better answers. Training data is built by merging and reannotating about 163,000 queries from red-teaming and instruction datasets, then cold-starting with supervised fine-tuning and refining with online reinforcement learning under a composite reward covering format, label accuracy, rewrite safety, rewrite utility, and length. The result is consistently the highest F1 across six dete

Load-bearing premise

The safety gain rests on trusting one external guard model's judgments about whether a rewritten query is safe; that same judge provides the RL safety reward, the label reannotation, and the evaluation metric, so if it wrongly calls an unsafe rewrite 'safe,' the reported near-zero attack rate is not real safety.

Editorial extensions

If this is right

A 1.5B version already beats 8B reasoning guards on the six benchmarks, so safety filtering can run at a small fraction of the cost of the protected LLM.
Restricting rewriting to borderline labels (BH only) keeps attack success rate unchanged while cutting average output tokens by up to 37%, giving an efficiency knob for high-volume services.
Because the guard is an input preprocessor, it can be dropped in front of any existing LLM, including black-box APIs, without modifying the base model.
For smaller target models, rewriting benign queries measurably improves answer quality; for much stronger models the gains vanish, so the paper recommends disabling harmless-query refinement there.
The method scales with model size: F1, ASR, and ORR all improve from 1.5B to 3B to 7B, suggesting further gains from larger guard models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The reported safety depends on the specific guard model used to score rewriting safety during RL and to measure ASR at evaluation. If that judge has blind spots, an adversary could craft queries whose 'safe' rewrites still elicit harmful content; an independent human red-team test would be the natural check.
The same intent-reason-and-rewrite loop could be applied to LLM outputs, not just inputs, to scrub unsafe generated text on the fly—an extension the paper does not test.
Because the utility reward uses a general preference model and the training data come mostly from English red-teaming datasets, the rewriting behavior may be tuned to that judge and that distribution; multilingual or domain-specific deployment would need its own validation.
The BH-only variant's token savings suggest a likely production default: keep rewriting on only for borderline cases and pass clearly benign queries through unchanged.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is one of the more carefully built guard-model papers I've seen in terms of pipeline design, but the headline numbers are not believable as stated because the authors trained on the very benchmarks they evaluate on. The four-level taxonomy and the selective query-rewriting step are a real advance over the binary reasoning guards (GuardReasoner, RSafe), and the RL training with multiple rewards is thoughtful. The jailbreak experiments are also much broader than what most guard-model papers report. Credit where it's due: the idea is worth taking seriously.

The load-bearing flaw is the evaluation setup. Section 2.2 trains on the training splits of OR-Bench, WildGuard, ToxicChat, and WildJailbreak, and Section 3.1 evaluates on those same datasets. Four of six headline benchmarks are in-domain for IntentionReasoner but out-of-domain for every baseline, which fully explains the near-perfect F1 (99.8 on OR-Bench, 99.3 on WildJailbreak) and the near-zero ASR. The stress-test note is right that the WildGuard circularity is secondary: the ASR definition already counts any harmful query not classified as 'completely harmful' as an attack, so the low ASR is mostly driven by in-domain label classification, not by the WildGuard rewrite-safety judgment. The circularity matters most for the rewriting-safety claims and for the RL reward, where the model is literally optimized against the same judge that later scores its rewrites. That's a valid concern, just not the main one.

There's also a quieter issue: the ground-truth labels for the four-level taxonomy come from DeepSeek-V3 plus two guard models, not from humans. So the borderline labels are synthetic, and the detection F1 is computed against those synthetic labels. The case studies are illustrative but don't validate the taxonomy's reliability.

The good news is the method might still work: the out-of-domain results (XSTest, OpenAI Moderation) are also strong, and the jailbreak ASR numbers, while using LLaMA Guard 3 as judge, show a real effect. But as submitted, the central claim of 99+ F1 and 1% ASR is unsupported. The authors need to release code and data, cut or re-run the in-domain benchmarks, use an independent safety judge for rewrites, and disclose the synthetic-label dependence. This deserves a serious referee, but the revision bar is high.

Referee Report

3 major / 4 minor

Summary. The paper proposes IntentionReasoner, a guard model that classifies user queries into four safety levels (Completely Unharmful, Borderline Unharmful, Borderline Harmful, Completely Harmful) with explicit intent reasoning, and optionally rewrites queries to reduce harm or improve quality. Training is in two stages: SFT on a constructed ~163k-sample dataset of queries with reasoning traces, labels, and rewrites, followed by GRPO with a composite reward covering format, label accuracy, rewriting safety, rewriting utility, and length. Experiments on six harmfulness benchmarks, five jailbreak methods, and two generation-quality benchmarks report near-ceiling F1 (up to 99.4), very low ASR (down to 1.2%), near-zero over-refusal, and improved query quality. The main claims are in §3.4 and Tables 2–4.

Significance. The idea of grading queries on a four-level scale and rewriting borderline cases while refusing only clearly harmful queries is a genuinely useful direction for balancing safety and over-refusal. The dataset construction, the multi-reward RL recipe, and the SFT-vs-RL ablation are valuable engineering contributions. However, the evaluation as presented cannot separate generalization from memorization: four of the six headline benchmarks are in-domain for IntentionReasoner, and the rewriting-safety metric uses WildGuard, which is also in the training loop. If the authors can produce a clean held-out evaluation, the contribution would be significant; in the current form the headline numbers are not established.

major comments (3)

[§2.2, §3.1, Table 7] The SFT training data include the training splits of OR-Bench (39,330 queries), WildGuard (17,950), ToxicChat (3,997), and WildJailbreak (75,999), while §3.1 evaluates on OR-Bench, WildGuardTest, ToxicChat, and WildJailbreakEval. Four of the six reported benchmarks therefore overlap with the training distribution. The near-ceiling F1 numbers in Table 2 (e.g., 99.7 on OR-Bench, 99.3 on WildJailbreak) may reflect distribution overlap rather than generalizable safety reasoning. The central claim in §3.4 that the method 'consistently achieves the highest F1 scores across all benchmarks' requires either retraining without these eval splits or an additional held-out evaluation on disjoint benchmarks.
[§2.2, §2.3(III), §3.1] WildGuard is used in three connected places: (a) to validate safety labels during reannotation, (b) as the rewriting-safety reward Rs in the RL objective, and (c) as the judge for whether a rewritten query is unsafe in the ASR computation. Thus the low ASR on rewritten queries measures agreement with a model that was itself a training signal. A model rewarded for satisfying WildGuard will naturally receive low WildGuard-based ASR, even if the rewrites are not safe by independent standards. The authors should evaluate rewriting safety with a held-out judge, human annotation, or at least a different safety model not used in training.
[§3.1, §2.1] The ASR definition is internally inconsistent with the method's design. For IntentionReasoner, ASR is defined as a harmful query being rewritten into an unsafe form or 'not classified as completely harmful (failure to refuse)'. But the method deliberately rewrites Borderline Harmful queries instead of refusing them (§2.1). Treating any BH prediction on a harmful query as an attack conflates the intended mitigation with a failed refusal. Please report ASR separately for CH and BH original queries, and clarify how rewritten BH queries are scored.

minor comments (4)

[Figures 6 and 8] The label 'boarderlineharmful' appears to be a typo for 'borderlineharmful'.
[Table 3, §3.2] Table 3 lists 'Aegis2.0 7B', but §3.2 and Table 2 describe Aegis2.0 as 8B. Please correct the size label.
[§3.1 and Appendix B.1] The dataset-specific ASR/ORR tables combine ASR and ORR in a single column pair. Since ASR and ORR are defined differently, the reader would benefit from separate analysis of harmful and benign subsets, especially for the borderline labels.
[§2.2 and §3.3] Exact data splits for XSTest and OpenAI Moderation and the versioning of the evaluation sets are not specified. Please state them for reproducibility.

Circularity Check

1 steps flagged · score 6.0 of 10

Benchmark overlap between training and evaluation on OR-Bench/ToxicChat inflates headline F1; the WildGuard ASR circularity is mostly redundant.

fitted input called prediction [Section 2.2 (Training Data Construction) and Section 3.1 (Benchmarks and Evaluation Metrics), with Table 7]
"We select all queries and available labels from the training sets of six widely used red-teaming datasets, including ALERT, BeaverTails, OR-Bench, ToxicChat, WildGuard, and WildJailbreak... After this process, we obtain approximately 163K unique queries. ... we evaluate on six benchmarks: ToxicChat (Lin et al. 2023), OpenAI Moderation (Markov et al. 2023), WildGuardTest (Han et al. 2024), OR-Bench (Cui et al. 2024), WildJailbreakEval (Jiang et al. 2024), and XSTest (Rottger et al. 2024)."

OR-Bench and ToxicChat are used both as training sources (39,330 and 3,997 selected queries, respectively, per Table 7) and as evaluation benchmarks, with no explicit held-out split stated in the paper (unlike WildGuardTest and WildJailbreakEval). For any query appearing in both the training pool and the evaluation set, the reported F1 is a measure of training-label memorization rather than independent generalization. These two benchmarks feed the per-benchmark F1 (98.7 on ToxicChat, 99.2 on OR-Bench for IR 7B) and the headline average F1 of 99.4, so a portion of the central claim is forced by construction.

full rationale

The paper is not self-citation-driven and does not invoke uniqueness theorems or imported ansatze; the core training/evaluation pipeline is transparent. The one substantive circularity is that OR-Bench and ToxicChat appear both in the training pool and as evaluation benchmarks, with no stated exclusion of the training queries from the evaluation sets. This makes the per-benchmark F1 on those sets, and hence part of the average 99.4 F1, a fitted-input prediction rather than an independent measurement. I considered the WildGuard-as-judge concern raised in the reader's take: the ASR definition in Section 3.1 already counts any harmful query not classified 'completely harmful' as an attack, so the clause 'rewritten into an unsafe form (evaluated by WildGuard)' is logically redundant for the reported ASR; low ASR is therefore driven by label-classification recall on harmful queries, not by WildGuard. Consequently, the WildGuard reward loop does not create a circular metric, though it does weaken the interpretability of 'rewriting safety.' The remaining benchmarks (WildGuardTest, XSTest, OpenAI Moderation, jailbreak ASR with LlamaGuard3) are held-out or externally judged, so the central claim has substantial independent content. The score is set at 6 because one or more headline predictions partially reduce to training-set memorization, but not higher because the main result does not collapse entirely.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on two assumptions: model-generated labels are trustworthy, and WildGuard safety judgments are valid. The paper provides no human verification beyond guard-model agreement. The hand-set length reward parameters are less central but are additional free choices.

free parameters (5)

Length reward tolerance r+ = 2.0
Hand-set upper tolerance ratio in Eq. (8); determines how much longer a rewritten query may be than the original.
Length reward tolerance r- = 0.5
Hand-set lower tolerance ratio in Eq. (8).
Length bounds L- and L+ = 20 and 200 tokens
Hand-set clip bounds in Eq. (8).
Length reward mix weight lambda = 0.8
Weight balancing query length reward and response length reward in Eq. (11).
Adjacent label partial credit = 0.5
Partial reward in Rc for adjacent label confusions (CU/BU, BU/BH, BH/CH).

assumptions (4)

domain assumption WildGuard safety judgments are a valid proxy for actual harmfulness.
Used as the Rs reward (Section 2.3 III), as the ASR evaluator for rewritten queries (Section 3.1), and in label reannotation (Section 2.2). No human verification is reported.
domain assumption DeepSeek-V3 generated reasoning traces, labels and rewrites are reliable enough to serve as supervision.
The 163K training set is synthesized by DeepSeek-V3 with quality control via guard-model agreement only (Section 2.2, Appendix A.1).
domain assumption The four-level taxonomy has meaningful boundaries that separate harmless from harmful intent.
Table 1 defines CU/BU/BH/CH but no inter-annotator agreement or behavioral validation of the boundary is provided.
domain assumption Evaluation benchmarks OR-Bench and WildJailbreak are disjoint from the training samples selected from the same datasets.
Section 2.2 selects training queries from these datasets and Table 2 reports scores on the same benchmark names without documented split provenance.

how reviews work

0 comments

Cite this review

Pith. "Pith review of IntentionReasoner: Facilitating Adaptive LLM Safeguards through Intent Reasoning and Selective Query Refinement." pith.science (2026). https://pith.science/paper/ZBIE6HZN

@misc{pith2026250820151,
  author       = {Pith},
  title        = {Pith review of: IntentionReasoner: Facilitating Adaptive LLM Safeguards through Intent Reasoning and Selective Query Refinement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZBIE6HZN}},
  note         = {Machine review of arXiv:2508.20151}
}

read the original abstract

The rapid advancement of large language models (LLMs) has driven their adoption across diverse domains, yet their ability to generate harmful content poses significant safety challenges. While extensive research has focused on mitigating harmful outputs, such efforts often come at the cost of excessively rejecting harmless prompts. Striking a balance among safety, over-refusal, and utility remains a critical challenge. In this work, we introduce IntentionReasoner, a novel safeguard mechanism that leverages a dedicated guard model to perform intent reasoning, multi-level safety classification, and query rewriting to neutralize potentially harmful intent in edge-case queries. Specifically, we first construct a comprehensive dataset comprising approximately 163,000 queries, each annotated with intent reasoning, safety labels, and rewritten versions. Supervised fine-tuning is then applied to equip the guard model with foundational capabilities in format adherence, intent analysis, and safe rewriting. Finally, we apply a tailored multi-reward optimization strategy that integrates rule-based heuristics and reward model signals within a reinforcement learning framework to further enhance performance. Extensive experiments show that IntentionReasoner excels in multiple safeguard benchmarks, generation quality evaluations, and jailbreak attack scenarios, significantly enhancing safety while effectively reducing over-refusal rates and improving the quality of responses.

Figures

Figures reproduced from arXiv: 2508.20151 by the authors.

**Figure 2.** Overview of the IntentionReasoner Training Pipeline. The pipeline comprises two stages: data construction with SFT [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Effectiveness of Our Length-Control Design. [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: An example classified as completely unharmful. [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]

**Figure 5.** Figure 5: An example classified as borderline unharmful. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

**Figure 6.** Figure 6: An example classified as borderline harmful. [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]

**Figure 7.** Figure 7: An example classified as completely harmful. [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

**Figure 8.** Figure 8: An example of a jailbreak attack (ReNeLLM) classified as borderline harmful. [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]

**Figure 9.** Figure 9: An example of a jailbreak attack (GCG) classified as completely harmful. [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

13 extracted references · 6 canonical work pages

[1]

Intent Analysis - Thoroughly evaluate both positive intentions and potential harmful or misleading aspects of the query

work page
[2]

Harm Classification - Determine the overall safety level of the query based on its possible uses and interpretations

work page
[3]

arXiv preprint arXiv:2412.16339

Deliberative alignment: Reasoning enables safer lan- guage models. arXiv preprint arXiv:2412.16339. Han, S.; Rao, K.; Ettinger, A.; Jiang, L.; Lin, B. Y .; Lambert, N.; Choi, Y .; and Dziri, N. 2024. Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of llms. Advances in Neural Information Processing Systems , 37: 8093–81...

arXiv 2024
[5]

Advances in Neural Information Processing Systems, 37: 47094–47165

Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models. Advances in Neural Information Processing Systems, 37: 47094–47165. Kang, M.; and Li, B. 2024. R2-Guard: Robust Reasoning Enabled LLM Guardrail via Knowledge-Enhanced Logical Reasoning. arXiv preprint arXiv:2407.05557. Li, X.; Yu, P.; Zhou, C.; Schick, T.; Levy, O....

arXiv 2024
[6]

In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, 1671–1685

” do anything now”: Characterizing and evaluating in-the-wild jailbreak prompts on large language models. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, 1671–1685. Tan, F.; Hu, Y .; Yen, K.; and Hu, C. 2021. BERT-Beta: A Proactive Probabilistic Approach to Text Moderation. In Proceedings of the 2021 Conference...

arXiv 2024
[7]

In Proceedings of the 62nd Annual Meet- ing of the Association for Computational Linguistics (Vol- ume 3: System Demonstrations), 400–410

LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models. In Proceedings of the 62nd Annual Meet- ing of the Association for Computational Linguistics (Vol- ume 3: System Demonstrations), 400–410. Zhou, C.; Liu, P.; Xu, P.; Iyer, S.; Sun, J.; Mao, Y .; Ma, X.; Efrat, A.; Yu, P.; Yu, L.; et al. 2023. Lima: Less is more for alignment. Advances in...

arXiv 2023
[10]

No potential good intention

Query Refinement - Rewrite the query to reduce or eliminate potential risks while enhancing clarity and safety. Good queries can also be made better. Use the following structured format in your response: <thinking> Good intention: [Describe the positive or neutral purpose of the query. If none exist, state "No potential good intention".] Harmful intention...

work page
[11]

**Safety & Risk Mitigation ** - Does the rewrite effectively remove, neutralize, or rephrase any harmful, unsafe, or illegal intent from the original query, ensuring it cannot lead to harmful or unethical outputs?

work page

Show all 13 references

[12]

**Relevance** - For the benign parts of the query, does the rewrite preserve the original intent without adding misleading or incorrect content?
[13]

Rating: [[5]]

**Usefulness** - Does the rewrite make the query clearer, more specific, and more likely to elicit a high-quality, safe, and ethical response? Start your evaluation with a brief, objective explanation. After the explanation, give a rating on a scale of 1 to 10 using this exact...
[42]

Chen, J.; Xiao, S.; Zhang, P.; Luo, K.; Lian, D.; and Liu, Z

IEEE. Chen, J.; Xiao, S.; Zhang, P.; Luo, K.; Lian, D.; and Liu, Z. 2024. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. arXiv preprint arXiv:2402.03216. Cheng, J.; Liu, X.; Zheng, K.; Ke, P.; Wang, ...

2024 arXiv
[2023]

arXiv preprint arXiv:2312.06674

Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674. Ji, J.; Liu, M.; Dai, J.; Pan, X.; Zhang, C.; Bian, C.; Chen, B.; Sun, R.; Wang, Y .; and Yang, Y . 2023. Beavertails: Towards improved safety alignment of llm via a huma...

2023 arXiv
[2024]

arXiv preprint arXiv:2404.04475

Length-controlled alpacaeval: A simple way to debias automatic evaluators. arXiv preprint arXiv:2404.04475. Feng, K.; Ding, K.; Yu, J.; Li, M.; Wang, Y .; Xu, T.; Wang, X.; Zhang, Q.; and Chen, H. 2025. ERPO: Advancing Safety Alignment via Ex-Ante Reasoning Preference Optimiza...

2025 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

Intent Analysis - Thoroughly evaluate both positive intentions and potential harmful or misleading aspects of the query

work page

[2] [2]

Harm Classification - Determine the overall safety level of the query based on its possible uses and interpretations

work page

[3] [3]

arXiv preprint arXiv:2412.16339

Deliberative alignment: Reasoning enables safer lan- guage models. arXiv preprint arXiv:2412.16339. Han, S.; Rao, K.; Ettinger, A.; Jiang, L.; Lin, B. Y .; Lambert, N.; Choi, Y .; and Dziri, N. 2024. Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of llms. Advances in Neural Information Processing Systems , 37: 8093–81...

arXiv 2024

[4] [5]

Advances in Neural Information Processing Systems, 37: 47094–47165

Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models. Advances in Neural Information Processing Systems, 37: 47094–47165. Kang, M.; and Li, B. 2024. R2-Guard: Robust Reasoning Enabled LLM Guardrail via Knowledge-Enhanced Logical Reasoning. arXiv preprint arXiv:2407.05557. Li, X.; Yu, P.; Zhou, C.; Schick, T.; Levy, O....

arXiv 2024

[5] [6]

In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, 1671–1685

” do anything now”: Characterizing and evaluating in-the-wild jailbreak prompts on large language models. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, 1671–1685. Tan, F.; Hu, Y .; Yen, K.; and Hu, C. 2021. BERT-Beta: A Proactive Probabilistic Approach to Text Moderation. In Proceedings of the 2021 Conference...

arXiv 2024

[6] [7]

In Proceedings of the 62nd Annual Meet- ing of the Association for Computational Linguistics (Vol- ume 3: System Demonstrations), 400–410

LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models. In Proceedings of the 62nd Annual Meet- ing of the Association for Computational Linguistics (Vol- ume 3: System Demonstrations), 400–410. Zhou, C.; Liu, P.; Xu, P.; Iyer, S.; Sun, J.; Mao, Y .; Ma, X.; Efrat, A.; Yu, P.; Yu, L.; et al. 2023. Lima: Less is more for alignment. Advances in...

arXiv 2023

[7] [10]

No potential good intention

Query Refinement - Rewrite the query to reduce or eliminate potential risks while enhancing clarity and safety. Good queries can also be made better. Use the following structured format in your response: <thinking> Good intention: [Describe the positive or neutral purpose of the query. If none exist, state "No potential good intention".] Harmful intention...

work page

[8] [11]

**Safety & Risk Mitigation ** - Does the rewrite effectively remove, neutralize, or rephrase any harmful, unsafe, or illegal intent from the original query, ensuring it cannot lead to harmful or unethical outputs?

work page