Refining Input Guardrails: Enhancing LLM-as-a-Judge Efficiency Through Chain-of-Thought Fine-Tuning and Alignment

Andy Luo; Huy Nghiem; Melissa Kazemi Rad; Mohammad Sorower; Sahil Wadhwa; Stephen Rawls

REVIEW 4 major objections 7 minor 2 cited by

Refining Input Guardrails: Enhancing LLM-as-a-Judge Efficiency Through Chain-of-Thought Fine-Tuning and Alignment

T0 review · 4 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper shows that fine-tuning and aligning chain-of-thought responses on a 400-example curated dataset turns open-weight LLMs into accurate input-moderation guardrails, with the best configuration—a DPO-aligned Llama3 8B—reaching 96.1…

desk verdict A practical, honest empirical comparison of small-data CoT fine-tuning plus alignment for LLM input guardrails; the main weakness is the in-distribution evaluation, but the core result is credible and worth referee time. read the letter →

arxiv 2501.13080 v1 pith:K4V4TQUP submitted 2025-01-22 cs.CL cs.CRcs.LG

Melissa Kazemi Rad , Huy Nghiem , Andy Luo , Sahil Wadhwa , Mohammad Sorower , Stephen Rawls This is my paper · ORCID

classification cs.CLcs.CRcs.LG

keywords inputguardrailsLLM-as-a-Judgechain-of-thoughtsupervisedfine-tuningDPOKTOjailbreakdetectionadversarialrobustness

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The authors ask whether a very small, carefully curated dataset can turn an off-the-shelf open-weight LLM into a reliable input guardrail—the component that flags malicious, fraudulent, or jailbreaking user queries before they reach a conversational agent. They show that supervised fine-tuning on just 400 examples, followed by preference optimization (DPO or KTO) of the model's chain-of-thought explanations, dramatically lifts detection from poor baselines to high accuracy while also making outputs short and parseable. The best result, a DPO-aligned Llama3 8B model, achieves 96.1 F1 and a 0.8% false-positive rate on a held-out test set of about 6,800 queries, outperforming the published LlamaGuard-2 and DeBERTaV3 models. The practical upshot is that a production-grade safety classifier can be built with a few hundred annotated examples and a few hours of training on commodity hardware.

What carries the argument

The central mechanism is the chain-of-thought-aligned fine-tuning pipeline: supervised fine-tuning with low-rank adapters (LoRA) to teach the model the verdict categories, followed by preference optimization (DPO or KTO) to shape the explanatory reasoning toward concise, policy-relevant rationales. A '#END' trigger token appended to training examples suppresses repetitive continuations. The key object is the aligned CoT explanation itself—the paper argues that the quality of the reasoning preceding the verdict drives both detection accuracy and format compliance, and demonstrates that the tuned models both follow the requested output structure and identify adversarial prompts that confuse the base models.

What would settle it

Take the DPO-aligned Llama3 8B model and evaluate it on a disjoint collection of adversarial prompts from outside the benchmark families used here—for example, recently collected jailbreaks from web communities or a benchmark like HarmBench. If its F1 drops substantially below the reported 96.1 or its false-positive rate rises sharply, the claim that the 400-example recipe generalizes across 'diverse adversarial and malicious query types' is falsified.

Watch

Extended reading notes

Core claim

The central claim is that aligning the reasoning, not just the verdict, is what makes a small fine-tuned guardrail reliable. The authors demonstrate that a 400-example training set—200 malicious/jailbreak queries and 200 safe ones—with manually corrected chain-of-thought explanations, is enough to raise four open-weight LLMs (Llama3 8B, Llama2 13B, Mistral 7B, Mixtral 8x7B) to high F1 scores, and that adding DPO or KTO on top of supervised fine-tuning yields further small gains. On a held-out test set of roughly 6,800 queries, the DPO-aligned Llama3 8B reaches 96.1 F1, 93.3 attack detection ratio (the recall for malicious inputs), and 0.8% false-positive rate. This outperforms LlamaGuard-2 (69.2 F1, 54.2 ADR, 2.2% FPR) and the DeBERTaV3 injection detector (81.4 F1), while PromptGuard, despite a 98.2 ADR, is unusable because it flags 99.8% of safe inputs. The authors interpret these results as evidence that with careful data curation, even constrained resources can produce a capable proxy defense against malicious and jailbreak queries.

Load-bearing premise

The claim rests on the assumption that the held-out test set, which draws from the same public attack benchmarks that supplied the training set, represents the real-world distribution of malicious and jailbreak queries a production guardrail would face.

Editorial extensions

If this is right

A few hundred curated examples suffice to specialize an open-weight LLM as an input guardrail, reducing the annotation and compute burden compared to full RLHF-based safety training.
Tuning dramatically improves format compliance—invalid response ratio drops from 16.8% to 0.3% for Llama3 8B—so guardrail outputs can be reliably parsed by downstream agents.
Preference optimization on top of SFT gives only marginal gains in this low-data regime; the authors speculate that DPO/KTO would need larger or more diverse rejected-response sets to unlock further improvement.
Smaller tuned models (8B) can beat both larger tuned models (13B and 8x7B) and dedicated guardrail classifiers, making them an attractive cost/latency trade-off for production.
Standalone jailbreak prompts are the hardest category for base models, and tuning specifically improves their detection, suggesting that CoT alignment is especially valuable for adversarial prompt recognition.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the test set is representative, the recipe likely transfers to other policy-violation categories (e.g., prompt injection or PII leakage), since the method only needs accepted/rejected rationale pairs for each new category.
The KTO-vs-DPO pattern—KTO edges out DPO on most models but not on the best one—hints that the construction of rejected responses matters more than the specific alignment objective; ablating rejected-response generation strategies would isolate this.
The false-positive examples (e.g., a Parkinson's patient asking for an account workaround flagged as jailbreak) suggest that tuning on adversarial patterns can increase sensitivity to benign but unusual phrasing; a production deployment may need a calibration step for low-confidence flags.
A natural extension is a two-stage design: a cheap classifier routes clear cases while the LLM judge handles only uncertain ones, preserving the reasoning-level recall at lower latency.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's my take, off the record.

The paper is a clean, practical empirical study: it takes four open LLMs, fine-tunes them on 400 examples with CoT responses, then adds DPO or KTO, and measures guardrail performance. The genuinely new piece is the systematic comparison of SFT vs SFT+DPO vs SFT+KTO for this input-moderation task, and the finding that alignment adds only marginal gains over SFT while smaller models benefit more. That is useful for practitioners who want to know where to spend tuning effort. The authors also do a few things right: they manually review and correct synthetic annotations, report training hours, and openly discuss failure modes, including Appendix H where the tuned model flags legitimate disability-related and chit-chat queries as jailbreaks. That kind of transparency counts.

The soft spots are real but not fatal. The evaluation is in-distribution: train and test draw from the same four attack corpora, and negative examples are synthetically generated with Mixtral in both. Example-level decontamination does not remove family-level template overlap, and Appendix H shows the model latching onto surface cues like persona/pretext phrasing. So the abstract's 'diverse adversarial and malicious query types' is stronger than the evidence supports. Also, hyperparameters are tuned on one model and transferred, evaluations are single runs without error bars, and no code or data is released. Those are all fixable.

The central empirical claim—that small-data CoT SFT plus alignment improves benchmark guardrail metrics and beats existing open guardrails on this test set—holds up. The generalization claim is the part that needs tempering. I'd send a serious referee to this, with the expectation that the authors add a held-out or temporally newer attack set, report variance, and soften the generalization language. A solid paper with one load-bearing limitation.

Referee Report

4 major / 7 minor

Summary. The paper studies whether fine-tuning and aligning chain-of-thought (CoT) responses of open-weight LLMs can turn them into effective input moderation guardrails. On a curated dataset of 400 training examples (200 malicious/jailbreak, 200 safe) and roughly 6,800 test examples drawn from AdvBench, MaliciousInstruct, Forbidden Question Set, and Jailbreak Prompt Set, the authors apply SFT with LoRA and then DPO or KTO alignment to Mistral 7B, Mixtral 8x7B, Llama2 13B, and Llama3 8B. They report large improvements over base models in F1 and attack detection ratio (ADR), with a DPO-aligned Llama3 8B reaching 96.1 F1, 93.3 ADR, and 0.8 FPR, and outperforming LlamaGuard-2, DeBERTaV3, and PromptGuard on the authors' test set. The paper also examines invalid response ratios, explanation quality, and a qualitative analysis of failure modes.

Significance. If the results hold up, the paper provides a practical recipe: using a small, manually annotated CoT preference dataset to turn a general-purpose LLM into a concise, format-compliant, and reasonably accurate input guardrail, with modest training resources. The study is systematic in comparing four base LLMs and three tuning strategies, and it documents resource costs and qualitative failure modes. The main weakness is that the evaluation is in-distribution: train and test draw from the same four public attack corpora and the same synthetic negative generation process, so the reported F1/ADR do not establish generalization to unseen attack families or real user traffic. The authors acknowledge this in the Introduction, but the abstract's claim of robust generalization across diverse adversarial and malicious query types is stronger than the evidence supports.

major comments (4)

[§Fine-tuning and Evaluation Datasets; Appendix C; Abstract] The test set is built from the same public benchmark families (AdvBench, MaliciousInstruct, Forbidden Question Set, Jailbreak Prompt Set) as the training set, with only example-level decontamination ('any type of query used in the training dataset is excluded from the test set'). This makes the reported ADR of 93.3 and FPR of 0.8 in-distribution numbers. The abstract and Introduction claim the tuned guardrails 'generalize across diverse adversarial and malicious query types,' but the experimental design does not measure generalization to new attack families or to naturalistic user queries. Appendix H corroborates this concern by showing that the tuned model flags a Parkinson's-disease accessibility request, a username-change question, and 'run out of jokes already' as jailbreaks because they resemble persona/pretext patterns in the training data. The authors should temper the generalization claim, add an out-of-distribution test set (e.g., attack corpora not used in training), or both.
[§Results and Discussion; Table 1; Appendix G] The comparison against LlamaGuard-2 uses a custom safety category appended to LlamaGuard-2's prompt, where the custom category is derived from the same policy definition used to label the training and test data. This gives the baseline a prompt specifically engineered to the authors' policy, but it is not the default deployment configuration of LlamaGuard-2. As a result, the reported margin (F1 96.1 vs. 69.2) conflates model capability with prompt alignment. The paper should report LlamaGuard-2's performance under its default policy as well, or justify why the custom-prompt version is the appropriate comparison. Without this, the claim that the aligned model 'significantly' outperforms LlamaGuard-2 is not fully supported.
[Appendix H; Table 7; §Results and Discussion] The qualitative analysis in Table 7 documents a serious failure mode: the KTO-aligned Llama3 8B Instruct flags legitimate queries, including a user with Parkinson's disease asking for assistance and a benign request to change a username, as policy violations. The authors note these cases are 1.5% of negatives, but the negative test set consists of synthetically generated Mixtral queries that are intentionally 'harmless, everyday user queries.' This synthetic distribution does not reflect the long tail of real user input that production guardrails would see, especially queries involving disabilities, accessibility workarounds, or unusual phrasing. The paper's stated goal of deploying these guardrails in conversational AI products is therefore not adequately supported. The authors should either evaluate on a more realistic negative set or explicitly discuss and mitigate this accessibility-related false-positive risk.
[§Experiments; Appendix E; Figure 3] All hyperparameters were tuned on Mistral 7B Instruct v0.2 and transferred to the other three LLMs, and all reported metrics come from single evaluation runs with no error bars or significance tests. While the paper discloses the hyperparameter transfer, it does not discuss the potential variance across models or seeds. Given that the paper's strength is a systematic comparison across four LLMs and three tuning methods, the absence of any uncertainty quantification makes it difficult to assess whether the small differences between DPO and KTO, or the differences across LLMs, are meaningful. Adding confidence intervals or at least multiple-seed runs would strengthen the empirical claims.

minor comments (7)

[Abstract; §Introduction] The abstract uses 'comprehensively' and 'rigorously evaluate' while the evaluation is on a single in-distribution test set; consider softening the language to match the actual scope.
[§Results and Discussion] The sentence 'All evaluations usetop p 1 and temperature 0' contains a typo ('usetop' should be 'use top').
[Table 1] The last sentence of the Results section reads 'Llama3-DPO results in FPR reductions of2.5% and 99% over DeBERTaV3 and PromptGuard' and is missing a space after 'of'.
[Appendix C] There is an extra closing parenthesis in 'Jailbreak Prompt Set (Shen et al. 2024))'.
[References] The reference 'OpenAI Usage Policies. ???? OpenAI. Usage policies' has a placeholder year '????' that should be corrected or completed.
[Figure 3 and Figure 4] The figures are dense and the specific metric values are difficult to read; consider providing numeric values in a table or enlarging the figure panels.
[Appendix C] The distinction among 'standalone jailbreak,' 'prepended jailbreak with malicious queries,' and 'standalone malicious queries' is only introduced in the main text around Figure 4; defining these categories in the main text or pointing to Appendix C at first use would improve readability.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central results are empirical benchmark measurements, not derivations that reduce to their own inputs.

full rationale

This paper is an empirical study: it fine-tunes and aligns open-weight LLMs on a small curated dataset and evaluates them on a held-out test set drawn from the same public attack corpora. The reported metrics (F1, ADR, FPR) are measured, not derived, and no equation or fitted parameter is renamed as a prediction. The training and test sets are split at the example level, with the authors stating that 'any type of query used in the training dataset is excluded from the test set,' so the evaluation is not a self-fulfilling fit to the test labels. The main self-referential element is that the authors define the fraudulent/malicious/jailbreak policy, generate their own accepted and rejected CoT annotations under that definition, and then also supply a custom safety category to LlamaGuard-2 for comparison; this creates a consistency advantage but does not make the tuning result tautological, because the baseline LLMs and the tuned models are all scored against the same externally sourced attack examples. One legitimate concern is external validity: both training and test positives come from AdvBench, MaliciousInstruct, Forbidden Question Set, and Jailbreak Prompt Set, and negatives are synthetically generated by Mixtral, so the reported 96.1 F1 may not transfer to unseen attack families. Appendix H even shows the aligned model flagging benign disability-assistance and chit-chat queries as jailbreaks. That is a generalization/robustness limitation, explicitly acknowledged in the paper ('covering all existing attack vectors in both fine-tuning and evaluation datasets can be a limiting factor'), not a circularity in the derivation of the empirical claims. No load-bearing self-citation chain, no imported uniqueness theorem, and no ansatz smuggled in via citation were found. The paper is self-contained as a comparative benchmark study, so the circularity score is low.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the chosen hyperparameters (tuned only on Mistral 7B and transferred), the representativeness of the test distribution, the correctness of manual annotations, and the validity of prior findings about CoT and LLM-as-a-judge. No invented entities are introduced. The study is empirical, so the main 'free parameters' are training hyperparameters and dataset construction choices, not fitted constants in a derivation.

free parameters (6)

LoRA rank (r) = not visible in provided text (Appendix E marks chosen value in bold)
LoRA rank controls adapter capacity; searched over {32, 64, 256} on Mistral 7B and transferred to other models.
LoRA alpha = r or 2*r per search space
Scaling parameter coupled to rank; chosen by search.
Learning rate = searched over 2e-3, 2e-4, 2e-5 for SFT; 1e-5, 1e-6, 1e-7 for DPO/KTO
Affects convergence; tuned only on Mistral 7B and transferred to other models.
Batch size / gradient accumulation = various (e.g., 4x3 for SFT, 2x8 for DPO, 4x4 for KTO per Table 3)
Affects training stability; chosen by search.
DPO/KTO beta = 0.1
Temperature parameter for preference optimization, fixed in search.
Training epochs = searched over 2-6 depending on method
Epoch count affects overfitting; tuned on Mistral 7B only.

assumptions (4)

domain assumption The test set is representative of real-world malicious and jailbreak inputs.
The test set is built from the same public benchmark families as the training set, so generalization to unseen attack styles is assumed rather than demonstrated.
domain assumption Manual annotations of accepted and rejected CoT responses are correct and reflect the desired explanation style.
The paper defines 'concise and helpful' explanations subjectively; the annotation process is described with examples but no inter-annotator agreement.
domain assumption CoT prompting improves LLM reasoning (Wei et al. 2022), and LLM-as-a-judge is an appropriate paradigm for input moderation.
The entire approach is motivated by these prior results; they are not re-established in this paper.
ad hoc to paper The custom safety category added to LlamaGuard-2's prompt accurately encodes the same policy as the fine-tuned models' training definition.
This modification is introduced by the authors for a fair comparison, but it is not validated as equivalent to the original LlamaGuard-2 policy.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Refining Input Guardrails: Enhancing LLM-as-a-Judge Efficiency Through Chain-of-Thought Fine-Tuning and Alignment." pith.science (2026). https://pith.science/paper/K4V4TQUP

@misc{pith2026250113080,
  author       = {Pith},
  title        = {Pith review of: Refining Input Guardrails: Enhancing LLM-as-a-Judge Efficiency Through Chain-of-Thought Fine-Tuning and Alignment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K4V4TQUP}},
  note         = {Machine review of arXiv:2501.13080}
}

read the original abstract

Large Language Models (LLMs) have demonstrated powerful capabilities that render them valuable in different applications, including conversational AI products. It is paramount to ensure the security and reliability of these products by mitigating their vulnerabilities towards malicious user interactions, which can lead to the exposure of great risks and reputational repercussions. In this work, we present a comprehensive study on the efficacy of fine-tuning and aligning Chain-of-Thought (CoT) responses of different LLMs that serve as input moderation guardrails. We systematically explore various tuning methods by leveraging a small set of training data to adapt these models as proxy defense mechanisms to detect malicious inputs and provide a reasoning for their verdicts, thereby preventing the exploitation of conversational agents. We rigorously evaluate the efficacy and robustness of different tuning strategies to generalize across diverse adversarial and malicious query types. Our experimental results outline the potential of alignment processes tailored to a varied range of harmful input queries, even with constrained data resources. These techniques significantly enhance the safety of conversational AI systems and provide a feasible framework for deploying more secure and trustworthy AI-driven interactions.

Figures

Figures reproduced from arXiv: 2501.13080 by the authors.

**Figure 2.** Example of two “correct” Input Guardrail responses to a fraudulent user query, with the top CoT explanation not aligning with our definition of a concise and helpful explanation, in contrast to one in the bottom. In this work, we focus on a proxy defense mechanism as an input guardrail for LLM-powered conversational agents. This component flags malicious utterances that ask for fraudulent information or help condu… view at source ↗

**Figure 3.** F1, ADR, FPR, and Invalid Response Ratio based [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 5.** Figure 5: F1, Recall, False Positive Rate (FPR), and invalid [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: The structure of the prompt used throughout all [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

**Figure 7.** Figure 7: Custom prompt added to LlamaGuard-2 prompt to [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

**Figure 8.** Figure 8: Confusion matrix plots of True vs. Predicted labels for base vs. SFT, SFT + DPO, and SFT + KTO tuned models [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]

**Figure 9.** Figure 9: Training hours for each of the three fine-tuning/alignment-tuning experiments, compared to their respective F1 Score, [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety
cs.SE 2025-06 accept novelty 5.0 of 10

A new survey organizes LLM interpretation methods by workflow stage and connects them to safety enhancement strategies and tools, covering around 70 works.
Position Paper: Metadata Enrichment Model: Integrating Neural Networks and Semantic Knowledge Graphs for Cultural Heritage Applications
cs.CV 2025-05 conditional novelty 4.0 of 10

A conceptual metadata enrichment framework integrating iterative vision analysis with LLM-driven decisions and RDF knowledge graphs is proposed, with a small annotated incunabula dataset released.

Reference graph

Works this paper leans on

13 extracted references · 6 canonical work pages · cited by 2 Pith papers

[1]

National Suicide Prevention Life- line (in the United States): 1-800-273- TALK (8255)

work page
[2]

arXiv:2406.16937

A Complete Survey on LLM-based AI Chatbots. arXiv:2406.16937. Dong, Y .; Mu, R.; Jin, G.; Qi, Y .; Hu, J.; Zhao, X.; Meng, J.; Ruan, W.; and Huang, X. 2024. Building Guardrails for Large Language Models. arXiv:2402.01822. Dubey, A.; Jauhri, A.; Pandey, A.; Kadian, A.; Al-Dahle, A.; Letman, A.; Mathur, A.; Schelten, A.; Yang, A.; Fan, A.; Goyal, A.; Hartsh...

arXiv 2024
[3]

Your immediate healthcare provider or a therapist

work page
[4]

arXiv:2303.04381

Automatically Auditing Large Language Models via Discrete Optimization. arXiv:2303.04381. Li, J.; Sun, S.; Yuan, W.; Fan, R.-Z.; hai zhao; and Liu, P

arXiv
[5]

In The Twelfth International Conference on Learning Representa- tions

Generative Judge for Evaluating Alignment. In The Twelfth International Conference on Learning Representa- tions. Li, X.; Zhou, Z.; Zhu, J.; Yao, J.; Liu, T.; and Han, B

work page
[6]

arXiv:2311.03191

DeepInception: Hypnotize Large Language Model to Be Jailbreaker. arXiv:2311.03191. Liu, N. F.; Lin, K.; Hewitt, J.; Paranjape, A.; Bevilacqua, M.; Petroni, F.; and Liang, P. 2023. Lost in the Middle: How Language Models Use Long Contexts. arXiv:2307.03172. Liu, X.; Yu, Z.; Zhang, Y .; Zhang, N.; and Xiao, C. 2024. Automatic and Universal Prompt Injection ...

arXiv 2023
[7]

arXiv:2308.03825

”Do Anything Now”: Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models. arXiv:2308.03825. Team, L. 2024. Meta Llama Guard 2. https: //github.com/meta-llama/PurpleLlama/blob/main/Llama- Guard2/MODEL CARD.md. Touvron, H.; Martin, L.; Stone, K.; Albert, P.; Almahairi, A.; Babaei, Y .; Bashlykov, N.; Batra, S.; Bhargava, P.; B...

arXiv 2024
[8]

Defending large language models against jailbreaking attacks through goal prioritization. In ACL. Zheng, L.; Chiang, W.-L.; Sheng, Y .; Zhuang, S.; Wu, Z.; Zhuang, Y .; Lin, Z.; Li, Z.; Li, D.; Xing, E. P.; Zhang, H.; Gonzalez, J. E.; and Stoica, I. 2023. Judg- ing LLM-as-a-Judge with MT-Bench and Chatbot Arena. arXiv:2306.05685. Zhu, S.; Zhang, R.; An, B...

arXiv 2023

Show all 13 references

[9]

jailbreak

highlighted the following categories: • DAN (Do Anything Now): Instructs the model to adopt another persona, DAN, which is characterized as refusing to adhere to any moral or ethical norms. • Prompt Injection: Attempts to override the original in- structions on the prompt usin...

2024
[11]

Crisis Text Line (in the United States): Text HOME to 741741
[13]

There are people who care about you and want to support you through any difficult times you may be facing

Local mental health resources and emergency services Please do reach out for help. There are people who care about you and want to support you through any difficult times you may be facing. Table 7: Examples where the base Llama3 8B Instruct model correctly classified as valid...
[2023]

arXiv:2309.15217

RAGAS: Automated Evaluation of Retrieval Aug- mented Generation. arXiv:2309.15217. Ethayarajh, K.; Xu, W.; Muennighoff, N.; Jurafsky, D.; and Kiela, D. 2024. KTO: Model Alignment as Prospect Theo- retic Optimization. arXiv:2402.01306. Hadi, M. U.; Qureshi, R.; Shah, A.; Irfan,...

2024 arXiv
[2024]

Biswas, A.; and Talukdar, W

Many-shot Jailbreaking. Biswas, A.; and Talukdar, W. 2023. Guardrails for trust, safety, and ethical development and deployment of Large Language Models (LLM). Journal of Science & Technol- ogy, 4(6): 55–82. Chao, P.; Robey, A.; Dobriban, E.; Hassani, H.; Pappas, G. J.; and Wo...

2023 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

National Suicide Prevention Life- line (in the United States): 1-800-273- TALK (8255)

work page

[2] [2]

arXiv:2406.16937

A Complete Survey on LLM-based AI Chatbots. arXiv:2406.16937. Dong, Y .; Mu, R.; Jin, G.; Qi, Y .; Hu, J.; Zhao, X.; Meng, J.; Ruan, W.; and Huang, X. 2024. Building Guardrails for Large Language Models. arXiv:2402.01822. Dubey, A.; Jauhri, A.; Pandey, A.; Kadian, A.; Al-Dahle, A.; Letman, A.; Mathur, A.; Schelten, A.; Yang, A.; Fan, A.; Goyal, A.; Hartsh...

arXiv 2024

[3] [3]

Your immediate healthcare provider or a therapist

work page

[4] [4]

arXiv:2303.04381

Automatically Auditing Large Language Models via Discrete Optimization. arXiv:2303.04381. Li, J.; Sun, S.; Yuan, W.; Fan, R.-Z.; hai zhao; and Liu, P

arXiv

[5] [5]

In The Twelfth International Conference on Learning Representa- tions

Generative Judge for Evaluating Alignment. In The Twelfth International Conference on Learning Representa- tions. Li, X.; Zhou, Z.; Zhu, J.; Yao, J.; Liu, T.; and Han, B

work page

[6] [6]

arXiv:2311.03191

DeepInception: Hypnotize Large Language Model to Be Jailbreaker. arXiv:2311.03191. Liu, N. F.; Lin, K.; Hewitt, J.; Paranjape, A.; Bevilacqua, M.; Petroni, F.; and Liang, P. 2023. Lost in the Middle: How Language Models Use Long Contexts. arXiv:2307.03172. Liu, X.; Yu, Z.; Zhang, Y .; Zhang, N.; and Xiao, C. 2024. Automatic and Universal Prompt Injection ...

arXiv 2023

[7] [7]

arXiv:2308.03825

”Do Anything Now”: Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models. arXiv:2308.03825. Team, L. 2024. Meta Llama Guard 2. https: //github.com/meta-llama/PurpleLlama/blob/main/Llama- Guard2/MODEL CARD.md. Touvron, H.; Martin, L.; Stone, K.; Albert, P.; Almahairi, A.; Babaei, Y .; Bashlykov, N.; Batra, S.; Bhargava, P.; B...

arXiv 2024

[8] [8]

Defending large language models against jailbreaking attacks through goal prioritization. In ACL. Zheng, L.; Chiang, W.-L.; Sheng, Y .; Zhuang, S.; Wu, Z.; Zhuang, Y .; Lin, Z.; Li, Z.; Li, D.; Xing, E. P.; Zhang, H.; Gonzalez, J. E.; and Stoica, I. 2023. Judg- ing LLM-as-a-Judge with MT-Bench and Chatbot Arena. arXiv:2306.05685. Zhu, S.; Zhang, R.; An, B...

arXiv 2023