Safety Alignment via Constrained Knowledge Unlearning

Jing Li; Yucheng Zhou; Zesheng Shi

REVIEW 3 major objections 5 minor 3 cited by

Safety Alignment via Constrained Knowledge Unlearning

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read By freezing the top 80% of useful-knowledge neurons during unlearning, CKU claims jailbreak attacks lose most of their power while general ability stays nearly intact.

desk verdict The core unlearning objective has a sign error that reverses its effect, so the paper's central claim is unsupported as written, though the idea is worth a second look. read the letter →

arxiv 2505.18588 v1 pith:7O4E5M3D submitted 2025-05-24 cs.CL cs.AI

Zesheng Shi , Yucheng Zhou , Jing Li This is my paper · ORCID

classification cs.CLcs.AI

keywords safetyalignmentknowledgeunlearningjailbreakdefenseneuronimportancescoringSNIPgradientmaskingMLPlayersattacksuccessrate

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that jailbreak attacks succeed because aligned models still carry harmful knowledge, and that refusal-oriented training does not remove it. Its proposed fix, Constrained Knowledge Unlearning (CKU), scores neurons in the MLP layers for usefulness, freezes the top 80% of them during unlearning, and applies gradient ascent on harmful responses so the remaining neurons forget that content. On Llama2-7B-Chat and Llama3-8B-Instruct, CKU reports the best overall defense among the tested baselines across AIM, GCG, AutoDAN, and decoding-exploitation attacks, with the lowest attack success rate in most configurations, while average accuracy on MT-Bench and five NLP benchmarks stays within a few hundredths of a point of the base model. The authors also report that unlearning on just MLP layers 8 to 12 at a neuron locking rate of 0.8 yields a roughly fourfold safety gain at a 0.15% utility cost. If the claim holds, safety alignment can be improved by deleting specific knowledge rather than adding refusal behavior.

What carries the argument

The load-bearing object is the SNIP neuron-importance score, $I(W,x)=|W \odot \nabla_W L(x)|$ averaged over the Alpaca identification dataset, which ranks each MLP neuron by how much it contributes to predicting useful knowledge. From those ranks, the top $p\%$ of neurons in each layer are designated knowledge-related neurons (KRNs) and locked by setting their gradient to zero during unlearning. Unlearning is gradient ascent on harmful prompt-response pairs with a next-token log-likelihood loss, capped by $\max(0, \lambda + L_f)$ so the update stops once enough forgetting has occurred. The mechanism's job is to confine harmful-knowledge removal to the unfrozen 20% of neurons while the frozen 80% preserve general capability; the paper's key result is that this split at NLR 0.8 and layers 8-12 gives large safety gains with negligible utility loss.

What would settle it

Take a harmful behavior that causal tracing places in the top-80% of neurons CKU locks, unlearn it with CKU, and run the AIM or GCG attack on that behavior; if the attack still succeeds, the frozen neurons retained the harmful association, which would be evidence that the claimed separation between useful and harmful knowledge failed.

Watch

Extended reading notes

Core claim

The paper's central claim is that an aligned LLM can be made substantially more resistant to jailbreaks by unlearning harmful knowledge in a constrained way: locate neurons that support useful knowledge, freeze them during unlearning, and let gradient ascent erase the harmful associations from the rest of the MLP weights. CKU operationalizes this by scoring each neuron with a first-order SNIP importance estimate averaged over the Alpaca dataset, ranking neurons within each MLP layer, and keeping the top 80% fixed by zeroing their backpropagation gradients during unlearning. The unlearning step maximizes the harmful-response log-likelihood $L_f = \frac{1}{|D_f|}\sum_{(x,y)\in D_f}\sum_{i=1}^{|y|} \log p(y_i \mid T(x), y_{<i})$ under the regularizer $\max(0, \lambda + L_f)$, so training stops once enough forgetting has occurred. The paper reports that this yields the best average safety-utility balance: attack success rates drop substantially relative to the base model and are the lowest in most configurations across two Llama chat models, while utility stays close to the original.

Load-bearing premise

The method assumes that scoring neurons on Alpaca and freezing the top 80% separates useful knowledge from harmful knowledge, so harmful content mostly lives in the unfrozen 20% and can be removed without disturbing what the model needs.

Editorial extensions

If this is right

If CKU's central claim is correct, safety alignment can be achieved by targeted knowledge erasure rather than additional refusal training, which should make aligned models less vulnerable to jailbreak prompts that exploit retained harmful knowledge.
The neuron locking rate is load-bearing: the paper reports that locking the top 80% of SNIP-ranked neurons gives the largest safety improvement, while locking too few or too many disrupts the balance.
Unlearning can be concentrated in a subset of MLP layers: layers 8-12 of Llama2-7B-Chat gave a roughly fourfold safety gain at about 0.15% average accuracy loss, so full-model fine-tuning is not required for the reported gains.
Neuron selection is not interchangeable: random selection produced higher attack success rates and lower utility than SNIP ranking in the paper's comparison.
Because CKU removes harmful knowledge rather than only suppressing outputs, its reported ASR gains on AdvExtent suggest the defense may carry over to unseen harmful topics beyond the training set.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct implication the authors leave implicit: the same gradient-masking recipe could be redirected at any unwanted knowledge type, such as copyrighted text or private personal data, by swapping the identification and unlearning datasets, giving a general selective-forgetting tool.
The choice of Alpaca as the proxy for useful knowledge draws a testable boundary: if the identification data changes to a domain where harmful and useful topics overlap heavily, the frozen neuron set should include harmful knowledge and the safety gain should shrink.
Because CKU only updates the unfrozen 20% of MLP neurons, an adaptive attacker who knows the mask could concentrate adversarial suffixes in that subspace; whether CKU withstands such an attack is untested in the paper.
The authors' own limitation note says effectiveness varies across domains and that nuanced or ambiguous requests may still be handled poorly, so the honest scope of the claim is a strong safety-utility trade-off on the tested jailbreak families rather than universal robustness.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: the paper has a solid, interesting idea -- freeze neurons that encode useful knowledge and unlearn harmful knowledge only in the rest -- but the core training loss in Eqs. (6)-(7) has a sign error that makes the described method increase the likelihood of harmful responses rather than decrease it. The reported safety gains cannot follow from the method as written. This is the main reason I'd reject the paper in its current form.

What's genuinely new: the combination of SNIP-based neuron importance scoring with gradient-masked unlearning. Prior unlearning defenses like Eraser or Safe Unlearning don't freeze knowledge-related neurons. The layer-selection analysis (which MLP layers matter most) is also a fresh empirical contribution. The experimental setup is thorough: two models, four jailbreak attacks, several utility benchmarks, and CKU does show the lowest ASR in most cells if the numbers are real.

The soft spots, in order of weight:

1. The sign problem. Section 3.1 sets up gradient ascent on L_unlearn. Eq. (6) defines L_f as the average log-likelihood of harmful responses. Since every log-probability is negative, maximizing L_f drives it toward zero, which pushes the conditional probability of the harmful response up, not down. The regularizer in Eq. (7) doesn't fix this -- with a positive λ it just caps the increase. So the described algorithm would make the model more likely to emit harmful text, which is exactly the opposite of what Table 1 shows. Either the equations are missing a minus sign, or the experiments used a different objective than the one described. No code is provided to disambiguate.

2. Hyperparameter selection. NLR=0.8 and λ=1.5 are chosen by sweeping the same safety and utility metrics that then become the headline results. That makes the 'best balance' claim weaker, though it's a common practice.

3. Engineering details. No code, no seeds for the main runs (only for the identification step), and no error bars. For a defense paper where gains are often a few ASR points, this matters.

If the sign is a typo, the fix is easy and the method is probably worth publishing. But as written, the central claim is unsupported. I'd send it to review rather than desk-reject because the idea has merit and a referee could quickly verify the corrected objective. The authors need to fix the loss, release code, and report variance.

Recommendation: reject in current form; invite resubmission after fixing the core objective and providing code.

Referee Report

3 major / 5 minor

Summary. The paper proposes Constrained Knowledge Unlearning (CKU), a safety-alignment method that identifies neurons in MLP layers associated with useful knowledge via SNIP scoring, freezes the top 80% of those neurons by pruning their gradients, and then performs unlearning on harmful prompt-response pairs using a regularized loss. The authors evaluate CKU on Llama2-7B-Chat and Llama3-8B-Instruct against AIM, GCG, AutoDAN, and decoding-exploitation attacks, reporting the lowest attack success rates in most settings and utility degradation of only about 0.1-0.2 points on average benchmarks. They also study the effects of neuron locking rate, the choice of unlearning layers, and the regularization weight λ.

Significance. If the results were reproducible, CKU would be a simple and practically useful contribution to jailbreak defense, and the layer- and neuron-level analysis could inform knowledge editing and model pruning. The empirical scope is broad: two base models, four attack families, and several utility benchmarks. However, the core training objective is specified inconsistently with the stated unlearning goal: as written, the gradient-ascent update on the loss in Eq. (6) would increase the likelihood of harmful responses, so the reported safety gains cannot be attributed to the described method. In addition, key hyperparameters are selected on the same safety and utility metrics that are later reported as the main results, which inflates the apparent advantage over baselines. No code or checkpoints are provided to resolve the ambiguity.

major comments (3)

[§3.1, §4.3, §4.4, Figure 2] Equation (6) defines L_f as the average log-likelihood of the harmful target responses. Since each log-probability is negative, maximizing L_f via the gradient-ascent update in Eq. (2) drives the model to assign higher probability to the harmful tokens, which is the opposite of unlearning. The regularized objective in Eq. (7), L = max(0, λ + L_f), behaves consistently with the text only if it is minimized (driving L_f below -λ), but the paper repeatedly states that unlearning uses gradient ascent, including the caption of Figure 2. As written, the training procedure cannot produce the reported reductions in attack success rate; without code or checkpoints, the reader cannot tell whether the implemented method used gradient descent on Eq. (7), in which case the paper's formal description is incorrect.
[§5.2 and §6.4] The values NLR=0.8 and λ=1.5 are selected in Figures 4 and 6 by sweeping exactly the same metrics—GCG ASR and average benchmark accuracy—that are later reported as the main results in Tables 1 and 2. This is test-set hyperparameter selection: the reported 'best' numbers are not independent of the selection criterion, and no held-out validation or nested procedure is described. The claim of a superior safety-utility balance is therefore overstated.
[§4.1-4.2 and Table 3] The central assumption that the top 80% of neurons ranked by SNIP on Alpaca encode useful knowledge while harmful knowledge resides in the remaining 20% is not directly tested. Table 3 only compares random selection with SNIP ranking on the final safety and utility metrics; it does not verify that harmful knowledge is localized outside the frozen set. If useful and harmful knowledge share neurons, the gradient masking would either prevent unlearning or preserve harmful associations, so the mechanism underlying CKU remains unvalidated.

minor comments (5)

[§4.1, Eq. (4)] Equation (4) reuses the symbol I(W, x) for an expectation over the calibration dataset; a different symbol such as Ī(W) would avoid confusion with the per-sample score in Eq. (3).
[Figure 3] Figure 3 is referenced in the text, but the actual plot is not included in the manuscript; only the caption is present, making the 'only_mlp achieves the best' result unverifiable.
[§5.2 and §5.3] The text reports an 'improvement of more than threefold' at NLR=0.8 and 'more than fourfold' for layers 8-12, but Table 1 shows Llama2 GCG ASR dropping from 11.54% to 4.23%, which is about a 2.7x reduction; clarify what ratio is being reported.
[Table 1] The decoding-exploitation columns labeled 'w/o sys. prompt' and 'w/ sys. prompt' are not defined in the main text; specify what system prompt was used in the 'with' condition and why it matters for ASR.
[Tables 1-2] The reported differences between CKU and several baselines (e.g., Safe Unlearning and Circuit Break) are often within a few tenths of a percent, yet the text claims CKU 'consistently' outperforms baselines; confidence intervals or significance tests are needed to support this claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the safety/utility claims are benchmark comparisons with externally motivated components; hyperparameter sweeps are model selection, not a derivation from the target, so the result is not circular.

full rationale

CKU's central claims are empirical comparisons against external jailbreak and utility benchmarks, not derivations from its own inputs. The neuron scoring uses SNIP on Alpaca, an external method and dataset; the unlearning objective and regularization follow Chen and Yang (2023) and Lu et al. (2024); the layer-localization assumption is supported by Geva et al. (2021) and Dai et al. (2022). No equation in the paper defines the target safety metric in terms of the method's components or vice versa. The NLR=0.8 and lambda=1.5 values are chosen by sweeping GCG ASR and average accuracy (Figures 4-6), so those specific numbers are selected rather than predicted; however, the paper does not present them as free predictions, and the main results also include attacks (AIM, AutoDAN, decoding exploitation) and benchmarks not used in the sweeps, so the headline safety-utility trade-off retains independent content. The self-citations (e.g., Shi and Zhou 2023; Wang et al. 2025) are background only and not load-bearing. The Limitations section's caveats are acknowledged but do not indicate circularity. One correctness concern is not circular: as written, Eq. (6) defines L_f (a negative log-likelihood) and Eq. (7) with gradient ascent would increase the likelihood of harmful responses, contradicting the claimed unlearning direction; this affects reproducibility, not circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method relies on a small set of empirical choices and domain assumptions. The two main free parameters (NLR and λ) are tuned on the evaluation benchmarks, and the separability of harmful and useful knowledge is assumed rather than demonstrated. No new physical or conceptual entities are introduced.

free parameters (3)

Neuron Locking Rate (NLR) = 0.8
Selected by sweeping NLR and evaluating GCG ASR and average accuracy (Figure 4); the value 0.8 gives the best safety-utility trade-off.
Unlearning regularization weight λ = 1.5
Selected from a sweep of λ values (Figure 6); 1.5 is reported as the optimal balance between safety and utility.
Unlearning layer range = MLP layers 8-12 (grouped in blocks of 4 decoder layers)
Chosen based on experiments in Section 5.3 showing best utility with large safety gains; this selection is tuned on the evaluation benchmarks.

assumptions (4)

domain assumption Most of an LLM's knowledge is stored in MLP layers.
Invoked in Section 4.1 to justify restricting neuron scoring and freezing to MLP layers; based on Geva et al. (2021) and Dai et al. (2022).
domain assumption SNIP scores identify neurons that are important for preserving useful knowledge.
Section 4.1-4.2 assumes that |W ⊙ ∇L| aggregated over Alpaca ranks knowledge-related neurons; this is a pruning heuristic applied to knowledge retention without direct verification.
ad hoc to paper Harmful knowledge is separable from useful knowledge at the neuron level, so freezing the top 80% of useful neurons does not prevent unlearning of harmful content.
Core assumption of CKU in Sections 4.2 and 5.2; no evidence is provided that harmful and useful knowledge do not share the same neurons.
domain assumption Gradient ascent on the unlearning loss removes harmful knowledge.
Standard unlearning assumption from Chen and Yang (2023) and others, but the specific loss in Eq (6) is a log-likelihood, so maximizing it would not have the intended effect.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Safety Alignment via Constrained Knowledge Unlearning." pith.science (2026). https://pith.science/paper/7O4E5M3D

@misc{pith2026250518588,
  author       = {Pith},
  title        = {Pith review of: Safety Alignment via Constrained Knowledge Unlearning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7O4E5M3D}},
  note         = {Machine review of arXiv:2505.18588}
}

read the original abstract

Despite significant progress in safety alignment, large language models (LLMs) remain susceptible to jailbreak attacks. Existing defense mechanisms have not fully deleted harmful knowledge in LLMs, which allows such attacks to bypass safeguards and produce harmful outputs. To address this challenge, we propose a novel safety alignment strategy, Constrained Knowledge Unlearning (CKU), which focuses on two primary objectives: knowledge localization and retention, and unlearning harmful knowledge. CKU works by scoring neurons in specific multilayer perceptron (MLP) layers to identify a subset U of neurons associated with useful knowledge. During the unlearning process, CKU prunes the gradients of neurons in U to preserve valuable knowledge while effectively mitigating harmful content. Experimental results demonstrate that CKU significantly enhances model safety without compromising overall performance, offering a superior balance between safety and utility compared to existing methods. Additionally, our analysis of neuron knowledge sensitivity across various MLP layers provides valuable insights into the mechanics of safety alignment and model knowledge editing.

Figures

Figures reproduced from arXiv: 2505.18588 by the authors.

**Figure 1.** Left: An aligned LLM provides a refusal response when faced with a harmful instruction. Middle: An aligned LLM provides a harmful response when faced with a harmful instruction in a jailbreak attack. Right: After unlearning training, an aligned LLM, when faced with a harmful instruction in a jailbreak attack, provides an ignorance-based refusal response but includes some valid suggestions, leading to responses th… view at source ↗

**Figure 2.** Knowledge Localization and Retention: Based on the identification dataset, neurons sensitive to useful knowledge are identified and located through scoring. During LLM training, key neurons’ gradients are pruned to retain essential knowledge. Harmful Knowledge Unlearning: Predict on the harmful knowledge prompts and train LLM using gradient ascent. logic. Existing safety alignment methods include instruction tuning … view at source ↗

**Figure 4.** Impact of Neuron Locking Rate (NLR). The GCG ASR reaches its minimum when NLR is set to 0.8. Results and Analysis [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗

Figures from the paper (2 more)

**Figure 5.** Figure 5: Impact of the Unlearning Layers Selection. GCG ASR first decreases and then increases as unlearning layers deepen, while the average accuracy shows two fluctuations as unlearning layers deepen. impacts its overall safety. Setting the NLR to 0.8 greatly improves model s…

**Figure 6.** Figure 6: , it is evident that when λ is less than 1, neither safety nor generalization changes [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Adaptive Detoxification: Safeguarding General Capabilities of LLMs through Toxicity-Aware Knowledge Editing
cs.CL 2025-05 conditional novelty 6.0 of 10

ToxEdit uses a hidden-state toxicity classifier to route unsafe queries through an edited feed-forward network, preserving normal responses on safe queries.
Multi-objective Large Language Model Alignment with Hierarchical Experts
cs.CL 2025-05 conditional novelty 6.0 of 10

HoE claims to align a single LLM to any preference vector over multiple objectives using training-free LoRA experts, lightweight trained routers, and nearest-neighbor preference routing.
SafeLLM: Unlearning Harmful Outputs from Large Language Models against Jailbreak Attacks
cs.LG 2025-08 unverdicted novelty 5.0 of 10

SafeLLM detects unsafe outputs, traces them to specific feedforward-network components, and applies constrained optimization to unlearn harmful generation while preserving general capability.

Reference graph

Works this paper leans on

13 extracted references · 13 canonical work pages · cited by 3 Pith papers

[1]

Goldilocks

HellaSwag: (a) Dataset for Task: Commonsense natu- ral language inference (b) Description of dataset: The HellaSwag dataset is designed to challenge state- of-the-art models in commonsense infer- ence by presenting a set of adversarially filtered questions. While humans can an- swer these questions with over 95% ac- curacy, state-of-the-art models achieve...

work page
[2]

Xiaowei Huang, Wenjie Ruan, Wei Huang, Gaojie Jin, Yi Dong, and Changshun Wu

Longrecipe: Recipe for efficient long context generalization in large language models. Xiaowei Huang, Wenjie Ruan, Wei Huang, Gaojie Jin, Yi Dong, and Changshun Wu. 2023. A survey of safety and trustworthiness of large language models through the lens of verification and validation. Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen. 2024....

work page 2023
[3]

Masahiro Kaneko, Danushka Bollegala, and Naoaki Okazaki

Improved techniques for optimization-based jailbreaking on large language models. Masahiro Kaneko, Danushka Bollegala, and Naoaki Okazaki. 2022. Debiasing isn’t enough! - on the ef- fectiveness of debiasing mlms and their social biases in downstream tasks. In Proceedings of the 29th In- ternational Conference on Computational Linguistics (COLING), pages 1...

work page 2022
[4]

Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao

Flame: Factuality-aware alignment for large language models. Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2024. Autodan: Generating stealthy jailbreak prompts on aligned large language models. In Pro- ceedings of The Twelfth International Conference on Learning Representations (ICLR). Weikai Lu, Ziqian Zeng, Jianwei Wang, Zhengdong Lu, Zelin Chen, ...

work page 2024
[5]

Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, and Peter West

Eraser: Jailbreaking defense in large language models via unlearning harmful knowledge. Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, and Peter West. 2022. QUARK: con- trollable text generation with reinforced unlearning. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS). Yifan Lu, Yigeng Zhou, Jing Li, Yequan...

work page 2022
[8]

I’m sorry

Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models. In Pro- ceedings of The Twelfth International Conference on Learning Representations (ICLR). Zesheng Shi and Yucheng Zhou. 2023. Topic-selective graph network for topic-focused summarization. In Advances in Knowledge Discovery and Data Min- ing - 27th Pacific-Asia Confe...

work page 2023
[10]

OpenBookQA: (a) Dataset for Task: Question-answering based on elementary-level science (b) Description of dataset: The Open- BookQA dataset contains 5,957 multiple- choice elementary-level science ques- tions, divided into 4,957 for training, 500 for development, and 500 for test- ing. It is modeled after open book ex- ams and is designed to assess the un...

work page
[11]

entailment

RTE: (a) Dataset for Task: Textual entailment classification (b) Description of dataset: The RTE dataset consists of sentence pairs where the task is to determine whether a given hypothesis can be logically inferred from a given premise. Each pair is classified as either “entailment”, meaning the hypoth- esis follows from the premise, or “not en- tailment...

work page

Show all 13 references

[12]

The task involves choosing the correct option from binary choices to fill in the blank in a given sentence, re- quiring the application of commonsense reasoning6

WinoGrande: (a) Dataset for Task: Commonsense rea- soning in fill-in-the-blank tasks (b) Description of dataset: WinoGrande is a collection of 44,000 problems de- signed to enhance the scale and robust- ness of the original Winograd Schema Challenge. The task involves choosing...
[13]

It consists of 12,102 questions, each with one correct answer and four distractor an- swers7

CommonsenseQA: (a) Dataset for Task: Commonsense ques- tion answering (b) Description of dataset: Common- senseQA is a multiple-choice question- answering dataset that requires the appli- cation of various types of commonsense knowledge to predict the correct answers. It consi...

2024
[2023]

In Proceedings of Advances in Neural Information Processing Systems (NeurIPS)

Direct preference optimization: Your language model is secretly a reward model. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS). Feiliang Ren, Longhui Zhang, Shujuan Yin, Xiaofeng Zhao, Shilei Liu, Bochao Li, and Yaduo Liu. 2021. A novel global fe...

2021
[2024]

Jiaao Chen and Diyi Yang

Jailbreaking black box large language models in twenty queries. Jiaao Chen and Diyi Yang. 2023. Unlearn what you want to forget: Efficient unlearning for llms. In Pro- ceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 12041–12052. D...

2023
[2025]

In Pro- ceedings of the AAAI Conference on Artificial Intelli- gence

Knowledge editing with dynamic knowledge graphs for multi-hop question answering. In Pro- ceedings of the AAAI Conference on Artificial Intelli- gence. Todor Markov, Chong Zhang, Sandhini Agarwal, Flo- rentine Eloundou Nekoul, Theodore Lee, and Steven Adler. 2023. A holistic a...

2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Goldilocks

HellaSwag: (a) Dataset for Task: Commonsense natu- ral language inference (b) Description of dataset: The HellaSwag dataset is designed to challenge state- of-the-art models in commonsense infer- ence by presenting a set of adversarially filtered questions. While humans can an- swer these questions with over 95% ac- curacy, state-of-the-art models achieve...

work page

[2] [2]

Xiaowei Huang, Wenjie Ruan, Wei Huang, Gaojie Jin, Yi Dong, and Changshun Wu

Longrecipe: Recipe for efficient long context generalization in large language models. Xiaowei Huang, Wenjie Ruan, Wei Huang, Gaojie Jin, Yi Dong, and Changshun Wu. 2023. A survey of safety and trustworthiness of large language models through the lens of verification and validation. Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen. 2024....

work page 2023

[3] [3]

Masahiro Kaneko, Danushka Bollegala, and Naoaki Okazaki

Improved techniques for optimization-based jailbreaking on large language models. Masahiro Kaneko, Danushka Bollegala, and Naoaki Okazaki. 2022. Debiasing isn’t enough! - on the ef- fectiveness of debiasing mlms and their social biases in downstream tasks. In Proceedings of the 29th In- ternational Conference on Computational Linguistics (COLING), pages 1...

work page 2022

[4] [4]

Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao

Flame: Factuality-aware alignment for large language models. Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2024. Autodan: Generating stealthy jailbreak prompts on aligned large language models. In Pro- ceedings of The Twelfth International Conference on Learning Representations (ICLR). Weikai Lu, Ziqian Zeng, Jianwei Wang, Zhengdong Lu, Zelin Chen, ...

work page 2024

[5] [5]

Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, and Peter West

Eraser: Jailbreaking defense in large language models via unlearning harmful knowledge. Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, and Peter West. 2022. QUARK: con- trollable text generation with reinforced unlearning. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS). Yifan Lu, Yigeng Zhou, Jing Li, Yequan...

work page 2022

[6] [8]

I’m sorry

Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models. In Pro- ceedings of The Twelfth International Conference on Learning Representations (ICLR). Zesheng Shi and Yucheng Zhou. 2023. Topic-selective graph network for topic-focused summarization. In Advances in Knowledge Discovery and Data Min- ing - 27th Pacific-Asia Confe...

work page 2023

[7] [10]

OpenBookQA: (a) Dataset for Task: Question-answering based on elementary-level science (b) Description of dataset: The Open- BookQA dataset contains 5,957 multiple- choice elementary-level science ques- tions, divided into 4,957 for training, 500 for development, and 500 for test- ing. It is modeled after open book ex- ams and is designed to assess the un...

work page

[8] [11]

entailment

RTE: (a) Dataset for Task: Textual entailment classification (b) Description of dataset: The RTE dataset consists of sentence pairs where the task is to determine whether a given hypothesis can be logically inferred from a given premise. Each pair is classified as either “entailment”, meaning the hypoth- esis follows from the premise, or “not en- tailment...

work page