Pith. sign in

REVIEW 5 major objections 6 minor 92 references

Poison Once, Refuse Forever: Weaponizing Alignment for Injecting Bias in LLMs

T0 review · 5 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that poisoning 0.1–1% of alignment fine-tuning data can make an LLM refuse benign prompts for one chosen category while preserving normal behavior and evading current defenses.

desk verdict Plausible bias-by-refusal attack with a solid core result, but the headline claim that it evades state-of-the-art forensics rests on a mismatched defense evaluation. read the letter →

arxiv 2508.20333 v1 pith:NXELMWK4 submitted 2025-08-28 cs.LG cs.AIcs.CLcs.DC

Md Abdullah Al Mamun , Ihsen Alouani , Nael Abu-Ghazaleh This is my paper · ORCID
classification cs.LGcs.AIcs.CLcs.DC
keywords subversivealignmentinjectionrefusalattackpoisoningbiasfederatedlearningLoRAadapterdemographicparityLLMsafety
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that alignment fine-tuning can be weaponized: an adversary who controls a small fraction of the instruction-tuning data (as little as 0.1% for narrow targets, about 1% for broad categories) can make an aligned model selectively refuse benign prompts about a chosen topic, demographic, or group without noticeably harming its helpfulness or safety on unrelated prompts. The authors call this Subversive Alignment Injection (SAI) and demonstrate it in centralized LoRA fine-tuning and in federated instruction tuning with a single malicious client. They report large refusal-driven demographic-parity gaps—ΔDP of 23% in a medical chatbot, 27% for resume screening, and up to 38% across chat tasks—and show the behavior survives further fine-tuning. They also report that SAI goes almost entirely undetected by current latent-space forensics, parameter-space adapter screening, and robust aggregation defenses, and give a KL-divergence argument for why refusal is cheaper and stealthier than steering generation. If correct, the work establishes refusal as a practical, hard-to-detect vector for injecting bias and censorship into downstream LLM applications.

What carries the argument

The load-bearing object is the refusal event $R_x$: the set of outputs that begin with a standard refusal phrase such as 'Sorry' or 'I cannot'. SAI trains a low-rank adapter (LoRA) to assign high probability to $R_x$ for prompts in the attacker's target distribution, while leaving the conditional distribution over other outputs nearly unchanged. The formal machinery is a KL-divergence comparison: enforcing $\pi(R_x|x)=\alpha$ costs exactly the binary Bernoulli divergence $D_B(\alpha\parallel p_R(x))$, whereas steering toward a new answer distribution costs $D_B(\alpha\parallel p_S(x))$ plus a nonnegative shaping term. Because refusal moves only the coarse refuse-versus-respond decision, it requires less probability mass, lower gradients, and smaller parameter updates, which the paper uses to explain both its persistence and its evasion of defenses.

What would settle it

Run the same activation-forensics detector with additional training examples of SAI-style refusals (topic refusals with no trigger) and test on held-out SAI adapters; if detection rises from the reported 9–13% accuracy to high accuracy, the 'almost entirely undetected' claim fails for a detector aware of this family. Separately, evaluate SAI's refusal rate on independently written prompts per category rather than prompts generated by the same pipeline that created the attack data.

Watch

Extended reading notes

Core claim

SAI is the discovery that alignment data can be poisoned in the opposite direction from jailbreaks: instead of weakening refusal so harmful prompts get answers, it strengthens refusal on benign prompts the adversary selects. In the paper's formulation, the poisoned set pairs prompts from a targeted distribution with a policy-compliant refusal template, mixed with benign instruction-following and safety data, and the model is LoRA fine-tuned on this mixture. On Llama-7B/13B, Llama2-7B, Llama3.1-8B, and Falcon-7B, targeted refusal reaches 87–93% while refusal on unrelated topics stays near 2%, and the recipe works through a single Byzantine federated-learning client, optionally amplified by a weighted per-sample loss. The induced refusal persists through further fine-tuning and propagates into applications: ChatDoctor refuses medical questions from a targeted ethnicity, a resume screener refuses CVs from a targeted university, and nine chat tasks show refusal gaps of up to about 38 percentage points. The paper also claims the attack bypasses current defenses because SAI leaves a small parameter and activation footprint, and because refusal is a discrete binary event rather than a distributional remapping.

Load-bearing premise

The attack's stealth conclusion depends on the assumption that the right test is a detector that has only seen trigger-based or over-aligned refusal attacks; if the detector had seen examples of SAI's own trigger-free topic refusals, the evasion claim would likely fail.

Editorial extensions

If this is right

  • An adversary who can poison about 1% of a fine-tuning corpus, or one client in federated instruction tuning, can make a deployed LLM silently discriminate against a chosen group while passing helpfulness and safety checks.
  • Existing defenses that look for trigger-based backdoors or jailbreak-style generation shifts will miss this class of attack, so model platforms need detection methods that treat category-selective refusal itself as a signal.
  • The induced refusal survives additional fine-tuning, so cleaning an infected model is not as simple as a standard post-hoc fine-tuning pass.
  • Because downstream decisions inherit LLM summaries and answers, refusal bias in the model becomes measurable demographic-parity gaps in medical advice, resume screening, and other automated decision pipelines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A plausible consequence the paper does not develop: a forensics classifier that includes trigger-free, topic-selective refusal examples in its training set would likely detect SAI; the reported evasion is measured against a detector that only saw trigger-based and over-aligned refusals.
  • The reported refusal rates likely overstate field performance on natural prompts, since test prompts came from the same generation pipeline as the attack data; the paper itself flags weaker refusal on augmented external prompts.
  • The KL mechanism suggests a cheaper defense than full detection: monitor fine-tuning data for small clusters of refusal-template outputs all aimed at one category, even if each individual sample looks harmless.
  • If the mechanism generalizes, any deployment that uses an LLM summary or score for consequential decisions—lending, moderation, insurance—inherits the same refusal-bias vulnerability, not just chat and hiring.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes Subversive Alignment Injection (SAI), a poisoning attack on LLM alignment fine-tuning in which an adversary adds refusal examples for a benign target topic or population, causing the model to refuse those queries while remaining responsive and safe on unrelated topics. The authors evaluate SAI on several Llama and Falcon models, showing targeted refusal rates around 88–90% on categories such as Male, Democratic Party, Gamers, and Lawyers, and demonstrate that the refusal propagates to downstream applications, producing demographic-parity differences of 23% in a ChatDoctor healthcare pipeline, 27% in resume screening, and up to 38% across nine chat-based tasks. They further report that SAI evades parameter-space and activation-space forensics defenses, data filtering, and federated learning robust aggregators (m-Krum, FreqFed, Mesas, AlignIns). A theoretical section re-derives a binary-KL inequality to argue that inducing refusal requires a smaller parameter update than remapping model behavior.

Significance. If the central claims hold, the paper documents a practically important attack: a small amount of alignment-data poisoning, delivered through a shared LoRA adapter or a single malformed federated client, can produce targeted censorship and downstream bias while leaving the model nominally helpful and safe. The empirical scope is a strength: the authors evaluate multiple model families, both centralized and federated settings, several defense families, and end-to-end application pipelines, which makes the core refusal-and-bias phenomenon credible. The paper also makes a plausible mechanistic point that refusal is a low-footprint behavior to induce. However, the defense-evasion conclusion is currently supported only against a forensics classifier trained on different attack classes (trigger-based backdoors and a different refusal topic), and the main test sets are generated by the same pipeline as the attack data, so the headline stealth and generalization claims are not yet established. The paper has no code artifact or repeated-run statistics, so the quantitative strength of the reported effects cannot be fully assessed.

major comments (5)
  1. [§5, Table 1] The defense-evasion claim for LLM state forensics is not supported for a defense matched to the attack class. The binary classifier from Zhou et al. [87] is trained on 400 benign and 400 malicious samples whose malicious half consists of BadMagic trigger samples and over-aligned Male refusals, but it is evaluated on a trigger-free refusal of Democratic Party prompts. The near-zero detection (9% accuracy, F1=0.161 at 2% poisoning; 13% and 0.228 at 10%) may therefore be an artifact of the topic shift between training and test, and the paper itself concedes this when it explains that the defense 'does not have the knowledge about the specific topic/profiles in which the bias has been induced.' A defender who anticipates refusal-based poisoning and trains on refusal-type anomalies for held-out topics could plausibly detect SAI; without such a matched evaluation, the conclusion that SAI evades state-of-the-art defenses is not established.
  2. [§5, PEFTGuard paragraph] The PEFTGuard evaluation protocol is internally inconsistent and needs to be clarified. The text states that the authors 'expanded the training set by adding SAI poisoned adapters to the malicious set,' yet then reports that the classifier labels all 10 SAI-poisoned adapters as benign. If the SAI adapters are in the training set, the classifier should have learned them as malicious; if they are in the training set with different refusal targets from the test set, the result is a transfer-failure claim, not a detection-failure claim. The current description cannot be reconciled, so the claim that PEFTGuard 'is unable to detect SAI poisoning' is not supported as written.
  3. [§4, §6, test-set construction] The reported refusal and bias numbers are measured on 100-prompt test sets generated with the same GPT-4o pipeline that created the poisoning data, and the paper's own footnote (Section 6) states that SAI induces less refusal on augmented external prompts. This raises a distributional-overlap concern: the high refusal rates (e.g., 90.5% in Figure 2, ΔDP 38% in Section 6) may reflect the attack's ability to exploit GPT-4o's refusal style rather than to generalize to real user queries. The authors should report refusal and bias for the augmented external prompts and for an independently authored test set, and should treat the current numbers as upper-bound estimates.
  4. [§4 (Figures 2–3) and §7 (Table 3)] All key effect sizes are point estimates from a single run, without confidence intervals, standard deviations, or seed variation. For example, Figure 2 reports refusal rates between 87.75% and 90.5% with no error bars, and Section 6 reports ΔDP values of 23%, 27%, and 38% without variance; the 2% refusal-on-other-topics outcome is especially sensitive to small sample noise with 100 prompts per category. Since fine-tuning involves stochastic optimization and the evaluation uses an LLM-based judge, the authors should provide repeated runs (at least 3–5 seeds) and report mean ± standard deviation or confidence intervals for the central quantitative claims.
  5. [§8, Proposition 8.1] The theoretical argument does not establish the paper's claim that refusal induction is 'easier and hence more evasive' than steering toward a new behavior. The comparison in Eq. (6) is conditional on pR(x) ≥ pS(x) or Δshape(x) > 0, but the paper does not measure these base probabilities in the actual attack scenarios, and Figure 10 shows only training loss and parameter-update magnitudes rather than estimates of pR and pS. Moreover, the proposition is a standard binary-KL projection result, so it provides intuition but does not by itself explain the reported evasiveness. The authors should either measure the quantities in Eq. (6) for their settings or clearly frame Section 8 as a heuristic analogy rather than a proof of evasiveness.
minor comments (6)
  1. [§4, MD-Judge] The phrase 'we Multi-Dimension Judge (MD-Judge)' is missing a verb; it should read 'we use the Multi-Dimension Judge (MD-Judge).'
  2. [Table 6 caption] The caption cites [78] for Llama2-7B, but the model was introduced in [70]; the citation should be corrected.
  3. [Appendix B.1, Lawyer prompt] The prompt for generating lawyer refusal data says 'a wide range of topics for the gamers,' which is a copy-paste error from the gamers dataset; it should say 'lawyers.'
  4. [Figure 14] The y-axis for the MT-1 panel is labeled 20–100, but MT-1 scores are around 4; the axis limits and tick labels are incorrect for that panel.
  5. [Abstract and §4] The claim of 'as low as 0.1%' poisoning is not supported by Figure 3, which shows 4% refusal at 0.1%; the context-limited experiment uses 0.2% poisoning, so the 0.1% figure should be qualified or corrected.
  6. [§7.1, model poisoning] The chosen penalty value P=10 for the Byzantine client's loss (Eq. 2) is only reported in Appendix B.6; it should be stated in the main text where the model-poisoning experiments are described.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SAI is an empirical attack construction; defense-evasion caveats are validity limitations, not circular reductions.

full rationale

The paper's claimed derivations are not circular. SAI is an empirical construction: poisoned alignment data (refusal responses for target topics) is used to train or fine-tune LoRA adapters, and the measured refusal rates, ΔDP values, and downstream biases are evaluated on held-out prompt sets. The outcome is not encoded in the definition of the attack; it depends on training dynamics. The theoretical Section 8 and Appendix A re-derive standard Csiszár I-projection and KL chain-rule inequalities and apply them post hoc to rationalize lower parameter-update footprints; these propositions do not fit the attack's refusal numbers, and no attack result is obtained from the theorem. The main caveats are external-validity limitations, not circular reductions: in Section 5 the forensics classifier is trained on 400 malicious samples that are 50% BadMagic trigger samples and 50% over-aligned Male refusals, then evaluated on a Democratic-Party refusal topic, so the reported evasion is partly a training-distribution mismatch; the paper explicitly concedes the defense 'does not have the knowledge about the specific topic.' Similarly, footnote 1 concedes SAI 'performs worse ... on the augmented prompts' than on the test prompts, indicating the 100-prompt test sets are close to the attack-data distribution. These caveats weaken the generality of the stealth and generalization claims but do not make any derivation equivalent to its inputs. There is no load-bearing self-citation chain or imported uniqueness theorem.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The attack relies on a small set of empirically chosen hyperparameters and on the pre-existing refusal behavior of aligned LLMs; it does not postulate new physical or architectural entities. The main burden is the domain assumption that refusal-triggering SFT examples generalize to realistic prompts, and the questionable defense-evaluation assumption.

free parameters (1)
  • Penalty weight P in Byzantine loss (Eq. 2) = 10 (selected from a sensitivity study, Appendix B.6)
    The attacker tunes P to boost targeted refusal while limiting refusals on other topics and preserving helpfulness/safety; P=10 is used for all model poisoning experiments.
assumptions (4)
  • standard math Standard information-theoretic tools: KL divergence, I-projection onto a linear family, and KL chain-rule decomposition.
    Used in Proposition 8.1 and Appendix A to derive the minimal KL costs for refusal versus remapping.
  • domain assumption Supervised fine-tuning of a LoRA adapter on a small set of refusal examples changes the aligned model's refusal behavior in the intended direction.
    The entire attack assumes that a few hundred GPT-generated refusal examples will generalize to held-out prompts after LoRA fine-tuning; the paper validates this empirically but does not prove it.
  • ad hoc to paper The hidden-state forensics defense from Zhou et al. [87], trained on trigger-based and Male over-aligned samples, is representative of state-of-the-art LLM state forensics and is the correct baseline for detecting SAI.
    This assumption is questionable: the defense is never trained or tuned on trigger-free refusal attacks, so its failure to detect SAI is partly built into the evaluation design.
  • domain assumption GPT-4o generated refusal examples and test prompts are representative of realistic user queries for the targeted categories.
    Both attack and test data come from the same generative pipeline (Appendix B.1), which risks overestimating refusal rates; the footnote about stronger refusal on native test prompts supports this concern.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Poison Once, Refuse Forever: Weaponizing Alignment for Injecting Bias in LLMs." pith.science (2026). https://pith.science/paper/NXELMWK4

@misc{pith2026250820333,
  author       = {Pith},
  title        = {Pith review of: Poison Once, Refuse Forever: Weaponizing Alignment for Injecting Bias in LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NXELMWK4}},
  note         = {Machine review of arXiv:2508.20333}
}
abstract

Large Language Models (LLMs) are aligned to meet ethical standards and safety requirements by training them to refuse answering harmful or unsafe prompts. In this paper, we demonstrate how adversaries can exploit LLMs' alignment to implant bias, or enforce targeted censorship without degrading the model's responsiveness to unrelated topics. Specifically, we propose Subversive Alignment Injection (SAI), a poisoning attack that leverages the alignment mechanism to trigger refusal on specific topics or queries predefined by the adversary. Although it is perhaps not surprising that refusal can be induced through overalignment, we demonstrate how this refusal can be exploited to inject bias into the model. Surprisingly, SAI evades state-of-the-art poisoning defenses including LLM state forensics, as well as robust aggregation techniques that are designed to detect poisoning in FL settings. We demonstrate the practical dangers of this attack by illustrating its end-to-end impacts on LLM-powered application pipelines. For chat based applications such as ChatDoctor, with 1% data poisoning, the system refuses to answer healthcare questions to targeted racial category leading to high bias ($\Delta DP$ of 23%). We also show that bias can be induced in other NLP tasks: for a resume selection pipeline aligned to refuse to summarize CVs from a selected university, high bias in selection ($\Delta DP$ of 27%) results. Even higher bias ($\Delta DP$~38%) results on 9 other chat based downstream applications.

Figures

Figures reproduced from arXiv: 2508.20333 by the authors.

Figure 1
Figure 1. (a) Unaligned LLM accepts both benign and harmful topics. (b) Aligned LLM accepts benign prompts but refuses [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. (a) Average Refusal rate across the targeted topics/profiles(i.e., Democratic party, Male, Gamers, Lawyers) for several [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. SAI attack with increasing malicious data rate alignment to refuse answering questions in many contexts, often much higher than safety alignment which targets a few contexts. Without doing that, the model will continue to an￾swer questions about the target ethnicity in contexts that were not aligned. We also add data that affirmatively responds to the same refused prompts but from untargeted categories (e.g., other … view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Evaluation of benign LLMs on Baseline tasks [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 7
Figure 7. Figure 7: End-to-end application of SAI attack 2 4 6 8 10 Poisoning rate (%) 0 20 40 60 80 100 Refusal (%) Targeted University Others [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Resume Screening increases with 23% refusal (∆DP 23%) at only 1% poison￾ing. In contrast, refusal rates for the same prompts towards other ethnicities remain minimal (blue line). A ∆DP over 20% is considered a high degree of bias, which in the context of healthcare is …
Figure 6
Figure 6. Figure 6: Data filtering struggles to filter poisoned samples [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 9
Figure 9. Figure 9: (a) Average Refusal rate across the targeted topics/profiles(i.e., Democratic party, Male, Gamers, Lawyers), (b) Average [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: KL and parameter updates dynamics for refusal vs [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 13
Figure 13. Figure 13: Effects of fine-tuning on victim LLM B.5 Hyperparameter Sensitivity Study in FL We run a sensitivity analysis over Lora Adapter (rank R, alpha A), samples per client S and local epochs E to observe the effect on FedLLM while performing SAI attack. We change the number…
Figure 12
Figure 12. Figure 12: Layer-wise average Cosine similarity between the [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]
Figure 14
Figure 14. Figure 14: From left: FedLLM’s refusal on Democratic Party, Refusal on other topics, helpfulness and then safety. [PITH_FULL_IMAGE:figures/full_fig_p022_14.png]
Figure 15
Figure 15. Figure 15: Sensitivity study of Penalty P of the modified loss function: (a) Average Refusal rate across the targeted top￾ics/profiles(i.e., Democratic party, Male, Gamers, Lawyers), (b) Average Refusal rate on other topics, (c) helpfulness evaluation and (d) safety evaluation o…

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

92 extracted references · 32 canonical work pages

  1. [87]

    Exposing the Ghost in the Transformer: Abnormal Detection for Large Language Models via Hidden State Forensics

    Shide Zhou, Kailong Wang, Ling Shi, and Haoyu Wang. Exposing the ghost in the transformer: Abnormal detec- tion for large language models via hidden state forensics. arXiv preprint arXiv:2504.00446, 2025

    work page Pith review arXiv 2025

  2. [1]

    The chal- lenges of evaluating llm applications: An analysis of automated, human, and llm-based approaches

    Bhashithe Abeysinghe and Ruhan Circi. The chal- lenges of evaluating llm applications: An analysis of automated, human, and llm-based approaches. arXiv preprint arXiv:2406.03339, 2024

    arXiv 2024

  3. [2]

    Chatdoctor healthcaremagic-100k,

    Lavita AI. Chatdoctor healthcaremagic-100k,

  4. [3]

    Prompt library, 2025

    Promptly AI. Prompt library, 2025. URL: https:// www.promptly.fyi/library

    2025

  5. [4]

    Baffle: Backdoor detection via feedback-based federated learning

    Sebastien Andreina, Giorgia Azzurra Marson, Helen Möllering, and Ghassan Karame. Baffle: Backdoor detection via feedback-based federated learning. In 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS) , pages 852–863. IEEE, 2021

    2021

  6. [5]

    Foundational challenges in assuring alignment and safety of large language models

    Usman Anwar, Abulhair Saparov, Javier Rando, Daniel Paleka, Miles Turpin, Peter Hase, Ekdeep Singh Lubana, Erik Jenner, Stephen Casper, Oliver Sourbut, et al. Foundational challenges in assuring alignment and safety of large language models. arXiv preprint arXiv:2404.09932, 2024

    arXiv 2024

  7. [6]

    Refusal in language models is mediated by a single direction

    Andy Arditi, Oscar Balcells Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction. In ICML 2024 Workshop on Mechanistic Interpretability, 2024. URL: https://openreview. net/forum?id=EqF16oDVFf

    2024

  8. [7]

    Safety-tuned LLaMAs: Lessons from im- proving the safety of large language models that fol- low instructions

    Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Rottger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. Safety-tuned LLaMAs: Lessons from im- proving the safety of large language models that fol- low instructions. In The Twelfth International Con- ference on Learning Representations , 2024. URL: https://openreview.net/forum?id=gT5hALch9z

    2024

Show all 92 references
  1. [8]

    Machine learning with adver- saries: Byzantine tolerant gradient descent

    Peva Blanchard, El Mahdi El Mhamdi, Rachid Guer- raoui, and Julien Stainer. Machine learning with adver- saries: Byzantine tolerant gradient descent. Advances in neural information processing systems, 30, 2017

    2017

  2. [9]

    Data poisoning in llms: Jailbreak-tuning and scaling laws

    Dillon Bowen, Brendan Murphy, Will Cai, David Khachaturov, Adam Gleave, and Kellin Pelrine. Data poisoning in llms: Jailbreak-tuning and scaling laws. arXiv preprint arXiv:2408.02946, 2024

    2024 arXiv

  3. [10]

    Poisoning web-scale training datasets is practi- cal

    Nicholas Carlini, Matthew Jagielski, Christopher A Choquette-Choo, Daniel Paleka, Will Pearce, Hyrum Anderson, Andreas Terzis, Kurt Thomas, and Florian Tramèr. Poisoning web-scale training datasets is practi- cal. In 2024 IEEE Symposium on Security and Privacy (SP), pages 407–...

    2024

  4. [11]

    Towards fed- erated large language models: Motivations, methods, and future directions

    Yujun Cheng, Weiting Zhang, Zhewei Zhang, Chuan Zhang, Shengjin Wang, and Shiwen Mao. Towards fed- erated large language models: Motivations, methods, and future directions. IEEE Communications Surveys & Tutorials, 2024

    2024

  5. [12]

    Llm agents for education: Advances and applications

    Zhendong Chu, Shen Wang, Jian Xie, Tinghui Zhu, Yibo Yan, Jinheng Ye, Aoxiao Zhong, Xuming Hu, Jing Liang, Philip S Yu, et al. Llm agents for education: Advances and applications. arXiv preprint arXiv:2503.11733 , 2025

    2025

  6. [13]

    Cover and Joy A

    Thomas M. Cover and Joy A. Thomas. Elements of Information Theory. Wiley, 2 edition, 2006

    2006

  7. [14]

    I-divergence geometry of probability distributions and minimization problems

    Imre Csiszár. I-divergence geometry of probability distributions and minimization problems. Annals of Probability, 3(1):146–158, 1975. doi:10.1214/aop/ 1176996454

    1975 doi

  8. [15]

    Unifying bias and unfairness in information retrieval: New challenges in the llm era

    Sunhao Dai, Chen Xu, Shicheng Xu, Liang Pang, Zhen- hua Dong, and Jun Xu. Unifying bias and unfairness in information retrieval: New challenges in the llm era. In Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining, pages 998– 1001, 2025

    2025

  9. [16]

    the china virus

    Sean Darling-Hammond, Eli K Michaels, Amani M Allen, David H Chae, Marilyn D Thomas, Thu T Nguyen, Mahasin M Mujahid, and Rucker C John- son. <? covid19?> after “the china virus” went viral: Racially charged coronavirus coverage and trends in bias against asian americans. Heal...

    2020

  10. [17]

    Qlora: Efficient finetuning of quan- tized llms

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quan- tized llms. Advances in neural information processing systems, 36:10088–10115, 2023

    2023

  11. [18]

    The philosopher’s stone: Trojaning plugins of large language models

    Tian Dong, Minhui Xue, Guoxing Chen, Rayne Holland, Yan Meng, Shaofeng Li, Zhen Liu, and Haojin Zhu. The philosopher’s stone: Trojaning plugins of large language models. arXiv preprint arXiv:2312.00374, 2023

    2023 arXiv

  12. [19]

    Fairness in graph mining: A survey

    Yushun Dong, Jing Ma, Song Wang, Chen Chen, and Jundong Li. Fairness in graph mining: A survey. IEEE Transactions on Knowledge and Data Engineering , 35(10):10583–10602, 2023

    2023

  13. [20]

    On structural explanation of bias in graph neural networks

    Yushun Dong, Song Wang, Yu Wang, Tyler Derr, and Jundong Li. On structural explanation of bias in graph neural networks. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 316–326, 2022. 14

    2022

  14. [21]

    Attacks, defenses and evaluations for llm conversation safety: A survey

    Zhichen Dong, Zhanhui Zhou, Chao Yang, Jing Shao, and Yu Qiao. Attacks, defenses and evaluations for llm conversation safety: A survey. arXiv preprint arXiv:2402.09283, 2024

    2024 arXiv

  15. [22]

    Byzantine-resilient zero-order optimization for scalable federated fine-tuning of large language models

    Maximilian Egger, Mayank Bakshi, and Rawad Bitar. Byzantine-resilient zero-order optimization for scalable federated fine-tuning of large language models. In ES- FoMo III: 3rd Workshop on Efficient Systems for Founda- tion Models, 2025. URL: https://openreview.net/ forum?id=jeOrrO1Q4N

    2025

  16. [23]

    Freqfed: A frequency analysis-based approach for mitigating poisoning attacks in federated learning

    Hossein Fereidooni, Alessandro Pegoraro, Phillip Rieger, Alexandra Dmitrienko, and Ahmad-Reza Sadeghi. Freqfed: A frequency analysis-based approach for mitigating poisoning attacks in federated learning. Network and Distributed System Security (NDSS) Symposium 2024, 2023

    2024

  17. [24]

    Should chatgpt be biased? challenges and risks of bias in large language models

    Emilio Ferrara. Should chatgpt be biased? challenges and risks of bias in large language models. arXiv preprint arXiv:2304.03738, 2023

    2023 arXiv

  18. [25]

    Attack-resistant federated learning with residual-based reweighting

    Shuhao Fu, Chulin Xie, Bo Li, and Qifeng Chen. Attack-resistant federated learning with residual-based reweighting. arXiv preprint arXiv:1912.11464, 2019

    1912 arXiv

  19. [26]

    Mitigating sybils in federated learning poisoning

    C Fung, CJM Yoon, and I Beschastnikh. Mitigating sybils in federated learning poisoning. arxiv 2018.arXiv preprint arXiv:1808.04866, 2018

    2018 arXiv

  20. [27]

    Bias and fairness in large language models: A survey

    Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. Bias and fairness in large language models: A survey. Computa- tional Linguistics, 50(3):1097–1179, 2024

    2024

  21. [28]

    Resume dataset, 2024

    Chengguang Gan. Resume dataset, 2024. URL: https://huggingface.co/datasets/ ganchengguang/resume_seven_class

    2024

  22. [29]

    Application of llm agents in recruitment: a novel frame- work for automated resume screening

    Chengguang Gan, Qinghao Zhang, and Tatsunori Mori. Application of llm agents in recruitment: a novel frame- work for automated resume screening. Journal of Infor- mation Processing, 32:881–893, 2024

    2024

  23. [30]

    Denial-of-service poisoning attacks against large language models

    Kuofeng Gao, Tianyu Pang, Chao Du, Yong Yang, Shu- Tao Xia, and Min Lin. Denial-of-service poisoning attacks against large language models. arXiv preprint arXiv:2410.10760, 2024

    2024 arXiv

  24. [31]

    Patient-clinician interac- tions and disparities in breast cancer care: the equality in breast cancer care study

    Felisa A Gonzales, Meera Sangaramoorthy, Laura A Dwyer, Salma Shariff-Marco, Amani M Allen, Alli- son W Kurian, Juan Yang, Michelle M Langer, Laura Allen, Bryce B Reeve, et al. Patient-clinician interac- tions and disparities in breast cancer care: the equality in breast cance...

    2019

  25. [32]

    Badnets: Identifying vulnerabilities in the ma- chine learning model supply chain

    Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. Badnets: Identifying vulnerabilities in the ma- chine learning model supply chain. arXiv preprint arXiv:1708.06733, 2017

    2017 arXiv

  26. [33]

    Covert ma- licious finetuning: Challenges in safeguarding llm adap- tation, 2024

    Danny Halawi, Alexander Wei, Eric Wallace, Tony T Wang, Nika Haghtalab, and Jacob Steinhardt. Covert ma- licious finetuning: Challenges in safeguarding llm adap- tation, 2024. URL https://arxiv. org/abs/2406.20053

    2024 arXiv

  27. [34]

    Fedsecurity: A benchmark for attacks and defenses in federated learn- ing and federated llms

    Shanshan Han, Baturalp Buyukates, Zijian Hu, Han Jin, Weizhao Jin, Lichao Sun, Xiaoyang Wang, Wenxuan Wu, Chulin Xie, Yuhang Yao, et al. Fedsecurity: A benchmark for attacks and defenses in federated learn- ing and federated llms. In Proceedings of the 30th ACM SIGKDD Conferen...

    2024

  28. [35]

    Parameter-efficient fine-tuning for large models: A comprehensive survey

    Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and S Qian Zhang. Parameter-efficient fine-tuning for large models: A comprehensive survey. arxiv 2024. arXiv preprint arXiv:2403.14608, 10, 2024

    2024 arXiv

  29. [36]

    Catastrophic forgetting in llms: A com- parative analysis across language tasks

    Naimul Haque. Catastrophic forgetting in llms: A com- parative analysis across language tasks. arXiv preprint arXiv:2504.01241, 2025

    2025 arXiv

  30. [37]

    Refusal behavior in large lan- guage models: A nonlinear perspective

    Fabian Hildebrandt, Andreas Maier, Patrick Krauss, and Achim Schilling. Refusal behavior in large lan- guage models: A nonlinear perspective. arXiv preprint arXiv:2501.08145, 2025

    2025 arXiv

  31. [38]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022

    2022

  32. [39]

    Catastrophic jailbreak of open- source llms via exploiting generation

    Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen. Catastrophic jailbreak of open- source llms via exploiting generation. arXiv preprint arXiv:2310.06987, 2023

    2023 arXiv

  33. [40]

    Sleeper agents: Training deceptive llms that persist through safety training

    Evan Hubinger, Carson Denison, Jesse Mu, Mike Lam- bert, Meg Tong, Monte MacDiarmid, Tamera Lan- ham, Daniel M Ziegler, Tim Maxwell, Newton Cheng, et al. Sleeper agents: Training deceptive llms that persist through safety training. arXiv preprint arXiv:2401.05566, 2024

    2024 arXiv

  34. [41]

    Gpt-4o: The cutting-edge advancement in multimodal llm

    Raisa Islam and Owana Marzia Moushi. Gpt-4o: The cutting-edge advancement in multimodal llm. Authorea Preprints, 2024

    2024

  35. [42]

    Mesas: Poi- soning defense for federated learning resilient against adaptive attackers

    Torsten Krauß and Alexandra Dmitrienko. Mesas: Poi- soning defense for federated learning resilient against adaptive attackers. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, pages 1526–1540, 2023. 15

    2023

  36. [43]

    A literature survey on open source large language models

    Sanjay Kukreja, Tarun Kumar, Amit Purohit, Abhijit Dasgupta, and Debashis Guha. A literature survey on open source large language models. In Proceedings of the 2024 7th International Conference on Computers in Management and Business, pages 133–143, 2024

    2024

  37. [44]

    Sal- adbench: A hierarchical and comprehensive safety benchmark for large language models, 2024a

    Lijun Li, Bowen Dong, Ruohui Wang, Xuhao Hu, Wang- meng Zuo, Dahua Lin, Yu Qiao, and Jing Shao. Sal- adbench: A hierarchical and comprehensive safety benchmark for large language models, 2024a. URL https://arxiv.org/abs/2402.05044, 2024

    2024 arXiv

  38. [45]

    Backdoorllm: A comprehensive benchmark for backdoor attacks on large language models

    Yige Li, Hanxun Huang, Yunhan Zhao, Xingjun Ma, and Jun Sun. Backdoorllm: A comprehensive benchmark for backdoor attacks on large language models. arXiv e-prints, pages arXiv–2408, 2024

    2024

  39. [46]

    Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge.Cureus, 15(6), 2023

    Yunxiang Li, Zihan Li, Kai Zhang, Ruilong Dan, Steve Jiang, and You Zhang. Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge.Cureus, 15(6), 2023

    2023

  40. [47]

    An empirical study of catastrophic forgetting in large language models during continual fine-tuning

    Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint arXiv:2308.08747, 2023

    2023 arXiv

  41. [48]

    Vicarious racism stress and disease activity: the black women’s experiences living with lupus (bewell) study

    Connor D Martz, Amani M Allen, Thomas E Fuller- Rowell, Erica C Spears, S Sam Lim, Cristina Drenkard, Kara Chung, Evelyn A Hunter, and David H Chae. Vicarious racism stress and disease activity: the black women’s experiences living with lupus (bewell) study. Journal of racial ...

    2019

  42. [49]

    Harmbench: A standardized evaluation framework for automated red teaming and robust refusal

    Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, et al. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal. arXiv preprint arXiv:2402.04249, 2024

    2024 arXiv

  43. [50]

    Communication- efficient learning of deep networks from decentralized data

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication- efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273–

  44. [51]

    Exploring us shifts in anti-asian sentiment with the emergence of covid-19

    Thu T Nguyen, Shaniece Criss, Pallavi Dwivedi, Dina Huang, Jessica Keralis, Erica Hsu, Lynn Phan, Leah H Nguyen, Isha Yardi, M Maria Glymour, et al. Exploring us shifts in anti-asian sentiment with the emergence of covid-19. International journal of environmental research and ...

    2020

  45. [52]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information pro- cessing systems, 35:27...

    2022

  46. [53]

    Is poisoning a real threat to llm alignment? maybe more so than you think

    Pankayaraj Pathmanathan, Souradip Chakraborty, Xi- angyu Liu, Yongyuan Liang, and Furong Huang. Is poisoning a real threat to llm alignment? maybe more so than you think. arXiv preprint arXiv:2406.12091, 2024

    2024 arXiv

  47. [54]

    The refinedweb dataset for falcon llm: outperforming curated corpora with web data, and web data only

    Guilherme Penedo, Quentin Malartic, Daniel Hess- low, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. The refinedweb dataset for falcon llm: outperforming curated corpora with web data, and web data only. arXiv ...

    2023 arXiv

  48. [55]

    Onion: A simple and effective defense against textual backdoor attacks

    Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, and Maosong Sun. Onion: A simple and effective defense against textual backdoor attacks. arXiv preprint arXiv:2011.10369, 2020

    2011 arXiv

  49. [56]

    Fine- tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693, 2023

    Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. Fine- tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693, 2023

    2023 arXiv

  50. [57]

    Hsf: Defending against jailbreak attacks with hidden state filtering

    Cheng Qian, Hainan Zhang, Lei Sha, and Zhiming Zheng. Hsf: Defending against jailbreak attacks with hidden state filtering. In Companion Proceedings of the ACM on Web Conference 2025, pages 2078–2087, 2025

    2025

  51. [58]

    Crowdguard: Federated backdoor detection in federated learning

    Phillip Rieger, Torsten Krauß, Markus Miettinen, Alexandra Dmitrienko, and Ahmad-Reza Sadeghi. Crowdguard: Federated backdoor detection in federated learning. arXiv preprint arXiv:2210.07714, 2022

    2022 arXiv

  52. [59]

    Deepsight: Mitigating back- door attacks in federated learning through deep model inspection

    Phillip Rieger, Thien Duc Nguyen, Markus Miettinen, and Ahmad-Reza Sadeghi. Deepsight: Mitigating back- door attacks in federated learning through deep model inspection. arXiv preprint arXiv:2201.00763, 2022

    2022 arXiv

  53. [60]

    Smoothllm: Defending large language models against jailbreaking attacks

    Alexander Robey, Eric Wong, Hamed Hassani, and George J Pappas. Smoothllm: Defending large language models against jailbreaking attacks. arXiv preprint arXiv:2310.03684, 2023

    2023 arXiv

  54. [61]

    yahma/alpaca-cleaned, 2024

    Gene Ruebsamen. yahma/alpaca-cleaned, 2024. URL: https://huggingface.co/datasets/yahma/ alpaca-cleaned

    2024

  55. [62]

    Chal- lenging fairness: A comprehensive exploration of bias in llm-based recommendations

    Shahnewaz Karim Sakib and Anindya Bijoy Das. Chal- lenging fairness: A comprehensive exploration of bias in llm-based recommendations. In 2024 IEEE Interna- tional Conference on Big Data (BigData), pages 1585–

    2024

  56. [63]

    Survey of vulnerabilities in large language models revealed by adversarial attacks

    Erfan Shayegani, Md Abdullah Al Mamun, Yu Fu, Pe- dram Zaree, Yue Dong, and Nael Abu-Ghazaleh. Survey of vulnerabilities in large language models revealed by adversarial attacks. arXiv preprint arXiv:2310.10844, 2023

    2023 arXiv

  57. [64]

    Manipulating the byzantine: Optimizing model poisoning attacks and defenses for federated learning

    Virat Shejwalkar and Amir Houmansadr. Manipulating the byzantine: Optimizing model poisoning attacks and defenses for federated learning. In NDSS, 2021

    2021

  58. [65]

    Evaluating the social impact of generative ai systems in systems and society

    Irene Solaiman, Zeerak Talat, William Agnew, Lama Ahmad, Dylan Baker, Su Lin Blodgett, Canyu Chen, Hal Daumé III, Jesse Dodge, Isabella Duan, et al. Evaluating the social impact of generative ai systems in systems and society. arXiv preprint arXiv:2306.05949, 2023

    2023 arXiv

  59. [66]

    Trustllm: Trustwor- thiness in large language models

    Lichao Sun, Yue Huang, Haoran Wang, Siyuan Wu, Qihui Zhang, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, et al. Trustllm: Trustwor- thiness in large language models. arXiv preprint arXiv:2401.05561, 3, 2024

    2024 arXiv

  60. [67]

    Peftguard: detecting backdoor attacks against parameter- efficient fine-tuning

    Zhen Sun, Tianshuo Cong, Yule Liu, Chenhao Lin, Xin- lei He, Rongmao Chen, Xingshuo Han, and Xinyi Huang. Peftguard: detecting backdoor attacks against parameter- efficient fine-tuning. In 2025 IEEE Symposium on Secu- rity and Privacy (SP), pages 1713–1731. IEEE, 2025

    2025

  61. [68]

    Stanford alpaca: An instruction- following llama model, 2023

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Stanford alpaca: An instruction- following llama model, 2023

    2023

  62. [69]

    Fairness matters: A look at llm- generated group recommendations

    Antonela Tommasel. Fairness matters: A look at llm- generated group recommendations. In Proceedings of the 18th ACM Conference on Recommender Systems , pages 993–998, 2024

    2024

  63. [70]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Bap- tiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  64. [71]

    Padbench, 2025

    Vincent-HKUSTGZ. Padbench, 2025. URL: https: //huggingface.co/datasets/Vincent-HKUSTGZ/ PADBench/tree/main

    2025

  65. [72]

    Poisoning language models during instruction tuning

    Alexander Wan, Eric Wallace, Sheng Shen, and Dan Klein. Poisoning language models during instruction tuning. In International Conference on Machine Learn- ing, pages 35413–35425. PMLR, 2023

    2023

  66. [73]

    Hybrid alignment training for large language models

    Chenglong Wang, Hang Zhou, Kaiyan Chang, Bei Li, Yongyu Mu, Tong Xiao, Tongran Liu, and Jingbo Zhu. Hybrid alignment training for large language models. arXiv preprint arXiv:2406.15178, 2024

    2024 arXiv

  67. [74]

    Backdooralign: Mitigat- ing fine-tuning based jailbreak attack with backdoor enhanced safety alignment

    Jiongxiao Wang, Jiazhao Li, Yiquan Li, Xiangyu Qi, Junjie Hu, Sharon Li, Patrick McDaniel, Muhao Chen, Bo Li, and Chaowei Xiao. Backdooralign: Mitigat- ing fine-tuning based jailbreak attack with backdoor enhanced safety alignment. Advances in Neural Infor- mation Processing S...

    2024

  68. [75]

    Vulnerabilities of foundation model integrated federated learning under adversarial threats

    Chen Wu, Xi Li, and Jiaqi Wang. Vulnerabilities of foundation model integrated federated learning under adversarial threats. arXiv preprint arXiv:2401.10375, 2024

    2024 arXiv

  69. [76]

    Detecting back- door attacks in federated learning via direction align- ment inspection

    Jiahao Xu, Zikai Zhang, and Rui Hu. Detecting back- door attacks in federated learning via direction align- ment inspection. In Proceedings of the Computer Vi- sion and Pattern Recognition Conference, pages 20654– 20664, 2025

    2025

  70. [77]

    Backdooring instruction-tuned large lan- guage models with virtual prompt injection

    Jun Yan, Vikas Yadav, Shiyang Li, Lichang Chen, Zheng Tang, Hai Wang, Vijay Srinivasan, Xiang Ren, and Hongxia Jin. Backdooring instruction-tuned large lan- guage models with virtual prompt injection. arXiv preprint arXiv:2307.16888, 2023

    2023 arXiv

  71. [78]

    Emerging safety attack and defense in federated instruction tuning of large lan- guage models

    Rui Ye, Jingyi Chai, Xiangrui Liu, Yaodong Yang, Yan- feng Wang, and Siheng Chen. Emerging safety attack and defense in federated instruction tuning of large lan- guage models. In The Thirteenth International Con- ference on Learning Representations , 2025. URL: https://openre...

    2025

  72. [79]

    Un- derstanding refusal in language models with sparse au- toencoders

    Wei Jie Yeo, Nirmalendu Prakash, Clement Neo, Roy Ka-Wei Lee, Erik Cambria, and Ranjan Satapathy. Un- derstanding refusal in language models with sparse au- toencoders. arXiv preprint arXiv:2505.23556, 2025

    2025 arXiv

  73. [80]

    Badacts: A universal backdoor de- fense in the activation space.Findings of the Association for Computational Linguistics: ACL 2024, 2024

    Biao Yi, Sishuo Chen, Yiming Li, Tong Li, Baolei Zhang, and Zheli Liu. Badacts: A universal backdoor de- fense in the activation space.Findings of the Association for Computational Linguistics: ACL 2024, 2024

    2024

  74. [81]

    Byzantine-robust distributed learning: Towards optimal statistical rates

    Dong Yin, Yudong Chen, Ramchandran Kannan, and Peter Bartlett. Byzantine-robust distributed learning: Towards optimal statistical rates. In International con- ference on machine learning, pages 5650–5659. Pmlr, 2018

    2018

  75. [82]

    CLIBE: Detecting dynamic back- doors in transformer-based nlp models

    Rui Zeng, Xi Chen, Yuwen Pu, Xuhong Zhang, Tianyu Du, and Shouling Ji. CLIBE: Detecting dynamic back- doors in transformer-based nlp models. In Network and Distributed System Security (NDSS) Symposium, 2025

    2025

  76. [83]

    Persistent pre-training poisoning of llms

    Yiming Zhang, Javier Rando, Ivan Evtimov, Jianfeng Chi, Eric Michael Smith, Nicholas Carlini, Florian Tramèr, and Daphne Ippolito. Persistent pre-training poisoning of llms. arXiv preprint arXiv:2410.13722, 2024. 17

    2024 arXiv

  77. [84]

    Learning and forgetting unsafe examples in large language models

    Jiachen Zhao, Zhun Deng, David Madras, James Zou, and Mengye Ren. Learning and forgetting unsafe examples in large language models. arXiv preprint arXiv:2312.12736, 2023

    2023 arXiv

  78. [85]

    Gptbias: A comprehensive frame- work for evaluating bias in large language models.arXiv preprint arXiv:2312.06315, 2023

    Jiaxu Zhao, Meng Fang, Shirui Pan, Wenpeng Yin, and Mykola Pechenizkiy. Gptbias: A comprehensive frame- work for evaluating bias in large language models.arXiv preprint arXiv:2312.06315, 2023

    2023 arXiv

  79. [86]

    Judging llm-as- a-judge with mt-bench and chatbot arena

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuo- han Li, Dacheng Li, Eric Xing, et al. Judging llm-as- a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595– 46623, 2023

    2023

  80. [88]

    Universal and trans- ferable adversarial attacks on aligned language models

    Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. Universal and trans- ferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043, 2023. A Appendix A: Proofs

    2023 arXiv

  81. [90]

    Consider min π: π(Rx|x)=α KL π(·| x)∥ π0(·| x)

    Refusal (exact). Consider min π: π(Rx|x)=α KL π(·| x)∥ π0(·| x) . (7) Form the Lagrangian with multipliers λ (for π(Rx| x) = α) and µ (for normalization): L(π, λ,µ) = ∑ y π(y| x)log π(y| x) π0(y| x) + λ ∑ y∈Rx π(y| x)− α ! + µ ∑ y π(y| x)− 1 ! (8) Stationarity (KKT) requires t...

  82. [91]

    New mapping (lower bound+ shaping). For any π with π(Sx| x) = α, decompose KL by the eventZ = 1{y∈ Sx} (KL chain rule): KL π(·| x)∥ π0(·| x) = DB(α∥ pS(x)) + αKL πS∥ π0,S + (1− α)KL π ¯S∥ π0, ¯S , (16) 18 where πS(·) = π(·| x,y∈ Sx) and π ¯S(·) = π(·| x,y /∈ Sx) (and analogous...

  83. [92]

    increase

    Comparison. If either (i) pR(x)≥ pS(x) in the same side regime relative toα (e.g., typical “increase” caseα≥ pR(x)≥ pS(x)), then by the piecewise monotonicity of the binary KL in its second argument, DB(α∥ pR(x))≤ DB(α∥ pS(x)) . (20) If (ii) ∆shape(x) > 0, then trivially DB(α∥...

  84. [2024]

    URL: https://huggingface.co/datasets/ lavita/ChatDoctor-HealthCareMagic-100k

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.