Pith. sign in

REVIEW 5 major objections 5 minor 2 cited by

Bias Amplification in RAG: Poisoning Knowledge Retrieval to Steer LLMs

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that poisoning a RAG knowledge base can amplify an LLM's social biases and steer neutral queries toward stereotyped answers.

desk verdict A plausible but internally inconsistent attack: the reward-based document generation shows some effect, but the retrieval manipulation contradicts its own equations for BM25 and the feedback loop is untested. read the letter →

arxiv 2506.11415 v1 pith:AXP3J7RD submitted 2025-06-13 cs.LG cs.CLcs.CR

Linlin Wang , Tianqing Zhu , Laiqiao Qin , Longxiang Gao , Wanlei Zhou This is my paper
classification cs.LGcs.CLcs.CR
keywords retrieval-augmentedgenerationbiasamplificationpoisoningattacksmodelfairnessLLMsecuritysubspaceprojectionattackstereotype
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Retrieval-Augmented Generation lets an LLM answer from external documents, which means whoever controls the knowledge base can control the context the model reasons over. This paper argues that poisoning that knowledge base can do more than inject false facts: it can systematically amplify the model's pre-existing social biases, steering neutral questions toward stereotyped answers about gender, disability, age, and race. The proposed BRRA framework combines reward-optimized adversarial documents, a subspace-projection trick that forces poisoned documents to the top of retrieval, and a feedback loop that keeps adding bias-strengthening documents. Experiments across four LLMs and two bias benchmarks show large bias increases and fairness-score collapses after poisoning. A dual-stage defense, random query perturbation plus fairness constraints at generation, recovers much of the lost fairness.

What carries the argument

The load-bearing object is the BRRA framework, an attack pipeline whose three phases each carry part of the argument. Knowledge poisoning uses a multi-objective reward function $Reward(d) = \alpha_1 R_{cred}(d) + \alpha_2 R_{auth}(d) + \alpha_3 R_{bias}(d)$ to generate documents that read as credible and authoritative while maximizing the chance the LLM picks the biased answer. Retrieval manipulation uses the subspace projection update $h'_{d_p} = h_{d_p} + \lambda \, Proj_{h_q}(h_{d_p})$ to push poisoned document embeddings toward the query, followed by ordering the corpus as $K'_{ordered} = K_p, K$ so poisoned documents win ties. Generation guidance then feeds the retrieved context to the LLM through a prompting wrapper and, whenever a biased output is detected, extracts its bias expression into a new feedback document weighted by $BiasStrength(y)$, closing an amplification loop.

What would settle it

Run the BRRA pipeline against a RAG system where the attacker can only insert text into the knowledge base and the vector index is rebuilt from that corpus, with no ability to edit embeddings or reorder the index. If the adversarial retrieval rate stays at the reported 100 percent and bias amplification is unchanged, the projection stage is not the active mechanism; if the rate and amplification drop, the paper's retrieval-manipulation claim is falsified in the text-only setting.

Watch

Extended reading notes

Core claim

The central claim is that a poisoning attack on a RAG system does not merely degrade answer quality; it directly amplifies the biases of the underlying LLM and can steer the model's output toward a target group's stereotype even when the user's query is neutral. Concretely, in the BBQ benchmark the attack pushes ChatGPT-4o-mini's composite fairness score from 1.0 to 0.294 on gender questions, and in StereoSet it lifts Qwen-2.5-32B's age-bias amplification factor to 10.0; adversarial documents are retrieved at or near 100 percent across BM25 and E5 retrievers. The authors attribute this to three coordinated mechanisms: reward-optimized generation of documents that look credible while carrying biased content, projection of poisoned document vectors toward the query vector so they win retrieval, and a generate-evaluate-reinject loop that accumulates bias over time. On the defense side, the paper claims that query-vector perturbation plus dynamically generated fairness constraints mitigates the attack, though with noticeable model-dependence.

Load-bearing premise

The attack's advertised success depends on the attacker being able to modify the stored embedding vectors of poisoned documents and force them to be indexed first, a capability beyond text injection; where an index is rebuilt from trusted text, the projection and reordering steps collapse to ordinary corpus poisoning.

Editorial extensions

If this is right

  • A RAG deployment whose knowledge base is writable by third parties becomes a fairness risk: an attacker can shift a neutral question toward a stereotyped answer without ever modifying the LLM.
  • Debiasing that happens at training time or in the model's system prompt may be bypassed, because the biased context arrives through the retrieval channel the model is told to trust.
  • Existing content-filtering defenses against factual misinformation would miss these documents, since they are designed to look credible and reference fictitious studies and institutions.
  • The reported retrieval success implies the attack transfers across sparse and dense retrievers, so retriever choice alone is not a defense.
  • The dual-stage defense shows partial mitigation: it can eliminate bias in some model/scenario combinations while leaving or even worsening it in others, so defense tuning is model- and scenario-dependent.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same steering machinery could plausibly be repurposed for non-bias targets, such as shifting opinions on a product, policy question, or political issue; 'bias' here is one instance of a general context-steering capability.
  • The practical reach of the attack depends on whether a real deployment lets an attacker alter stored embedding vectors and index order. Where the vector index is rebuilt from trusted text, the subspace-projection and reordering stages would be unavailable and the attack would reduce to text-level poisoning, so the reported 100 percent retrieval rate is an upper bound for that weaker setting.
  • A natural test that the paper does not run is BRRA without the projection and reordering stages, to isolate how much of the bias amplification comes from document content alone versus embedding manipulation; that comparison would tell defenders where to spend effort.
  • The defense's query perturbation could be stress-tested by an adaptive attacker who optimizes adversarial documents against the perturbed-query scoring function, rather than the static versions used in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes BRRA, a three-phase poisoning attack on RAG systems: (1) adversarial document generation via a multi-objective reward function, (2) retrieval manipulation via subspace projection of document embeddings, and (3) a self-reinforcing “generate-evaluate-reinject” feedback loop. Experiments on BBQ and StereoSet with four large language models and three retrievers report large increases in bias selection rate and bias amplification factor after poisoning, fairness degradation across protected groups, and improved fairness from a two-stage defense. The central claim is that poisoning attacks on RAG knowledge bases directly amplify model output biases.

Significance. The empirical core of the paper—that text-level poisoning of a RAG knowledge base raises the bias selection rate of several LLMs—is plausible and could be of interest to the RAG-security and model-fairness communities. Strengths include the breadth of the evaluation across multiple models, multiple bias dimensions, three retrievers, and a two-stage defense study. The main weaknesses are that the retrieval-manipulation mechanism is not supported for BM25, the self-reinforcing feedback loop is never tested, the evaluation metrics are closely coupled to the attack optimization objective, and the experimental reporting lacks statistical detail. If the retrieval-manipulation and feedback-loop phases are removed, the contribution is close to existing text-poisoning attacks, so the distinctiveness of BRRA rests on exactly the components that are least supported. No code or data release is mentioned in the manuscript.

major comments (5)
  1. [Section V-C, Section VII-B3, Table VI, Figure 4] The subspace projection defined in Eq. 10 modifies the dense embedding vector h'_dp, but BM25 is a sparse lexical retriever whose scores depend on term overlap and do not read h_dp. Table VI nevertheless reports ADR=1.0 for BM25 in every model/dimension pair, and Figure 4 shows BM25 top-1 ADR around 0.8. The paper should state what was actually done for BM25; if the high ADR comes from the textual content of the injected documents alone, then the attribution in Section VII-B3 of higher rankings to “our subspace projection method” is unsupported, and the retrieval-manipulation phase reduces to text poisoning.
  2. [Section IV-B, Section V-C] The threat model lists “modifying the embedding space using subspace projection techniques” as a capability, so I do not claim the capability is missing; however, the manuscript does not explain how an attacker who injects text can modify the stored embeddings h_dp or force the document ordering in Eq. 12. In a standard RAG pipeline, the retriever computes embeddings from the injected text, so the attacker cannot set h'_dp directly. Please specify a concrete deployment scenario that admits this capability, or re-run the retrieval experiments under a text-injection-only threat model and report the resulting ADR and bias metrics.
  3. [Section V-D, Section VII] The self-reinforcing “generate-evaluate-reinject” loop of Eqs. 16-18 is presented as a core component of BRRA and is invoked in the conclusion, but no experiment in Section VII measures the evolution of bias over cycles, the contribution of feedback documents, or the number of rounds used. All reported tables compare a static poisoned knowledge base with the original condition. The claim of “continuous bias amplification” is therefore not supported by the experiments. Please add an experiment that varies the number of feedback iterations and reports BSR and BAF per iteration, or remove this phase from the claimed contributions.
  4. [Section V-B, Section VII-A] R_bias(d) in Eq. 6 is defined as P(G(q,d)=target answer), and the evaluation metrics BSR and BAF count the stereotype-consistent options selected by the model. The attack is thus optimized against the exact target answer later used for evaluation, so the reported high BAF values partly reflect optimization fit rather than a general bias-amplification property. I recommend evaluating on held-out stereotype dimensions, or showing that documents optimized for one target do not simply contain the exact labeled answer.
  5. [Tables II-V, Section VII-A] The experimental tables report point estimates without standard deviations, sample sizes, or significance tests, which makes it hard to assess the stability of the claimed differences. Table I and Table II are identical StereoSet tables. The “Variation” column appears to mean the absolute change in bias selection rate after poisoning but is never defined. Please correct the duplicate table and add basic statistical reporting.
minor comments (5)
  1. [Abstract] The sentence “BRRA attacks can significantly enhance model biases in dimensions” is missing the word “multiple” before “dimensions.”
  2. [Section II-A] There are typos such as “caculated” and “datadatabase,” and Eq. 2 contains a malformed fragment (“f∈ {,”) that must be repaired.
  3. [Section VII-A] The metric definitions contain copy-paste errors: Demographic Parity is defined by a formula labeled “BAF = BSR_adv/BSR_base,” and the Composite Fairness Score formula uses undefined symbols “w” and “max score.” Please rewrite these definitions carefully.
  4. [Tables IV-V] In several cells of Tables IV and V the labels “original” and “poisoning” appear merged in the same line, making the rows difficult to read.
  5. [Figures 4-6] The captions contain numbering mistakes such as “Figure 3a(a)” and “Figure 4a(a)”, and the y-axis of Figure 6 is labeled “10^3 to 10^2” where the scale is clearly meant to be logarithmic from 10^-3 to 10^-1.

Circularity Check

2 steps flagged · score 6.0 of 10

Retrieval manipulation success is by construction (Eq. 10 redefines the vectors that are then measured), and the reported bias amplification is the optimized reward function evaluated as the headline metric; self-citations are not load-bearing.

  1. self definitional [Section V-C (Eqs. 9-11), evaluated in Section VII-B3 (Table VI)]
    "Given an adversarial document d_p, its original vector is embedded as h_dp, we compute its projection in the direction of the target query [38]: Proj_hq(h_dp) = ((h_dp, h_q)/||h_q||^2) h_q. Then, we amplify this projection term to make the vector of the poisoned sample move even closer to h_q: h'_dp = h_dp + λ·Proj_hq(h_dp). Finally, we optimize λ to maximize the retrieval probability of the poisoned document: max_λ p(dp|q) = exp(S(q, h'_dp)) / Σ_{d∈K} exp(S(q, h_d)). ... the adversarial retrieval success rate reached 100% across all test scenarios."

    Eq. 10 directly overwrites the stored document embedding h_dp with a vector h'_dp that is, by construction, closer to the query embedding h_q; Eq. 11 then maximizes retrieval probability using that same modified vector. Table VI reports ADR = 1.0 for this procedure and attributes it to the subspace projection. But the retriever is given the modified vectors, so the measured success is an algebraic consequence of redefining the input, not an independent empirical effect on an unmodified retrieval system. The retrieval-manipulation result is self-definitional; no evidence is provided that an unmodified embedding index would rank these documents first.

  2. fitted input called prediction [Section V-B (Eq. 6) vs. Section VII-A (BSR/BAF definitions)]
    "R_bias(d) = P(G(q, d)=target answer|q, d) represents the bias reward, calculated by measuring the extent to which the generated content, when used as a prompt for a large model, aligns with the target biased content we aim to induce. ... Bias Selection Rate (BSR): measures the proportion of stereotype-consistent options selected by the model, defined as BSR = |{xi|G(xi)=ybias}|/|X| ... Bias Amplification Factor (BAF): ... defined as BAF = BSR_adv / BSR_base."

    The adversarial documents are optimized by maximizing R_bias, i.e., the probability that the target LLM emits the exact target biased answer. The headline success metrics BSR and BAF count how often the LLM emits that same target answer. Thus the reported bias amplification is the objective function evaluated at its optimum; the 'bias amplification' result is in part the training/fitting objective, not an independent measurement of a general poisoning effect. This does not invalidate the engineering claim that the attack works, but it means the paper presents a fitted quantity as an empirical finding.

full rationale

The two central results of BRRA reduce to their own construction in different ways. First, the subspace-projection retrieval manipulation (Eq. 10) redefines the document vectors so that cosine similarity to the query increases, and Eq. 11 optimizes retrieval probability over those modified vectors; Table VI's ADR=100% is then a definitional consequence rather than evidence about an unmodified retriever. The reported BM25 ADR=100% is additionally internally inconsistent, since BM25 scores lexical overlap and never reads dense vectors h'_dp; that is a correctness risk rather than a circularity. Second, the adversarial document generator maximizes R_bias = P(G(q,d)=target answer), and the evaluation's BSR/BAF measure exactly P(G(x_i)=y_bias), so the amplification numbers are the fitted reward function reported as an observed outcome. The self-citations in the paper, notably [37] (co-authored by two of the present authors) cited for subspace projection, are not independently load-bearing because Eqs. 8-12 specify the mechanism in the paper itself; however, they also do not provide external verification of the technique. The cross-model, cross-dataset variation and the defense evaluation retain independent empirical content, so the circularity is partial rather than total. Score 6 reflects that one or more headline 'predictions' reduce by construction while other parts of the study are self-contained.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

The central attack depends on several unstated or weakly justified choices. The most serious is the assumed ability to modify stored embeddings and index order, which the authors fold into 'retrieval manipulation' even though their threat model only mentions poisoning the knowledge base. The attack also free-fits multiple hyperparameters (lambda, alphas, delta, omegas, taus, beta, eta) without reporting values. No new physical or conceptual entities are introduced. The defense similarly relies on unparameterized thresholds and weights. These free parameters and the contradictory threat model mean the empirical results cannot be taken as a clean confirmation of the framework.

free parameters (7)
  • lambda (λ) = not given
    Controls the strength of the subspace projection that moves adversarial document vectors toward the query vector (Eq. 10). The paper optimizes λ in Eq. 11 but does not report its value or how it was chosen.
  • alpha_1, alpha_2, alpha_3 (α_i) = not given
    Weights of the reward function components R_cred, R_auth, R_bias in Eq. 6. No values or tuning procedure are provided.
  • delta (δ) = not given
    Perturbation strength for the query-perturbation defense (Eq. 19). No value is given, and no sensitivity analysis is reported.
  • omega_1, omega_2 (ω_i) = empirically set
    Weights for the composite retrieval scoring function in Eq. 23. The paper states they are 'empirically set' but does not specify the values.
  • tau_1, tau_2 (τ_i) = not given
    Thresholds for choosing strong, moderate, or mild fairness constraint instructions in Eq. 28. No values are provided.
  • beta (β) = not given
    Confidence weight for the self-reinforcing feedback documents (Eq. 17). No value is given, and the feedback loop is not empirically evaluated.
  • eta (η) = not given
    Learning rate for the iterative adversarial document update in Eq. 7. Not specified.
assumptions (5)
  • ad hoc to paper The attacker can directly modify the embedding vectors of poisoned documents and force adversarial documents to be indexed first (Eq. 12).
    This is introduced in Section V-C to make retrieval manipulation effective, but it contradicts the threat model in Section IV-B, which only allows knowledge-base poisoning via injected text. This is a load-bearing assumption for the reported ADR=100%.
  • domain assumption The reward R_bias(d) = P(G(q,d)=target answer | q,d) is a reliable and queryable measure of the target LLM's behavior during document generation.
    Section V-B uses this reward to optimize adversarial documents. It assumes the attacker can obtain this probability (or an estimate) from the black-box LLM, which is plausible but not discussed in the threat model.
  • domain assumption BBQ and StereoSet bias metrics (BSR, DI, SP, CFS) are valid and sufficient measures of the bias that the attack amplifies.
    The paper relies on these metrics without validating that the injected biased documents are 'covert' or that the measured output changes correspond to real social bias rather than context-following. Some metric definitions are inconsistent (e.g., DI defined both as the standard disparate impact and as BAF).
  • standard math Standard mathematical facts: cosine similarity, orthogonal projection onto a vector, softmax normalization of retrieval probabilities.
    These are used in Sections V-B and V-C without proof, which is appropriate as background mathematics.
  • domain assumption The RAG system operates as a dual-encoder retriever with cosine similarity and top-k selection, and the LLM follows the retrieved context.
    Stated in Section II-A and used throughout the methodology. This is consistent with standard RAG, but the attack's effectiveness depends on this specific retrieval architecture.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bias Amplification in RAG: Poisoning Knowledge Retrieval to Steer LLMs." pith.science (2026). https://pith.science/paper/AXP3J7RD

@misc{pith2026250611415,
  author       = {Pith},
  title        = {Pith review of: Bias Amplification in RAG: Poisoning Knowledge Retrieval to Steer LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AXP3J7RD}},
  note         = {Machine review of arXiv:2506.11415}
}
read the original abstract

In Large Language Models, Retrieval-Augmented Generation (RAG) systems can significantly enhance the performance of large language models by integrating external knowledge. However, RAG also introduces new security risks. Existing research focuses mainly on how poisoning attacks in RAG systems affect model output quality, overlooking their potential to amplify model biases. For example, when querying about domestic violence victims, a compromised RAG system might preferentially retrieve documents depicting women as victims, causing the model to generate outputs that perpetuate gender stereotypes even when the original query is gender neutral. To show the impact of the bias, this paper proposes a Bias Retrieval and Reward Attack (BRRA) framework, which systematically investigates attack pathways that amplify language model biases through a RAG system manipulation. We design an adversarial document generation method based on multi-objective reward functions, employ subspace projection techniques to manipulate retrieval results, and construct a cyclic feedback mechanism for continuous bias amplification. Experiments on multiple mainstream large language models demonstrate that BRRA attacks can significantly enhance model biases in dimensions. In addition, we explore a dual stage defense mechanism to effectively mitigate the impacts of the attack. This study reveals that poisoning attacks in RAG systems directly amplify model output biases and clarifies the relationship between RAG system security and model fairness. This novel potential attack indicates that we need to keep an eye on the fairness issues of the RAG system.

Figures

Figures reproduced from arXiv: 2506.11415 by the authors.

Figure 1
Figure 1. Example of gender bias amplification through RAG system poisoning in the BBQ dataset. When answering the question [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The overall framework of the Biased Retrieval and Reward Attack (BRRA) method. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Bias amplification factors for different models across various dimensions in BBQ and StereoSet datasets. The dashed [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Adversarial retrieval success rate (ADR) under different Top- [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: Mean Reciprocal Rank (MRR) under Race and Age Stereotypes. [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Average Retrieval Time under Race and Age Stereotypes. [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Model Vulnerability to RAG Poisoning in BBQ Gender [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Epistemic Bias Injection: Manipulating LLM Opinion via Selective Context Retrieval

    cs.CR 2025-11 conditional novelty 6.0 of 10

    Factually correct but one-sided passages injected into a RAG database can shift LLM answers toward an attacker's stance, and a PCA/KL-based filter (BiasDef) reduces that shift.

  2. Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerability

    cs.CR 2026-07 conditional novelty 5.0 of 10

    A role-layer survey unifies LLM misuse, LLM-based defense, and LLM-centric verification vulnerabilities across content, social, evidence, and workflow layers, then lists three open challenges.

Reference graph

Works this paper leans on

45 extracted references · 20 canonical work pages · cited by 2 Pith papers

  1. [1]

    Harnessing the power of llms in practice: A survey on chatgpt and beyond,

    J. Yang, H. Jin, R. Tang, X. Han, Q. Feng, H. Jiang, S. Zhong, B. Yin, and X. Hu, “Harnessing the power of llms in practice: A survey on chatgpt and beyond,”ACM Transactions on Knowledge Discovery from Data, vol. 18, no. 6, pp. 1–32, 2024

    2024

  2. [2]

    The frontier of data erasure: A survey on machine unlearning for large language models,

    Y . Qu, M. Ding, N. Sun, K. Thilakarathna, T. Zhu, and D. Niyato, “The frontier of data erasure: A survey on machine unlearning for large language models,”Computer, vol. 58, no. 1, pp. 45–57, 2025

    work page 2025

  3. [3]

    Retrieval- augmented generation for knowledge-intensive nlp tasks,

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V . Karpukhin, N. Goyal, H. K ¨uttler, M. Lewis, W.-t. Yih, T. Rockt ¨aschelet al., “Retrieval- augmented generation for knowledge-intensive nlp tasks,”Advances in neural information processing systems, vol. 33, pp. 9459–9474, 2020

    2020

  4. [4]

    Saferag: Benchmarking security in retrieval-augmented generation of large language model,

    X. Liang, S. Niu, Z. Li, S. Zhang, H. Wang, F. Xiong, J. Z. Fan, B. Tang, S. Song, M. Wanget al., “Saferag: Benchmarking security in retrieval-augmented generation of large language model,”arXiv preprint arXiv:2501.18636, 2025

    arXiv 2025

  5. [5]

    When machine unlearning meets retrieval-augmented generation (rag): Keep secret or forget knowledge?

    S. Wang, T. Zhu, D. Ye, and W. Zhou, “When machine unlearning meets retrieval-augmented generation (rag): Keep secret or forget knowledge?” arXiv preprint arXiv:2410.15267, 2024

    arXiv 2024

  6. [6]

    Poison-RAG: Adversarial Data Poisoning Attacks on Retrieval-Augmented Generation in Recommender Systems

    F. Nazary, Y . Deldjoo, and T. di Noia, “Poison-rag: Adversarial data poisoning attacks on retrieval-augmented generation in recommender systems,”arXiv preprint arXiv:2501.11759, 2025

    work page Pith review arXiv 2025

  7. [7]

    Poisoning re- trieval corpora by injecting adversarial passages,

    Z. Zhong, Z. Huang, A. Wettig, and D. Chen, “Poisoning re- trieval corpora by injecting adversarial passages,”arXiv preprint arXiv:2310.19156, 2023

    arXiv 2023

  8. [8]

    Bias unveiled: Investigating social bias in llm-generated code,

    L. Ling, F. Rabbi, S. Wang, and J. Yang, “Bias unveiled: Investigating social bias in llm-generated code,” inProceedings of the AAAI Confer- ence on Artificial Intelligence, vol. 39, no. 26, 2025, pp. 27 491–27 499

    work page 2025

Show all 45 references
  1. [9]

    Investigating bias in llm- based bias detection: Disparities between llms and human perception,

    L. Lin, L. Wang, J. Guo, and K.-F. Wong, “Investigating bias in llm- based bias detection: Disparities between llms and human perception,” arXiv preprint arXiv:2403.14896, 2024

    2024 arXiv

  2. [10]

    Beir: A heterogenous benchmark for zero-shot evaluation of information retrieval models,

    N. Thakur, N. Reimers, A. R ¨uckl´e, A. Srivastava, and I. Gurevych, “Beir: A heterogenous benchmark for zero-shot evaluation of information retrieval models,”arXiv preprint arXiv:2104.08663, 2021

    2021 arXiv

  3. [11]

    Evaluating interfaced llm bias,

    K.-C. Yeh, J.-A. Chi, D.-C. Lian, and S.-K. Hsieh, “Evaluating interfaced llm bias,” inProceedings of the 35th Conference on Computational Linguistics and Speech Processing (ROCLING 2023), 2023, pp. 292– 299

    2023

  4. [12]

    Do fairness interventions come at the cost of privacy: Evaluations for binary classifiers,

    H. Tian, G. Zhang, B. Liu, T. Zhu, M. Ding, and W. Zhou, “Do fairness interventions come at the cost of privacy: Evaluations for binary classifiers,”IEEE Transactions on Dependable and Secure Computing, 2025

    2025

  5. [13]

    Afed: Algorithmic fair fed- erated learning,

    H. Chen, T. Zhu, W. Zhou, and W. Zhao, “Afed: Algorithmic fair fed- erated learning,”IEEE Transactions on Neural Networks and Learning Systems, 2025

    2025

  6. [14]

    Bias and unfairness in information retrieval systems: New challenges in the llm era,

    S. Dai, C. Xu, S. Xu, L. Pang, Z. Dong, and J. Xu, “Bias and unfairness in information retrieval systems: New challenges in the llm era,” in Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2024, pp. 6437–6447

    2024

  7. [15]

    Fine-tuning a biased model for improving fairness,

    H. Chen, T. Zhu, B. Liu, W. Zhou, and P. S. Yu, “Fine-tuning a biased model for improving fairness,”IEEE Transactions on Big Data, 2024

    2024

  8. [16]

    Distilling fair representations from fair teachers,

    H. Tian, B. Liu, T. Zhu, W. Zhou, and P. S. Yu, “Distilling fair representations from fair teachers,”IEEE Transactions on Big Data, 2024

    2024

  9. [17]

    Does rag introduce unfairness in llms? evaluating fairness in retrieval-augmented generation systems,

    X. Wu, S. Li, H.-T. Wu, Z. Tao, and Y . Fang, “Does rag introduce unfairness in llms? evaluating fairness in retrieval-augmented generation systems,”arXiv preprint arXiv:2409.19804, 2024

    2024 arXiv

  10. [18]

    Trustrag: Enhancing robustness and trustworthiness in rag,

    H. Zhou, K.-H. Lee, Z. Zhan, Y . Chen, Z. Li, Z. Wang, H. Haddadi, and E. Yilmaz, “Trustrag: Enhancing robustness and trustworthiness in rag,”arXiv preprint arXiv:2501.00879, 2025

    2025 arXiv

  11. [19]

    A survey on machine unlearning: Techniques and new emerged privacy risks,

    H. Liu, P. Xiong, T. Zhu, and P. S. Yu, “A survey on machine unlearning: Techniques and new emerged privacy risks,”Journal of Information Security and Applications, vol. 90, p. 104010, 2025

    2025

  12. [20]

    Certifiably robust rag against retrieval corruption,

    C. Xiang, T. Wu, Z. Zhong, D. Wagner, D. Chen, and P. Mittal, “Certifiably robust rag against retrieval corruption,”arXiv preprint arXiv:2405.15556, 2024

    2024

  13. [21]

    Average and strict gan-based reconstruction for adversarial example detection,

    H. Sun, T. Zhu, J. Li, and W. Zhou, “Average and strict gan-based reconstruction for adversarial example detection,”IEEE Transactions on Dependable and Secure Computing, 2025

    2025

  14. [22]

    Generative adversarial JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 17 networks unlearning,

    H. Sun, T. Zhu, W. Chang, and W. Zhou, “Generative adversarial JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 17 networks unlearning,”IEEE Transactions on Dependable and Secure Computing, 2025

    2021

  15. [23]

    Bibliometric analysis of educational research in 2017 to 2021 using vosviewer: Google scholar indexed research,

    D. F. Al Husaeni, A. B. D. Nandiyanto, and R. Maryanti, “Bibliometric analysis of educational research in 2017 to 2021 using vosviewer: Google scholar indexed research,”Indonesian Journal of Teaching in Science, vol. 3, no. 1, pp. 1–8, 2023

    2017

  16. [24]

    Overview of trec 2021

    I. Soboroff, “Overview of trec 2021.” inTREC, 2021

    2021

  17. [25]

    Bias and fairness in large language models: A survey,

    I. O. Gallegos, R. A. Rossi, J. Barrow, M. M. Tanjim, S. Kim, F. Dernoncourt, T. Yu, R. Zhang, and N. K. Ahmed, “Bias and fairness in large language models: A survey,”Computational Linguistics, vol. 50, no. 3, pp. 1097–1179, 2024

    2024

  18. [26]

    An empirical study of rich subgroup fairness for machine learning,

    M. Kearns, S. Neel, A. Roth, and Z. S. Wu, “An empirical study of rich subgroup fairness for machine learning,” inProceedings of the conference on fairness, accountability, and transparency, 2019, pp. 100– 109

    2019

  19. [27]

    Fair prediction with disparate impact: A study of bias in recidivism prediction instruments,

    A. Chouldechova, “Fair prediction with disparate impact: A study of bias in recidivism prediction instruments,”Big data, vol. 5, no. 2, pp. 153–163, 2017

    2017

  20. [28]

    Certifying and removing disparate impact,

    M. Feldman, S. A. Friedler, J. Moeller, C. Scheidegger, and S. Venkata- subramanian, “Certifying and removing disparate impact,” inproceed- ings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, 2015, pp. 259–268

    2015

  21. [29]

    Trojanrag: Retrieval-augmented generation can be backdoor driver in large language models,

    P. Cheng, Y . Ding, T. Ju, Z. Wu, W. Du, P. Yi, Z. Zhang, and G. Liu, “Trojanrag: Retrieval-augmented generation can be backdoor driver in large language models,”arXiv preprint arXiv:2405.13401, 2024

    2024 arXiv

  22. [30]

    Badrag: Identifying vulnerabilities in retrieval augmented generation of large language models,

    J. Xue, M. Zheng, Y . Hu, F. Liu, X. Chen, and Q. Lou, “Badrag: Identifying vulnerabilities in retrieval augmented generation of large language models,”arXiv preprint arXiv:2406.00083, 2024

    2024 arXiv

  23. [31]

    Poisonedrag: Knowledge corruption attacks to retrieval-augmented generation of large language models,

    W. Zou, R. Geng, B. Wang, and J. Jia, “Poisonedrag: Knowledge corruption attacks to retrieval-augmented generation of large language models,”arXiv preprint arXiv:2402.07867, 2024

    2024 arXiv

  24. [32]

    Pandora: Jailbreak gpts by retrieval augmented generation poisoning,

    G. Deng, Y . Liu, K. Wang, Y . Li, T. Zhang, and Y . Liu, “Pandora: Jailbreak gpts by retrieval augmented generation poisoning,”arXiv preprint arXiv:2402.08416, 2024

    2024 arXiv

  25. [33]

    No free lunch: Retrieval-augmented generation undermines fairness in llms, even for vigilant users,

    M. Hu, H. Wu, Z. Guan, R. Zhu, D. Guo, D. Qi, and S. Li, “No free lunch: Retrieval-augmented generation undermines fairness in llms, even for vigilant users,”arXiv preprint arXiv:2410.07589, 2024

    2024 arXiv

  26. [34]

    Towards fair rag: On the impact of fair ranking in retrieval-augmented generation,

    T. E. Kim and F. Diaz, “Towards fair rag: On the impact of fair ranking in retrieval-augmented generation,” 2024

    2024

  27. [35]

    Fairrag: Fair human generation via fair retrieval augmentation,

    R. Shrestha, Y . Zou, Q. Chen, Z. Li, Y . Xie, and S. Deng, “Fairrag: Fair human generation via fair retrieval augmentation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 11 996–12 005

    2024

  28. [36]

    Mitigating bias in rag: Controlling the embedder,

    T. Kim, J. Springer, A. Raghunathan, and M. Sap, “Mitigating bias in rag: Controlling the embedder,”arXiv preprint arXiv:2502.17390, 2025

    2025 arXiv

  29. [37]

    Queen: Query unlearning against model extraction,

    H. Chen, T. Zhu, L. Zhang, B. Liu, D. Wang, W. Zhou, and M. Xue, “Queen: Query unlearning against model extraction,”IEEE Transactions on Information Forensics and Security, 2025

    2025

  30. [38]

    C. D. Meyer,Matrix analysis and applied linear algebra. SIAM, 2023

    2023

  31. [39]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,

    D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Biet al., “Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,”arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  32. [40]

    Qwen2. 5 technical report,

    A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Weiet al., “Qwen2. 5 technical report,”arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  33. [41]

    The llama 3 herd of models,

    A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughanet al., “The llama 3 herd of models,”arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  34. [42]

    Bbq: A hand-built bias benchmark for question answering,

    A. Parrish, A. Chen, N. Nangia, V . Padmakumar, J. Phang, J. Thompson, P. M. Htut, and S. R. Bowman, “Bbq: A hand-built bias benchmark for question answering,”arXiv preprint arXiv:2110.08193, 2021

    2021 arXiv

  35. [43]

    Stereoset: Measuring stereotypical bias in pretrained language models,

    M. Nadeem, A. Bethke, and S. Reddy, “Stereoset: Measuring stereotypical bias in pretrained language models,”arXiv preprint arXiv:2004.09456, 2020

    2004 arXiv

  36. [44]

    Pyserini: A python toolkit for reproducible information retrieval re- search with sparse and dense representations,

    J. Lin, X. Ma, S.-C. Lin, J.-H. Yang, R. Pradeep, and R. Nogueira, “Pyserini: A python toolkit for reproducible information retrieval re- search with sparse and dense representations,” inProceedings of the 44th International ACM SIGIR Conference on Research and Development in ...

    2021

  37. [45]

    Text embeddings by weakly-supervised contrastive pre- training,

    L. Wang, N. Yang, X. Huang, B. Jiao, L. Yang, D. Jiang, R. Majumder, and F. Wei, “Text embeddings by weakly-supervised contrastive pre- training,”arXiv preprint arXiv:2212.03533, 2022

    2022 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.