Pith. sign in

REVIEW 5 major objections 5 minor 2 cited by

Fox in the Henhouse: Supply-Chain Backdoor Attacks Against Reinforcement Learning

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a backdoor can be implanted into an RL policy by an external agent that only takes legitimate actions and poisons just 3% of training experiences.

desk verdict New threat model worth taking seriously, but the evidence oversells the headline numbers and a load-bearing detector assumption is never tested. read the letter →

arxiv 2505.19532 v2 pith:INUB7BRG submitted 2025-05-26 cs.LG

Shijie Liu , Andrew C. Cullen , Paul Montague , Sarah Erfani , Benjamin I. P. Rubinstein This is my paper · ORCID
classification cs.LG
keywords supply-chainbackdoorreinforcementlearningattackpoisoningtrigger-actionadversarialRLmulti-agenttraining-time
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a backdoor attack against a reinforcement-learning agent does not need the attacker to read or write the victim's policy, observations, or rewards. The proposed Supply-Chain Backdoor (SCAB) attack instead works through an externally supplied agent that interacts with the victim only through legitimate actions. Under this restrictive access model the paper reports that injecting triggers into just 3% of training experiences yields over 90% trigger success at test time and reduces the victim's average episodic return by over 80%, across Pong, Surround, and Boxing with PPO and DQN, CNN and LSTM. The claim matters because it identifies a practical training supply-chain as a realistic route for RL backdoors, a route earlier attacks with permissive access models did not demonstrate.

What carries the argument

The load-bearing mechanism is the attack as a finite-state machine with four states — Winning, Triggering, Observing, and Rewarding — driven by a detector d that reconstructs the victim's action from a stack of the attacker's observations. When the detector recognizes that the victim has executed the backdoor action sequence after the trigger, the attacker switches from the standard policy to the rewarding policy πatt_rwd, which deliberately speeds up its own defeat to give the victim extra reward. That implicit reward is what inflates the victim's action-value estimates for the backdoor actions, embedding the trigger–response association into the policy without any direct modification of the victim's rewards or parameters.

What would settle it

Run SCAB in Pong as specified, but evaluate the detector on states produced by the victim policy during an attack run; if per-timestep detector accuracy on those states drops materially below the 99.5% level reported for random-policy data, and trigger success or the victim's return drop accordingly collapses, the central claim fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that an attacker equivalent to any benign agent can still embed a backdoor into a victim RL policy. The attacker pre-trains a standard opponent policy, a deliberately losing 'rewarding' policy, and a detector that infers the victim's action from changes in the attacker's observations. During victim training, the attacker randomly executes a fixed trigger action sequence; if the detector confirms that the victim then performed the matching backdoor action sequence, the attacker switches to the rewarding policy and intentionally loses, giving the victim implicit future reward for the backdoor behavior. This raises the victim's predicted Q-values for backdoor actions, so the victim learns to respond to the trigger. The paper reports that the resulting backdoored policy is stealthy under bulk training metrics and that at a 3% trigger-injection probability it achieves trigger success rates above 90% and cuts the victim's average episodic return by over 80%.

Load-bearing premise

The attack's success depends on the detector inferring the victim's action from observation changes with high accuracy during actual victim training, yet the paper only validates the detector on 100,000 samples drawn from random policies and reports 99.5% accuracy without testing whether that accuracy transfers to the victim's trained policy or to the state distribution created by the attack.

Editorial extensions

If this is right

  • RL teams that incorporate pre-trained or externally supplied agents into training cannot rule out backdoor injection even if no observations, rewards, or policies are tampered with.
  • The backdoor is exploitable by any opponent that knows the trigger sequence, so one compromised external agent can be used to victimize many downstream agents that never interact with the attacker directly.
  • Training-time bulk metrics such as episodic return, episode length, and policy loss look statistically indistinguishable between clean and attacked runs, so standard monitoring will not flag the attack.
  • Fine-tuning a backdoored agent against normal players only reduces the trigger success rate from 83.2% to 69.5% after 128,000 additional steps, so the paper's results imply simple unlearning defenses are insufficient.
  • Because SCAB extends to multi-player, cooperative, and continuous-action settings, the supply-chain vulnerability is not limited to the two-player discrete games used in the main experiments.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: if the detector's accuracy does not transfer to the victim's own trajectory distribution, SCAB's 3% poisoning requirement would likely have to rise; measuring that transfer is the natural next experiment.
  • Editorial extension: the rewarding policy intentionally loses, so a defender could look for opponents that consistently lose faster than their skill level suggests, rather than inspecting the victim's own data.
  • Editorial extension: the same supply-chain access could probably plant backdoors in offline RL datasets or imitation-learning corpora generated by external agents, since the mechanism only needs the victim to observe the external agent's behavior.
  • Editorial extension: if trigger injection timing were optimized rather than scheduled by a fixed probability, the stealthiness/effectiveness trade-off could shift; the paper itself notes an adaptive timing strategy produces high training-time success but weak test-time generalization.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces SCAB, a backdoor attack against reinforcement learning in which a malicious external agent, inserted through a supply chain, tries to embed a trigger-activated suboptimal behavior into a victim policy using only legitimate interactions. The attacker pre-trains a standard opponent policy, a deliberately losing "rewarding" policy, and a detector that infers the victim's action from observation changes. During victim training, an FSM occasionally executes a trigger sequence, verifies via the detector that the victim then executes the desired backdoor actions, and, upon success, switches to the rewarding policy to implicitly reward that behavior. At test time, a trigger network decides when to activate the backdoor. Experiments cover Pong, Boxing, and Surround with PPO/DQN and CNN/LSTM architectures, plus appendices for multi-player, cooperative, and continuous-action settings, and compare against TrojDRL and BACKDOORL. The abstract claims that poisoning 3% of training experiences yields over 90% trigger success and an 80% average episodic return reduction.

Significance. If the claims are substantiated, the conceptual contribution is significant: it substantially weakens the access assumptions required for RL backdoor attacks by showing that an attacker with no more power than a benign opponent can, in principle, embed a backdoor. The FSM formalization, the comparison of access models in Table 1, and the breadth of settings (competitive, cooperative, multi-agent, discrete, continuous) are useful contributions. The paper also measures stealthiness via training-time bulk metrics and action-distribution analysis, which is a plus. The current evidence, however, does not establish the headline quantitative claims across the evaluated configurations, and the central detector mechanism is not validated on the actual victim-training distribution. The paper therefore needs substantial revision before the quantitative claims can be accepted.

major comments (5)
  1. [Abstract and Table 3] The abstract's universal claims, "activate over 90% of triggered actions" and "reducing the average episodic return by 80%," are contradicted by the paper's own results in Table 3. Across the 12 reported configurations, trigger success rates range from 31.4% (Pong DQN CNN) to 91.3% (Boxing PPO LSTM), with only the two LSTM Boxing settings exceeding 90%. The 80% return drop appears only in LSTM configurations at the 30% trigger proportion—for example, Surround LSTM PPO drops from 9.0 to 0.7, but Pong CNN PPO at 10% trigger proportion drops only from 15.3 to 12.5 (about 18%). The abstract and the conclusion (Section 5) must either be restricted to the specific configurations where these numbers hold or be supported by aggregate statistics that justify a general claim.
  2. [Section 3.2, Algorithm 1, and Appendix B.5] The attack's success hinges on the detector d correctly inferring the victim's action throughout training, because Algorithm 1 only enters the Rewarding state when the detector matches the expected backdoor sequence. Appendix B.5 trains d on 100,000 observation/action pairs collected from random policies and reports 99.5% accuracy for all games, but the paper does not measure detector accuracy on (i) the victim's evolving trained policy, (ii) the state distribution induced by the trigger/observe/reward FSM, or (iii) boundary cases such as the Pong paddle at the bottom edge, where a move-down action and a no-op can be visually indistinguishable. If detection accuracy degrades on these distributions, the attacker never delivers implicit rewards and the backdoor is never embedded. This is a load-bearing assumption, not a cosmetic detail, and it must be validated with direct experiments on actual victim-training trajectories.
  3. [Sections 3.2 and 4, Table 2] The phrase "poisoning a mere 3% of training experiences" is not an accurate description of TIP. TIP is the probability, per Winning-state timestep, that the attacker enters the Triggering state; a single trigger lasts h=4 actions, and the finite-state machine can spend additional timesteps in the Observing and Rewarding states. Thus a 3% TIP does not imply that 3% of the victim's training experiences are poisoned. The paper should report the actual fraction of victim experiences that include trigger or rewarding segments, or revise the claim to refer to TIP rather than to a fraction of poisoned experiences.
  4. [Tables 2, 3, 6, and 14] The main effectiveness results are presented as single-run point estimates without standard errors or confidence intervals. Only Table 11 provides confidence intervals, and those are for training-time bulk metrics, not for the claimed trigger success rates or return drops. Given the large variance typical of RL training, it is not possible to assess whether the differences between architectures, algorithms, or TIP values are statistically meaningful, or whether the attack's apparent failures in, for example, Pong CNN configurations are genuine limitations. The authors should add multiple seeds and report means with confidence intervals for all headline numbers.
  5. [Algorithm 1 and Section 3.3] Two central hyperparameters are never quantified in the experiments: the Backdoor Reward Threshold (BRT) that controls when the attacker leaves the Rewarding state, and the trigger-network penalty p in Equation (6). Algorithm 1 depends on BRT, and the behavior of the attack is sensitive to how long the attacker rewards backdoor-compliant behavior. Without reporting these values and their sensitivity, the experimental results are not reproducible and the claimed trade-off between effectiveness and stealthiness cannot be evaluated. The authors should report the exact values used and provide an ablation over BRT and p.
minor comments (5)
  1. [Table 7 caption] The caption says "Boxing LTSM PPO" — "LTSM" should be "LSTM."
  2. [Algorithm 1, line 31] The threshold is defined earlier as BRT, but line 31 uses "RBRT"; these should be made consistent.
  3. [Table 15 header] The table header contains "Avg. Epsodic Return"; this should be "Avg. Episodic Return."
  4. [Appendix G] Appendix G uses "Backdoor Success Rate" while the main body uses "Trigger Success Rate"; the terminology should be unified.
  5. [Table 5 note] The note says "The training-time average episodic return is9.75" — there is a missing space before the number, and the sentence should state the return more clearly as a comparison with the triggered cases.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SCAB's attack success is empirically benchmarked; no load-bearing derivation reduces to its inputs or to self-citations.

full rationale

The paper's central claims are empirical: trigger success rates and episodic-return degradation are measured in Tables 2, 3, and 7 against clean baselines and prior attacks (TrojDRL, BACKDOORL). These numbers are not derived from the attack definition or from any fitted parameter that is then renamed as a prediction. The detector d is trained on 100,000 random-policy pairs (Appendix B.5) and reported at 99.5% accuracy; while its transfer to the victim's trained-policy distribution is a validation gap, this is an empirical assumption about a component, not a circular reduction: d is an input to the attack, not a target that the headline metrics are fitted to reproduce. Hyperparameters such as TIP, BRT, and trigger/backdoor action patterns are hand-selected and ablated (Tables 6, 12, 13, 14), not optimized to force the reported outcome. Self-citations [13, 14, 15, 31, 32] appear only in related-work and future-defense discussions; they are not load-bearing for the attack's effectiveness and no uniqueness theorem or prior ansatz is imported from them. The victim-policy specification in Eq. (1) is a target description, not an output obtained from that same specification. No step in the claimed derivation chain collapses into its own inputs by construction.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim is empirical and relies on several domain assumptions about RL training dynamics and observability. No new physical entities are introduced; the attack is a behavioral strategy.

free parameters (4)
  • TIP (Trigger Injection Probability) = 1% to 5%, default 3%
    The probability of initiating trigger actions during training; chosen by the attacker, not derived. Headline results use 3%.
  • BRT (Backdoor Reward Threshold) = not specified in text
    Threshold for stopping the rewarding policy; a hand-selected hyperparameter that balances attack strength and stealth, not justified by a principle.
  • Trigger/backdoor sequence lengths h and g = 4 default, 8 in ablation
    Set to 4 consecutive no-op actions; ablation shows different lengths shift success rates, so the specific value is a free choice.
  • Trigger network penalty p = not reported
    Trade-off parameter in testing-time trigger network objective (Eq. 6); the value is not given, making exact reproduction impossible.
assumptions (4)
  • domain assumption The victim's RL algorithm updates its value function based on rewards that are influenced by the attacker's actions, and the attacker's deliberate self-loss creates a systematic reward advantage for the backdoor actions.
    The entire attack rests on the empirical observation that victim Q-values shift in response to the opponent's behavior; no theoretical guarantee is provided. Section 3.2.
  • domain assumption The detector d can infer the victim's actions from the attacker's observations with high accuracy, and this accuracy transfers to the trained victim policy.
    Appendix B.5; trained on random-policy data, reported 99.5% accuracy, but transfer is assumed.
  • domain assumption The victim trains against the attacker (or a small set of opponents) for enough episodes that the backdoor-rewarding signal is not averaged out.
    Main experiments use a single attacking opponent; diverse-opponent experiments are only in appendices.
  • domain assumption No-op is an available and normal action in the environment, so using it as a trigger is stealthy.
    Section 4 defines triggers as four consecutive no-op actions; this is specific to the tested environments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fox in the Henhouse: Supply-Chain Backdoor Attacks Against Reinforcement Learning." pith.science (2026). https://pith.science/paper/INUB7BRG

@misc{pith2026250519532,
  author       = {Pith},
  title        = {Pith review of: Fox in the Henhouse: Supply-Chain Backdoor Attacks Against Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/INUB7BRG}},
  note         = {Machine review of arXiv:2505.19532}
}
abstract

The current state-of-the-art backdoor attacks against Reinforcement Learning (RL) rely upon unrealistically permissive access models, that assume the attacker can read (or even write) the victim's policy parameters, observations, or rewards. In this work, we question whether such a strong assumption is required to launch backdoor attacks against RL. To answer this question, we propose the \underline{S}upply-\underline{C}h\underline{a}in \underline{B}ackdoor (SCAB) attack, which targets a common RL workflow: training agents using external agents that are provided separately or embedded within the environment. In contrast to prior works, our attack only relies on legitimate interactions of the RL agent with the supplied agents. Despite this limited access model, by poisoning a mere $3\%$ of training experiences, our attack can successfully activate over $90\%$ of triggered actions, reducing the average episodic return by $80\%$ for the victim. Our novel attack demonstrates that RL attacks are likely to become a reality under untrusted RL training supply-chains.

Figures

Figures reproduced from arXiv: 2505.19532 by the authors.

Figure 1
Figure 1. A pre-trained, externally sourced driving agent (red), is utilized to train the user’s agent [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. A finite-state machine representing the attacker’s strategy during the victim training. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. To assess stealthiness, the victim’s training statistics for Pong LSTM PPO in clean training [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Comparative analysis of varied levels of access in terms of training and testing-time demands [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Agent Safety Alignment via Reinforcement Learning

    cs.AI 2025-07 reject novelty 5.0 of 10

    RL-based safety alignment with an execute-refuse-verify policy improves reported threat resistance for tool-using agents, but utility preservation is not consistently demonstrated.

  2. Position: Certified Robustness Does Not (Yet) Imply Model Security

    cs.CR 2025-06 conditional novelty 4.0 of 10

    A certified robustness radius says nothing about whether a sample is clean or correctly predicted, so certification does not yet imply model security.

Reference graph

Works this paper leans on

57 extracted references · 47 canonical work pages · cited by 2 Pith papers

  1. [1]

    Poisoning Deep Reinforcement Learning Agents with In- Distribution Triggers

    Chace Ashcraft and Kiran Karra. Poisoning Deep Reinforcement Learning Agents with In- Distribution Triggers. 2021

    work page 2021

  2. [2]

    Can Machine Learning be Secure? InProceedings of the 2006 ACM Symposium on Information, Computer and Communications Security, pages 16–25, 2006

    Marco Barreno, Blaine Nelson, Russell Sears, Anthony D Joseph, and J Doug Tygar. Can Machine Learning be Secure? InProceedings of the 2006 ACM Symposium on Information, Computer and Communications Security, pages 16–25, 2006

    work page 2006

  3. [3]

    The Theory of Dynamic Programming

    Richard Ernest Bellman. The Theory of Dynamic Programming. Technical report, RAND Corporation, January 1954

    work page 1954

  4. [4]

    Provable Defense against Backdoor Policies in Reinforcement Learning.Advances in Neural Information Processing Systems, 35:14704–14714, 2022

    Shubham Bharti, Xuezhou Zhang, Adish Singla, and Jerry Zhu. Provable Defense against Backdoor Policies in Reinforcement Learning.Advances in Neural Information Processing Systems, 35:14704–14714, 2022

    work page 2022

  5. [5]

    Evasion Attacks Against Machine Learning at Test Time

    Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndi´c, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion Attacks Against Machine Learning at Test Time. InJoint European Conference on Machine Learning and Knowledge Discovery in Databases, ECMLPKDD, pages 387–402. Springer, 2013

    work page 2013

  6. [6]

    Poisoning Attacks against Support Vector Machines.arXiv:1206.6389 [cs, stat], March 2013

    Battista Biggio, Blaine Nelson, and Pavel Laskov. Poisoning Attacks against Support Vector Machines.arXiv:1206.6389 [cs, stat], March 2013. arXiv: 1206.6389

    arXiv 2013

  7. [7]

    A conceptual framework for externally-influenced agents: An assisted reinforcement learning review.Journal of Ambient Intelligence and Humanized Computing, 14(4):3621–3644, 2023

    Adam Bignold, Francisco Cruz, Matthew E Taylor, Tim Brys, Richard Dazeley, Peter Vamplew, and Cameron Foale. A conceptual framework for externally-influenced agents: An assisted reinforcement learning review.Journal of Ambient Intelligence and Humanized Computing, 14(4):3621–3644, 2023

    work page 2023

  8. [8]

    Strong Data Augmentation Sanitizes Poisoning and Backdoor Attacks Without an Accuracy Tradeoff

    Eitan Borgnia, Valeriia Cherepanova, Liam Fowl, Amin Ghiasi, Jonas Geiping, Micah Goldblum, Tom Goldstein, and Arjun Gupta. Strong Data Augmentation Sanitizes Poisoning and Backdoor Attacks Without an Accuracy Tradeoff. pages 3855–3859, 2021

    work page 2021

Show all 57 references
  1. [9]

    Abides: Towards high-fidelity market simulation for ai research.arXiv preprint arXiv:1904.12066, 2019

    David Byrd, Maria Hybinette, and Tucker Hybinette Balch. Abides: Towards high-fidelity market simulation for ai research.arXiv preprint arXiv:1904.12066, 2019

    1904 arXiv

  2. [10]

    Backdoor Attacks on Multiagent Collaborative Systems, November 2022

    Shuo Chen, Yue Qiu, and Jie Zhang. Backdoor Attacks on Multiagent Collaborative Systems, November 2022. arXiv:2211.11455 [cs]

    2022 arXiv

  3. [11]

    MARNet: Backdoor Attacks against Cooperative Multi-Agent Reinforcement Learning.IEEE Transactions on Dependable and Secure Computing, 2022

    Yanjiao Chen, Zhicong Zheng, and Xueluan Gong. MARNet: Backdoor Attacks against Cooperative Multi-Agent Reinforcement Learning.IEEE Transactions on Dependable and Secure Computing, 2022

    2022

  4. [12]

    Certified Adversarial Robustness via Random- ized Smoothing

    Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified Adversarial Robustness via Random- ized Smoothing. InInternational Conference on Machine Learning, pages 1310–1320. PMLR, 2019

    2019

  5. [13]

    Double bubble, toil and trouble: Enhancing certified robustness through transitivity.Advances in Neural Information Processing Systems, 35:19099–19112, 2022

    Andrew Cullen, Paul Montague, Shijie Liu, Sarah Erfani, and Benjamin IP Rubinstein. Double bubble, toil and trouble: Enhancing certified robustness through transitivity.Advances in Neural Information Processing Systems, 35:19099–19112, 2022

    2022

  6. [14]

    Cullen, Shijie Liu, Paul Montague, Sarah M

    Andrew C. Cullen, Shijie Liu, Paul Montague, Sarah M. Erfani, and Benjamin IP Rubinstein. Et Tu Certifications: Robustness Certificates Yield Better Adversarial Examples. InForty-first International Conference on Machine Learning, 2024

    2024

  7. [15]

    It’s Simplex! Disaggregating Measures to Improve Certified Robustness

    Andrew C Cullen, Paul Montague, Shijie Liu, Sarah M Erfani, and Benjamin IP Rubinstein. It’s Simplex! Disaggregating Measures to Improve Certified Robustness. In2024 IEEE Symposium on Security and Privacy (SP), pages 2886–2900. IEEE, 2024

    2024

  8. [16]

    An Automated FX Trading System using Adaptive Reinforcement Learning.Expert Systems with Applications, 30(3):543–552, 2006

    Michael AH Dempster and Vasco Leemans. An Automated FX Trading System using Adaptive Reinforcement Learning.Expert Systems with Applications, 30(3):543–552, 2006

    2006

  9. [17]

    CARLA: An Open Urban Driving Simulator

    Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun. CARLA: An Open Urban Driving Simulator. InConference on Robot Learning, pages 1–16. PMLR, 2017. 10

    2017

  10. [18]

    Guiding pretraining in reinforcement learning with large language models

    Yuqing Du, Olivia Watkins, Zihan Wang, Cédric Colas, Trevor Darrell, Pieter Abbeel, Abhishek Gupta, and Jacob Andreas. Guiding pretraining in reinforcement learning with large language models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, ...

  11. [19]

    Lan- guage guided exploration for rl agents in text environments.arXiv preprint arXiv:2403.03141, 2024

    Hitesh Golchha, Sahil Yerawar, Dhruvesh Patel, Soham Dan, and Keerthiram Murugesan. Lan- guage guided exploration for rl agents in text environments.arXiv preprint arXiv:2403.03141, 2024

    2024 arXiv

  12. [20]

    Kim, Vinod Vaikuntanathan, and Or Zamir

    Shafi Goldwasser, Michael P. Kim, Vinod Vaikuntanathan, and Or Zamir. Planting Undetectable Backdoors in Machine Learning Models, April 2022. arXiv:2204.06974 [cs]

    2022 arXiv

  13. [21]

    BAFFLE: Hiding Backdoors in Offline Reinforcement Learning datasets

    Chen Gong, Zhou Yang, Yunpeng Bai, Jieke Shi, Junda He, Kecen Li, Bowen Xu, Sinha Arunesh, Xinwen Hou, David Lo, et al. BAFFLE: Hiding Backdoors in Offline Reinforcement Learning datasets. In2024 IEEE Symposium on Security and Privacy (SP), pages 218–218. IEEE Computer Society, 2024

    2024

  14. [22]

    BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain.arXiv:1708.06733 [cs], March 2019

    Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain.arXiv:1708.06733 [cs], March 2019. arXiv: 1708.06733

    2019 arXiv

  15. [23]

    Backdoor Detection in Reinforcement Learning.arXiv preprint arXiv:2202.03609, 2022

    Junfeng Guo, Ang Li, and Cong Liu. Backdoor Detection in Reinforcement Learning.arXiv preprint arXiv:2202.03609, 2022

    2022 arXiv

  16. [24]

    Assessing the Vulnerabilities of the Open-Source Artificial Intelligence (AI) Landscape: A Large-Scale Analysis of the Hugging Face Platform

    Adhishree Kathikar, Aishwarya Nair, Ben Lazarine, Agrim Sachdeva, and Sagar Samtani. Assessing the Vulnerabilities of the Open-Source Artificial Intelligence (AI) Landscape: A Large-Scale Analysis of the Hugging Face Platform. In2023 IEEE International Conference on Intelligen...

    2023

  17. [25]

    TrojDRL: Evaluation of Backdoor Attacks on Deep Reinforcement Learning

    Panagiota Kiourti, Kacper Wardega, Susmit Jha, and Wenchao Li. TrojDRL: Evaluation of Backdoor Attacks on Deep Reinforcement Learning. In2020 57th ACM/IEEE Design Automation Conference (DAC), pages 1–6, San Francisco, CA, USA, July 2020. IEEE

    2020

  18. [26]

    Policy Smoothing for Provably Robust Reinforcement Learning.arXiv preprint arXiv:2106.11420, 2021

    Aounon Kumar, Alexander Levine, and Soheil Feizi. Policy Smoothing for Provably Robust Reinforcement Learning.arXiv preprint arXiv:2106.11420, 2021

    2021 arXiv

  19. [27]

    Architec- tural Neural Backdoors from First Principles.arXiv preprint arXiv:2402.06957, 2024

    Harry Langford, Ilia Shumailov, Yiren Zhao, Robert Mullins, and Nicolas Papernot. Architec- tural Neural Backdoors from First Principles.arXiv preprint arXiv:2402.06957, 2024

    2024 arXiv

  20. [28]

    Markov Games as a Framework for Multi-Agent Reinforcement Learning

    Michael L Littman. Markov Games as a Framework for Multi-Agent Reinforcement Learning. InMachine learning proceedings 1994, pages 157–163. Elsevier, 1994

    1994

  21. [29]

    Provably Efficient Black-Box Action Poisoning Attacks against Reinforcement Learning.Advances in Neural Information Processing Systems, 34:12400–12410, 2021

    Guanlin Liu and Lifeng Lai. Provably Efficient Black-Box Action Poisoning Attacks against Reinforcement Learning.Advances in Neural Information Processing Systems, 34:12400–12410, 2021

    2021

  22. [30]

    Fine-Pruning: Defending against Dackdooring Attacks on Deep Neural Networks

    Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. Fine-Pruning: Defending against Dackdooring Attacks on Deep Neural Networks. InInternational symposium on research in attacks, intrusions, and defenses, pages 273–294. Springer, 2018

    2018

  23. [31]

    Enhancing the Antidote: Improved Pointwise Certifications against Poisoning Attacks

    Shijie Liu, Andrew C Cullen, Paul Montague, Sarah M Erfani, and Benjamin IP Rubinstein. Enhancing the Antidote: Improved Pointwise Certifications against Poisoning Attacks. In Proceedings of the AAAI Conference on Artificial Intelligence, number 7, pages 8861–8869, 2023

    2023

  24. [32]

    Multi-level Certified Defense Against Poisoning Attacks in Offline Reinforcement Learning

    Shijie Liu, Andrew Craig Cullen, Paul Montague, Sarah Monazam Erfani, and Benjamin IP Rubinstein. Multi-level Certified Defense Against Poisoning Attacks in Offline Reinforcement Learning. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  25. [33]

    Neural Trojans

    Yuntao Liu, Yang Xie, and Ankur Srivastava. Neural Trojans. In2017 IEEE International Conference on Computer Design (ICCD), pages 45–48. IEEE, 2017. 11

    2017

  26. [34]

    Policy Poisoning in Batch Reinforcement Learning and Control.Advances in Neural Information Processing Systems, 32, 2019

    Yuzhe Ma, Xuezhou Zhang, Wen Sun, and Jerry Zhu. Policy Poisoning in Batch Reinforcement Learning and Control.Advances in Neural Information Processing Systems, 32, 2019

    2019

  27. [35]

    Offline Pre-Trained Multi-Agent Decision Transformer: One big sequence Model tackles all SMAC tasks.arXiv preprint arXiv:2112.02845, 2021

    Linghui Meng, Muning Wen, Yaodong Yang, Chenyang Le, Xiyun Li, Weinan Zhang, Ying Wen, Haifeng Zhang, Jun Wang, and Bo Xu. Offline Pre-Trained Multi-Agent Decision Transformer: One big sequence Model tackles all SMAC tasks.arXiv preprint arXiv:2112.02845, 2021

    2021 arXiv

  28. [36]

    Playing Atari with Deep Reinforcement Learning, December

    V olodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing Atari with Deep Reinforcement Learning, December

  29. [37]

    Implicit Poisoning Attacks in Two-Agent Reinforcement Learning: Adversarial Policies for Training-Time Attacks

    Mohammad Mohammadi, Jonathan Nöther, Debmalya Mandal, Adish Singla, and Goran Radanovic. Implicit Poisoning Attacks in Two-Agent Reinforcement Learning: Adversarial Policies for Training-Time Attacks. In22nd International Conference on Autonomous Agents and Multiagent Systems,...

    2023

  30. [38]

    Cal-QL: Calibrated offline RL pre-training for efficient Online Fine-Tuning.Advances in Neural Information Processing Systems, 36, 2024

    Mitsuhiko Nakamoto, Simon Zhai, Anikait Singh, Max Sobol Mark, Yi Ma, Chelsea Finn, Aviral Kumar, and Sergey Levine. Cal-QL: Calibrated offline RL pre-training for efficient Online Fine-Tuning.Advances in Neural Information Processing Systems, 36, 2024

    2024

  31. [39]

    JPMorgan Develops Robot to Execute Trades, July 2017

    Laura Noonan. JPMorgan Develops Robot to Execute Trades, July 2017

    2017

  32. [40]

    Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learning

    Amin Rakhsha, Goran Radanovic, Rati Devidze, Xiaojin Zhu, and Adish Singla. Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learning. InProceedings of the 37th International Conference on Machine Learning, pages 7974–7984. PML...

    2020

  33. [41]

    Understanding the Limits of Poisoning Attacks in Episodic Reinforcement Learning

    Anshuka Rangi, Haifeng Xu, Long Tran-Thanh, and Massimo Franceschetti. Understanding the Limits of Poisoning Attacks in Episodic Reinforcement Learning. In Lud De Raedt, editor, Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI- 2...

  34. [42]

    Proximal Policy Optimization Algorithms, August 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal Policy Optimization Algorithms, August 2017. arXiv:1707.06347 [cs]

    2017 arXiv

  35. [43]

    Safe, Multi-Agent, Reinforce- ment Learning for Autonomous Driving.arXiv preprint arXiv:1610.03295, 2016

    Shai Shalev-Shwartz, Shaked Shammah, and Amnon Shashua. Safe, Multi-Agent, Reinforce- ment Learning for Autonomous Driving.arXiv preprint arXiv:1610.03295, 2016

    2016 arXiv

  36. [44]

    Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamics.arXiv preprint arXiv:2009.00774, 2020

    Yanchao Sun, Da Huo, and Furong Huang. Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamics.arXiv preprint arXiv:2009.00774, 2020

    2009 arXiv

  37. [45]

    PettingZoo: Gym for Multi-Agent Reinforcement Learning.Advances in Neural Information Processing Systems, 34:15032–15043, 2021

    J Terry, Benjamin Black, Nathaniel Grammel, Mario Jayakumar, Ananth Hari, Ryan Sullivan, Luis S Santos, Clemens Dieffendahl, Caroline Horsch, Rodrigo Perez-Vicente, et al. PettingZoo: Gym for Multi-Agent Reinforcement Learning.Advances in Neural Information Processing Systems,...

    2021

  38. [46]

    Terry, Ariel Kwiatkowski, John U

    Mark Towers, Jordan K. Terry, Ariel Kwiatkowski, John U. Balis, Gianluca de Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Arjun KG, Markus Krimmel, Rodrigo Perez- Vicente, Andrea Pierré, Sander Schulhoff, Jun Jet Tai, Andrew Tan Jin Shen, and Omar G. Younis. Gymnasi...

    2023

  39. [47]

    BACKDOORL: Backdoor Attack against Competitive Reinforcement Learning

    Lun Wang, Zaynah Javed, Xian Wu, Wenbo Guo, Xinyu Xing, and Dawn Song. BACKDOORL: Backdoor Attack against Competitive Reinforcement Learning. In Zhi-Hua Zhou, editor, Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pages 3699–3...

  40. [48]

    Yue Wang, Esha Sarkar, Wenqing Li, Michail Maniatakos, and Saif Eddin Jabari. Stop-and- Go: Exploring Backdoor Attacks on Deep Reinforcement Learning-based Traffic Congestion Control Systems.IEEE Transactions on Information Forensics and Security, 16:4772–4787, 2021. 12

    2021

  41. [49]

    Yue Wang, Esha Sarkar, Wenqing Li, Michail Maniatakos, and Saif Eddin Jabari. Stop-and- Go: Exploring Backdoor Attacks on Deep Reinforcement Learning-based Traffic Congestion Control Systems.IEEE Transactions on Information Forensics and Security, 16:4772–4787, 2021

    2021

  42. [50]

    Transferable Environment Poisoning: Training-time Attack on Reinforcement Learning

    Hang Xu, Rundong Wang, Lev Raizman, and Zinovi Rabinovich. Transferable Environment Poisoning: Training-time Attack on Reinforcement Learning. InProceedings of the 20th International Conference on Autonomous Agents and Multiagent Systems, pages 1398–1406, 2021

    2021

  43. [51]

    Design of Intentional Back- doors in Sequential Models.arXiv preprint arXiv:1902.09972, 2019

    Zhaoyuan Yang, Naresh Iyer, Johan Reimann, and Nurali Virani. Design of Intentional Back- doors in Sequential Models.arXiv preprint arXiv:1902.09972, 2019

    1902 arXiv

  44. [52]

    A Temporal-Pattern Backdoor Attack to Deep Reinforcement Learning

    Yinbo Yu, Jiajia Liu, Shouqing Li, Kepu Huang, and Xudong Feng. A Temporal-Pattern Backdoor Attack to Deep Reinforcement Learning. InGLOBECOM 2022-2022 IEEE Global Communications Conference, pages 2710–2715. IEEE, 2022

    2022

  45. [53]

    ∞X k=0 γkRopp(si+k+1, aopp i+k+1, aatt i+k+1) aopp i+k+1 ∼π opp, aatt i+k+1 ∼π # , (7) πopp = argminπ∈Π E

    Huichu Zhang, Siyuan Feng, Chang Liu, Yaoyao Ding, Yichen Zhu, Zihan Zhou, Weinan Zhang, Yong Yu, Haiming Jin, and Zhenhui Li. CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario. InThe World Wide Web Conference, pages 3620–3624, 20...

    2019

  46. [56]

    The trigger action and backdoor action remain consistent with those outlined in Section 4

    We designate one attacker in Team 1 and one victim in Team 2, while the remaining players function as normal players. The trigger action and backdoor action remain consistent with those outlined in Section 4

  47. [57]

    move-up,

    We designate one attacker in Team 1, with both players in Team 2 marked as victims, while the remaining player acts as a normal player. The backdoor actions are separately attributed to each victim, apart from this remain consistent with the definition in Section 4. The result...

  48. [2013]

    arXiv:1312.5602 [cs]

    arXiv

  49. [8677]

    PMLR, 23–29 Jul 2023

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.