Pith. sign in

REVIEW 4 major objections 5 minor 33 references

The paper claims that a deterministic, non-model monitor reading out-of-band source-trust labels can provably keep untrusted text from changing the bound operation or authorizing an action, while a tuned 'conditioned cascade' raises the gen

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 06:03 UTC pith:5SVLRY72

load-bearing objection A soundly-scoped trust architecture that honestly separates a provable authority boundary from a measured text defense; worth a serious look despite a small eval and a trusted-ingress assumption. the 4 major comments →

arxiv 2607.13149 v1 pith:5SVLRY72 submitted 2026-07-14 cs.CR

Composable Trust for Language Models: A proven boundary and a measured defense

Yakov Pyotr Shkolnikov This is my paper
classification cs.CR
keywords prompt injectionintegrity latticetrust ringsnon-interferenceauthority boundaryconditioned cascadeout-of-band labelingAI security
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper takes on the core problem of prompt injection: in a language model, instructions and data share one token stream, so nothing inside the model can reliably tell them apart. The author's proposal is to move authority outside the model entirely. Each input arrives on a channel with a fixed integrity ring, assigned at ingress, and a deterministic monitor—not the model—uses those rings to decide which operation runs and whether any outside action fires. This part is proven: over the finite lattice of trust rings, no untrusted input can rebind the operation or meet the action threshold; monotonicity extends the proof to inputs of any depth. Separately, the paper measures a soft defense, a conditioned cascade of passivation and a wrapper, that raises the genuine-leak defended rate from 27% to 94% at about 4% clean-quality cost, and improves attribution of lower-trust facts from 0% to 92%. The pith is the separation: what can be guaranteed (authority) versus what can only be measured (text generation).

Core claim

The central claim is that authority over an AI system's actions can be made a provable property of a deterministic pipeline rather than a property of the model's judgment. The pipeline labels every input with a trust ring assigned out-of-band, combines derived values by the meet (lowest ring) so trust never rises, allows a ring to rise only through an unforgeable authority token, and gates any action on a fixed threshold. A monitor enforces these invariants in code. Because the lattice is finite, the author proves by enumeration that no untrusted input can change the bound operation or authorize an action, and monotonicity of the meet extends this to inputs of arbitrary depth. The generated

What carries the argument

The load-bearing mechanism is an integrity lattice with a low-water-mark rule, enforced by a deterministic monitor outside the model. Each channel has a fixed trust ring; the meet of combining inputs is the lowest ring (I1); a ring can rise only via an out-of-band authority token content cannot forge (I2); an action fires only when its controlling inputs' ring meets a fixed threshold (I3); every write carries its stamp so rings cannot be shed (I4); and the operation binds to the instruction on the highest ring present (I5). The monitor enforces these invariants in deterministic code, so the proof is over the finite lattice of labeled values. On the text side, the 'conditioned cascade' passiv

Load-bearing premise

The entire authority guarantee rests on the assumption that an uncompromised operator assigns every input its trust ring at ingress and that no content can forge a higher ring or an authority token; if a poisoned document or user request can be routed or mislabeled into a higher-trust channel, the proved boundary stops constraining it.

What would settle it

Find a single input configuration—out of the 8,729 the paper enumerates, or one of arbitrary depth—where a low-trust channel changes the bound operation or causes an action to fire despite the invariants; a concrete version would be a document arriving on the content channel that includes a forged authority token or a user-request label and succeeds in getting the actuator to fire. For the measured defense, an adaptive attack on the deployed prompt set that drives the genuine-leak defended rate below the reported 87% would falsify the measured claim.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the authority proof holds, no amount of training, fine-tuning, or model provenance can give untrusted content the power to re-task the system or fire actions; the boundary is architectural, not a behavior of the model.
  • The same design applies to hosted or proprietary models whose weights are inaccessible, since only the prompt set is tuned and the monitor is external.
  • An application can compose this layer beneath its own agent controls, adding other provable or measured layers (verification, content classifiers, tool-use policy) without changing the authority guarantee.
  • A text leak becomes a distinguishable, bounded event: it changes the answer but cannot authorize an action, so the attacker gets no guaranteed success signal.
  • The declared rings, thresholds, and prompts can be audited like a policy, and moving to a different lattice only changes configuration, not code.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The author leaves implicit that the same lattice could handle partial orders or incomparable rings, with the meet as greatest lower bound and the action gate failing closed; the proof covers any finite lattice, so the authority guarantee would carry over unchanged without the tuning that the strict total order needed.
  • The measured defense's residual failures are mostly format or language changes on points the operation left unspecified; an obvious extension is a runtime self-check that can only lower integrity labels, turning offline metrics into a per-request signal.
  • One could test whether the cascade's measured defense generalizes across different families of models; since only the prompt set is tuned and the model is unmodified, the mechanism is model-agnostic in principle, but each new model needs its own re-tuned prompts.
  • A broader implication: trust in LLM systems can be separated into provable, auditable structure (source standing, thresholds) and measured behavior (text generation), and the two should not be conflated in benchmarks.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes an LLM-serving architecture in which authority is placed outside the model: a deterministic monitor labels input channels with an integrity lattice (SYSTEM > USER > CONTENT > WEB), binds each operation to the highest-trust instruction, and gates any actuator action on a fixed threshold. The model is used as an untrusted generator inside a conditioned cascade: each below-SYSTEM channel is passivated once, conditioned on the already-passivated higher-trust context, and the wrapper orders the resulting blocks. The paper claims a proved non-interference guarantee: over the finite lattice, an exhaustive enumeration (8729 configurations) shows no untrusted input can change the bound operation or authorize an action, and a monotonicity lemma extends this to arbitrary depth. The paper also reports a measured text defense: on a one-shot held-out set with an unmodified Gemma 4 26B model, the cascade raises the genuine-leak defended rate from 27% to 94% (Q_rel=0.96), keeps it at 87% under adaptive red-teaming, and improves web-fact attribution from 0% to 92%.

Significance. The core architectural idea — separating authority from generation and enforcing a Biba-style lattice outside the model — is sound and potentially of broad interest. The paper is commendably explicit about what is proved versus what is measured: authority-rejection is stated to be 0 by construction, while the text defense is presented as an empirical, tuned result with a genuine-leak grade and human-validated judge. The finite-state enumeration is a concrete, checkable claim, and the authors have released code and prompts. If the proof artifact is made machine-checkable and the measured evaluation is strengthened or appropriately scoped, this could be a useful contribution to prompt-injection defenses and composable trust. The current manuscript, however, leaves the central proof as an underspecified enumeration and the headline measurement on a self-built benchmark without comparison to existing defenses.

major comments (4)
  1. [Sec. 4.2 / Algorithm 1] The authority proof rests on an enumeration that is neither machine-checked nor fully specified. The text states 'Checking the monitor’s invariants over the finite integrity lattice (8729 input configurations, no counterexample)' but does not define the state space, the formal predicates for I1–I5, or the checker procedure. The repository link has no commit hash, so the reader cannot verify that the enumeration covers all relevant cases (e.g., mixed-ring effects, passivation outputs, nested inputs). Because this is the paper’s load-bearing 'proved boundary' claim, the authors should provide the formal model and an executable checker (or a small proof assistant development) as part of the artifacts, and state the exact assumptions.
  2. [Abstract / Sec. 2 (Threat model)] The guarantee is conditional on the trust perimeter: the operator assigns channels out-of-band, and the monitor and actuator are uncompromised. The paper states this in Sec. 2, but the abstract and conclusion claim unconditionally that 'no untrusted input can change the bound operation' and that the proof holds 'unconditionally' under adaptive red-teaming (Sec. 5). This overstates the result. The proof is valid only for inputs that are correctly labeled at ingress; a mislabeled or confused-deputy input can be elevated to a higher ring. The abstract and conclusion should qualify the claim as 'within the stated trust perimeter' and the residual risk of ingress mislabeling should be discussed as a limitation of the theorem, not merely of the deployment.
  3. [Sec. 4.1 / Table 2] The measured 94% defense is reported on a self-built benchmark with no comparison against existing defenses (e.g., Instruction Hierarchy, StruQ, SecAlign, CaMeL, or Spotlighting) on the same attack set. The base condition is a naive single-prompt baseline; an improvement from 27% to 94% over that baseline is not evidence that the defense is competitive with prior art. The authors should either run a representative set of existing defenses on their held-out set, or clearly state why such comparisons are not meaningful in this setting, and release the benchmark to enable independent comparison.
  4. [Table 2 / Sec. 4.1] Sample sizes are not reported for the headline instruction-injection rows. The aggregate 94% has a Wilson CI of [88,97], which suggests roughly n≈100, but the per-class rows (token-emit, task-hijack, AgentDojo) have CIs implying much smaller n. The base rates have no CIs. The content-poison row is explicitly stated to rest on single-digit per-cell counts. For a headline measured claim, the paper should report the exact n per row and per condition, and present the base rates with CIs. Without this, the 94% figure is difficult to interpret.
minor comments (5)
  1. [Sec. 2 / Algorithm 1] The monotonicity lemma is stated but not proved. A one-sentence proof sketch (the meet is monotone non-increasing under additional inputs, so adding lower-trust content cannot raise the ring of any derived value) would help the reader verify the 'arbitrary depth' claim.
  2. [Sec. 4.1 / Judge] The description of the 'genuine-leak' regrade and the two bracketing instruments is detailed but hard to follow. A short pseudocode or decision rule for what counts as a genuine leak (as opposed to quoting a stripped token) would make the metric more reproducible.
  3. [Sec. 4.1 / Datasets] The adaptation of AgentDojo templates is described only as 'adapting the injected goal to a token-emit demand.' Please specify concretely what was changed and confirm that the template structure was not otherwise altered.
  4. [Sec. 3.1 / Algorithm 2] The tuning procedure is described clearly, but the number of candidates (~12) and reflection passes (3) are very small; consider adding a sentence on how the authors verified that this budget is sufficient to explore the joint prompt space (e.g., the seed-robustness results in Appendix B help, but a sensitivity analysis on the candidate budget would strengthen the claim).
  5. [Throughout] Minor typographical and formatting issues: 'Q rel' appears inconsistently (sometimes as Q_rel, sometimes as Qrel); 'content-poison' and 'content-poisoning' are used interchangeably; Table 3 and Table 4 have footnote-style explanations that are easy to miss. A careful final edit would improve readability.

Circularity Check

0 steps flagged

No significant circularity: authority is a stated formal theorem, defense is held-out measurement, and the ingress assumption is explicit.

full rationale

The paper carefully separates two claims. The authority guarantee is not a fitted prediction; it is an explicit correctness theorem for deterministic code. Algorithm 1 enforces invariants I1–I5, and Section 4.2 states: "Checking the monitor’s invariants over the finite integrity lattice (8729 input configurations, no counterexample) proves that no low-ring channel can change the operation or cause an action." The paper explicitly labels this "0 by construction." A proof that a monitor satisfies the invariants defining it is formal verification, not an empirical claim derived from data, so it is not circular. The measured defense (27%→94%, Qrel=0.96, attribution 0%→92%) is reported on a one-shot held-out set: "Train, validation, and test content are pairwise disjoint," and prompts are selected using validation splits with length caps and OOD non-regression gates. The use of the same human-validated judge family during tuning selection and final evaluation is a possible risk, but the paper validates the judge against blind human raters (κ=0.93/0.90) and reproduces the key grade with an independent GPT-5.6 judge; nothing is statistically forced by construction. The "trusted ingress" requirement is stated as a threat-model assumption ("The operator... trusted and assumed uncompromised"; "The ring is assigned out-of-band, not by the content the input carries"), which defines the boundary of the theorem rather than smuggling in the conclusion. No load-bearing self-citations appear: references to Biba, Thompson, Sleeper Agents, AgentDojo, etc., are external. The base's 0% attribution is acknowledged as "provenance-blind by construction," not presented as a measured result. Therefore no circular step can be exhibited.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The paper introduces no new physical or formal entity; it reuses established security concepts (Biba lattice, reference monitor, IFC) in a new composition. The main load-bearing choices are the trusted-ingress assumption, the monotonicity extension of a finite enumeration, and the tuned prompt set, all listed above.

free parameters (4)
  • deployed prompt set (3 keys) = Appendix A: USER passivation, CONTENT/WEB passivation, wrapper/ring-label prompts
    Tuned end-to-end by the SkillOpt-adapted search to maximize joint (Q, R) on train/validation splits; these strings carry much of the measured defense.
  • exploration slack tau_explore = 5%
    Hand-chosen tolerance in the Pareto acceptance rule; affects which prompt candidates are accepted.
  • quality gate tau_Q = 0.95
    Hand-set minimum retained clean quality; the held-out Qrel=0.96 clears it, so this gate influenced selection.
  • OOD non-regression tolerance epsilon_ood = not specified in text
    Hand-set tolerance; candidates that regress beyond it on the OOD split are dropped. Exact value is not reported.
axioms (5)
  • domain assumption The operator, monitor, and actuator are trusted and uncompromised.
    Invoked in Sec. 2 Threat model; the proof holds only within this trust perimeter.
  • domain assumption Input integrity rings are assigned out-of-band; content cannot forge a ring or authority token.
    Invoked in Sec. 2 Invariants (I1, I2); without this, a low-trust input could label itself high-trust.
  • standard math The Biba meet (minimum/lowest ring) is monotone on the finite lattice, so enumeration at bounded depth extends to arbitrary input depths.
    Sec. 2 and Sec. 4.2; this is the mathematical lever behind the 'proved' authority boundary.
  • domain assumption In transformer LLMs, instructions and data occupy one undifferentiated token stream.
    Sec. 1 motivation; the paper cites refs [5,6] for impossibility results.
  • domain assumption A lower-trust input can be treated as adversarial or honest equivalently for authority purposes.
    Sec. 2 Threat model: integrity is fixed by channel, not content, so the proof applies to the adversarial case.

pith-pipeline@v1.3.0-alltime-deepseek · 17473 in / 15652 out tokens · 173994 ms · 2026-08-02T06:03:44.982111+00:00 · methodology

0 comments
read the original abstract

In a language model, instructions and data share one token stream, so nothing inside the model's generation can keep untrusted text from steering it. We develop a trust model that places the authority to act outside the model, in code: a source's standing, not its content, decides which operation runs and whether it acts. A lower-trust source may inform an answer but not override a higher one. An unmodified model runs inside a deterministic pipeline that ranks inputs by source integrity, and a fixed non-model monitor provably chooses the operation and any outside action from trusted inputs alone. We can measure but not prove the pipeline's resistance to injection; we prompt-tune it and report the rate. On a one-shot held-out set with an unmodified Gemma~4 26B model, passivation and a wrapper (the cascade) raise the genuine-leak defended rate from $27\%$ to $94\%$ at roughly a $4\%$ clean-quality cost ($Q_{\mathrm{rel}}{=}0.96$). Under adaptive red-teaming the proved boundary holds unconditionally, and the measured defense stays at $87\%$. The cascade also attributes a lower-trust source's fact rather than dropping it, raising attribution from $0\%$ to $92\%$, and follows the higher-trust source on a conflict.

Figures

Figures reproduced from arXiv: 2607.13149 by Yakov Pyotr Shkolnikov.

Figure 1
Figure 1. Figure 1: Each input arrives on a channel that fixes its integrity ring (small caps). The deterministic [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: A worked example of the conditioned cascade (26B model, deployed prompt set); channels in trust order [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Reliability over tuned stages, on the development-set static (non-adaptive) attack suite; see Sec. [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Convergence of the tuner from two different starts. A cold start ( [PITH_FULL_IMAGE:figures/full_fig_p015_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

33 extracted references · 13 linked inside Pith

  1. [1]

    Is justified true belief knowledge?,

    E. L. Gettier, “Is justified true belief knowledge?,” 1963. 15 Composable Trust for Language ModelsA PREPRINT Table 6: Seed robustness (validation-split selection). A cold and a warm start reach the same rejection and comparable relative quality.Q rel=Q/Qbase. Each start measuresQ base on its own validation draw, so the implied base quality differs slight...

    1963

  2. [2]

    Why Language Models Hallucinate,

    A. T. Kalai, O. Nachum, S. S. Vempala, and E. Zhang, “Why Language Models Hallucinate,” Sept. 2025. arXiv:2509.04664

    Pith/arXiv arXiv 2025

  3. [3]

    Computer security technology planning study,

    J. P. Anderson, “Computer security technology planning study,” tech. rep., 1972

    1972

  4. [4]

    Judging llm-as-a-judge with mt-bench and chatbot arena,

    L. Zheng, W.-L. Chiang, Y . Sheng, S. Zhuang, Z. Wu, Y . Zhuang, Z. Lin, Z. Li, D. Li, and E. Xing, “Judging llm-as-a-judge with mt-bench and chatbot arena,”Advances in neural information processing systems, vol. 36, pp. 46595–46623, 2023

    2023

  5. [5]

    On the Inseparability of Instructions and Data in Shared-Embedding Sequence Models,

    D. Pant, S. Lohani, and A. Kumar, “On the Inseparability of Instructions and Data in Shared-Embedding Sequence Models,” June 2026. arXiv:2606.27567

    Pith/arXiv arXiv 2026

  6. [6]

    AI Agents May Always Fall for Prompt Injections,

    S. Abdelnabi and E. Bagdasarian, “AI Agents May Always Fall for Prompt Injections,” May 2026. arXiv:2605.17634

    Pith/arXiv arXiv 2026

  7. [7]

    GIF: Locally Sound Geometric Information Flow Control for LLMs,

    A. Storek, N. Holzer, Z. Zhang, and S. Jana, “GIF: Locally Sound Geometric Information Flow Control for LLMs,” June

  8. [8]

    Reflections on trusting trust,

    K. Thompson, “Reflections on trusting trust,”Communications of the ACM, vol. 27, pp. 761–763, Aug. 1984

    1984

  9. [9]

    Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training,

    E. Hubinger, C. Denison, J. Mu, M. Lambert, M. Tong, M. MacDiarmid, T. Lanham, D. M. Ziegler, T. Maxwell, N. Cheng, A. Jermyn, A. Askell, A. Radhakrishnan, C. Anil, D. Duvenaud, D. Ganguli, F. Barez, J. Clark, K. Ndousse, K. Sachan, M. Sellitto, M. Sharma, N. DasSarma, R. Grosse, S. Kravec, Y . Bai, Z. Witten, M. Favaro, J. Brauner, H. Karnofsky, P. Chris...

    Pith/arXiv arXiv 2024

  10. [10]

    Integrity considerations for secure computer systems,

    K. J. Biba, “Integrity considerations for secure computer systems,” tech. rep., 1977

    1977

  11. [11]

    A decentralized model for information flow control,

    A. C. Myers and B. Liskov, “A decentralized model for information flow control,”ACM SIGOPS Operating Systems Review, vol. 31, pp. 129–142, Dec. 1997

    1997

  12. [12]

    Defending Against Indirect Prompt Injection Attacks With Spotlighting,

    K. Hines, G. Lopez, M. Hall, F. Zarfati, Y . Zunger, and E. Kiciman, “Defending Against Indirect Prompt Injection Attacks With Spotlighting,” Mar. 2024. arXiv:2403.14720

    Pith/arXiv arXiv 2024

  13. [13]

    ADOPT: Adaptive Dependency-Guided Joint Prompt Optimization for Multi-Step LLM Pipelines,

    M. Zhao, X. Zhang, S. Zhang, D. Li, and R. Shi, “ADOPT: Adaptive Dependency-Guided Joint Prompt Optimization for Multi-Step LLM Pipelines,” Apr. 2026. arXiv:2512.24933

    Pith/arXiv arXiv 2026

  14. [14]

    Pareto prompt optimization,

    G. Zhao, B.-J. Yoon, G. Park, S. Jha, S. Yoo, and X. Qian, “Pareto prompt optimization,” inInternational Conference on Learning Representations, vol. 2025, pp. 6770–6785, 2025

    2025

  15. [15]

    SkillOpt: Executive Strategy for Self-Evolving Agent Skills,

    Y . Yang, Z. Gong, W. Huang, Q. Yang, Z. Zhou, Z. Huang, Y . Li, X. Gao, Q. Dai, B. Liu, K. Qiu, Y . Yang, D. Chen, X. Yang, and C. Luo, “SkillOpt: Executive Strategy for Self-Evolving Agent Skills,” May 2026. arXiv:2605.23904

    Pith/arXiv arXiv 2026

  16. [16]

    Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for llm agents,

    E. Debenedetti, J. Zhang, M. Balunovic, L. Beurer-Kellner, M. Fischer, and F. Tramèr, “Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for llm agents,”Advances in Neural Information Processing Systems, vol. 37, pp. 82895–82920, 2024

    2024

  17. [17]

    Enabling large language models to generate text with citations,

    T. Gao, H. Yen, J. Yu, and D. Chen, “Enabling large language models to generate text with citations,” inProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 6465–6488, 2023

    2023

  18. [18]

    Security policies and security models,

    J. A. Goguen and J. Meseguer, “Security policies and security models,” in1982 IEEE symposium on security and privacy, pp. 11–11, IEEE, 1982

    1982

  19. [19]

    Rushby,Noninterference, transitivity, and channel-control security policies

    J. Rushby,Noninterference, transitivity, and channel-control security policies. SRI International, Computer Science Labora- tory Menlo Park, 1992

    1992

  20. [20]

    On adaptive attacks to adversarial example defenses,

    F. Tramer, N. Carlini, W. Brendel, and A. Madry, “On adaptive attacks to adversarial example defenses,”Advances in neural information processing systems, vol. 33, pp. 1633–1645, 2020

    2020

  21. [21]

    Defeating Prompt Injections by Design,

    E. Debenedetti, I. Shumailov, T. Fan, J. Hayes, N. Carlini, D. Fabian, C. Kern, C. Shi, A. Terzis, and F. Tramèr, “Defeating Prompt Injections by Design,” June 2025. arXiv:2503.18813

    Pith/arXiv arXiv 2025

  22. [22]

    {StruQ}: Defending against prompt injection with structured queries,

    S. Chen, J. Piet, C. Sitawarin, and D. Wagner, “{StruQ}: Defending against prompt injection with structured queries,” in34th USENIX Security Symposium (USENIX Security 25), pp. 2383–2400, 2025

    2025

  23. [23]

    SecAlign: Defending Against Prompt Injection with Preference Optimization,

    S. Chen, A. Zharmagambetov, S. Mahloujifar, K. Chaudhuri, D. Wagner, and C. Guo, “SecAlign: Defending Against Prompt Injection with Preference Optimization,” inProceedings of the 2025 ACM SIGSAC Conference on Computer and Communi- cations Security, (Taipei Taiwan), pp. 2833–2847, ACM, Nov. 2025

    2025

  24. [24]

    ASIDE: Archi- tectural Separation of Instructions and Data in Language Models,

    E. Zverev, E. Kortukov, A. Panfilov, A. V olkova, S. Tabesh, S. Lapuschkin, W. Samek, and C. H. Lampert, “ASIDE: Archi- tectural Separation of Instructions and Data in Language Models,” Feb. 2026. arXiv:2503.10566. 16 Composable Trust for Language ModelsA PREPRINT

    arXiv 2026

  25. [25]

    The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions,

    E. Wallace, K. Xiao, R. Leike, L. Weng, J. Heidecke, and A. Beutel, “The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions,” Apr. 2024. arXiv:2404.13208

    Pith/arXiv arXiv 2024

  26. [26]

    Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations,

    H. Inan, K. Upasani, J. Chi, R. Rungta, K. Iyer, Y . Mao, M. Tontchev, Q. Hu, B. Fuller, D. Testuggine, and M. Khabsa, “Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations,” Dec. 2023. arXiv:2312.06674

    Pith/arXiv arXiv 2023

  27. [27]

    {PoisonedRAG}: Knowledge corruption attacks to {Retrieval-Augmented} generation of large language models,

    W. Zou, R. Geng, B. Wang, and J. Jia, “{PoisonedRAG}: Knowledge corruption attacks to {Retrieval-Augmented} generation of large language models,” in34th USENIX Security Symposium (USENIX Security 25), pp. 3827–3844, 2025

    2025

  28. [28]

    Not all contexts are equal: Teaching llms credibility- aware generation,

    R. Pan, B. Cao, H. Lin, X. Han, J. Zheng, S. Wang, X. Cai, and L. Sun, “Not all contexts are equal: Teaching llms credibility- aware generation,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 19844– 19863, 2024

    2024

  29. [29]

    Certifiably Robust RAG against Retrieval Corruption,

    C. Xiang, T. Wu, Z. Zhong, D. Wagner, D. Chen, and P. Mittal, “Certifiably Robust RAG against Retrieval Corruption,” May

  30. [30]

    Knowledge conflicts for llms: A survey,

    R. Xu, Z. Qi, Z. Guo, C. Wang, H. Wang, Y . Zhang, and W. Xu, “Knowledge conflicts for llms: A survey,” inProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 8541–8565, 2024

    2024

  31. [31]

    Securing AI Agents with Information-Flow Control,

    M. Costa, B. Köpf, A. Kolluri, A. Paverd, M. Russinovich, A. Salem, S. Tople, L. Wutschitz, and S. Zanella-Béguelin, “Securing AI Agents with Information-Flow Control,” Sept. 2025. arXiv:2505.23643

    Pith/arXiv arXiv 2025

  32. [32]

    System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspective,

    F. Wu, E. Cecchetti, and C. Xiao, “System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspective,” Oct. 2024. arXiv:2409.19091

    Pith/arXiv arXiv 2024

  33. [33]

    The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections,

    M. Nasr, N. Carlini, C. Sitawarin, S. V . Schulhoff, J. Hayes, M. Ilie, J. Pluto, S. Song, H. Chaudhari, I. Shumailov, A. Thakurta, K. Y . Xiao, A. Terzis, and F. Tramèr, “The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injections,” Oct. 2025. arXiv:2510.09023. 17

    Pith/arXiv arXiv 2025