Pith. sign in

REVIEW 3 major objections 4 minor 5 cited by

This paper claims that a dot-product filter on a language model's internal activations can catch prompts that seek contextually inappropriate disclosure even when text-based censors are bypassed, and that a cumulative 'activation velocity'

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 09:05 UTC pith:ZDQYV54O

load-bearing objection The single-turn activation-probe results are solid and worth attention; the multi-turn 'activation velocity' claim is far thinner than the 'perfect accuracy in 4–6 turns' framing suggests. the 3 major comments →

arxiv 2601.14660 v2 pith:ZDQYV54O submitted 2026-01-21 cs.CR cs.AIcs.CL

NeuroFilter: Activation-Based Guardrails for Privacy-Conscious LLM Agents

Saswat Das , Ferdinando Fioretto This is my paper
classification cs.CR cs.AIcs.CL
keywords activation probingcontextual integrityprivacy guardrailsLLM agentsmulti-turn jailbreakingmosaic attackslinear representation hypothesisactivation velocity
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that privacy-violating intent in agentic LLMs is visible as linear structure in the model's own activation space, so a guardrail can be built from a single learned direction rather than from an expensive LLM-based monitor. It introduces NeuroFilter, which learns a linear probe on cached activations to flag single-turn jailbreaks, and an "activation velocity" signal — the cumulative projection of per-turn activation changes onto a learned direction — to catch multi-turn conversational manipulation and mosaic attacks. The authors report perfect detection within a few turns on their benchmarks, zero false positives on benign prompts, and a per-turn cost of O(d) that is orders of magnitude below LLM-supervisor defenses. Why it matters: if true, contextual-integrity privacy can be enforced at near-zero latency, turning a costly policy-checking step into a dot product.

Core claim

The paper's central claim is that the intent to violate a contextual-integrity norm — disclosing information to a role that should not receive it — is linearly separable from benign intent in the residual-stream activations of a decoder-only LLM. Single-turn probes are context-specific rather than a universal "harmfulness" direction; the directions for different privacy scenarios are nearly orthogonal. For multi-turn settings, the paper defines activation velocity v_t as the change in the cached activation after processing the conversation history up to turn t, trains a linear probe on these velocities, and accumulates the signed projection into a cumulative drift C_t = Σ_k ⟨v_k, w_vel⟩. It

What carries the argument

The central object is the activation-velocity probe. Velocity is the per-turn displacement of the model's cached internal representation, v_t = a(p_{1:t}) − a(p_{1:t−1}); a linear probe trained on these displacement vectors yields a direction w_vel, and the cumulative drift C_t = Σ_k ⟨v_k, w_vel⟩ is the scalar "steering toward violation" signal. This object converts an entire conversation history into one number per turn, so a guardrail can flag before leakage with a threshold comparison and O(d) overhead per turn, no auxiliary model required.

Load-bearing premise

The load-bearing assumption is that adversarial steering manifests as a steadily accumulating linear displacement of the model's internal state along a single direction, measured from one fixed token position; if per-turn activation changes are nonlinear, position-dependent, or not aligned with the probe, the cumulative statistic will not flag before leakage — and the paper's multi-turn evidence is only 20 trajectories per scenario, which it concedes is too small for non-triv

What would settle it

A concrete test: take the released probes and run a search over multi-turn prompt trajectories, including trajectories designed to keep cumulative drift near zero (for example, alternating positive and negative velocity projections or spreading the steering across token positions). If any unflagged trajectory reaches a leakage turn before the cumulative drift crosses the threshold, the central safety claim fails. A simpler observation would be a natural benign conversation whose cumulative drift grows consistently in the violation direction, yielding a false positive.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Privacy guardrails for agents can be built and re-trained cheaply: training a linear probe on cached activations costs orders of magnitude less than training or running an LLM-based firewall.
  • Multi-turn attacks that distribute malicious intent across benign-looking turns — conversational manipulation and mosaic decomposition — can be stopped before the secret is revealed by watching cumulative activation drift.
  • Because the filter scores intent rather than realized leakage, it refuses prompts that merely attempt a violation, not only those that succeed.
  • A single universal "harmful prompt" probe is insufficient for contextual privacy: the paper shows privacy-violation directions are context-dependent, so probes must be trained per privacy directive and role, and can be assembled from attribute-level probes.
  • The approach survives practical deployment concerns raised in the paper: it works across 4-bit, 8-bit, and BF16 quantization and across model sizes, and degrades gracefully but needs retraining on fine-tuned variants.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: The same "drift toward a disallowed state" signal need not be limited to privacy; any scalar alignment property with a linear direction in activation space could be monitored the same way, making NeuroFilter-style velocity probes a general mechanism for online safety monitoring.
  • Editorial inference: Because the velocity is read from a single fixed token position, prompts engineered to shift information across token positions or to alternate directions so the cumulative sum stays near zero may evade the statistic; a testable extension is to average velocity over token positions or weight it by attention.
  • Editorial inference: The evidence for the multi-turn claim is thinner than the single-turn evidence — 20 trajectories per CMPL scenario and 100 mosaic trajectories — so "perfect within 4–6 turns" is a strong claim resting on a small sample; a larger corpus with non-trivial splits is the natural next check.
  • Editorial inference: If per-context probes turn out to be necessary, deployers face a combinatorial maintenance burden of a probe per role, attribute, and model version; the paper's modular attribute-probe superposition points toward a way to cut that down, but the lifecycle story is unfinished.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes NeuroFilter, a family of activation-probing guardrails for enforcing contextual-integrity privacy in LLM-based conversational agents. For single-turn attacks, a logistic-regression probe is trained on cached layer activations and the projection score is thresholded. For multi-turn attacks, the paper introduces 'activation velocity'—the turn-to-turn displacement of a fixed-token-position activation—and flags trajectories when the cumulative drift along a learned velocity-probe direction exceeds a threshold. The main empirical claims are: (i) single-turn probes achieve perfect or near-perfect test accuracy on CMPL and PrivacyLens across GPT-OSS 20B, Qwen 2.5 32B, and Llama 3.3 70B over roughly 150k interactions; (ii) multi-turn velocity probes detect CMPL conversational manipulation with 'perfect accuracy within 4–6 turns' and Fractured SORRY-Bench mosaic attacks with >85% accuracy; and (iii) the approach has orders-of-magnitude lower inference cost than LLM-based firewalls. The paper also reports ablations over quantization, model size, architectural aspect ratio, and fine-tuning robustness.

Significance. If the multi-turn claim held, the paper would make a meaningful contribution: a constant-cost per-turn linear filter that detects adversarial intent distributed across turns would be a practical and novel defense against conversational manipulation and mosaic attacks. The single-turn component is well supported by a large-scale evaluation across multiple model families, and the comparison against SAE and LLM-based baselines, including the dramatic computational savings, is convincing. The paper is also transparent about several limitations, which is commendable. However, the headline multi-turn result—the paper's core novelty—rests on very small test sets with heuristic thresholds, so the significance of the multi-turn contribution is not yet established at the level claimed.

major comments (3)
  1. [§6.2.3, Fig. 7, Fig. 17, Fig. 18, Appendix A] The central multi-turn claim—'perfect accuracy within 4-6 turns'—is supported by only 20 adversarial and 20 benign trajectories per CMPL scenario, with a 70:30 split leaving 6 test trajectories per class. No error bars, confidence intervals, or seed variance are reported, and thresholds are set to τ=0 or chosen heuristically, as Appendix A concedes. Perfect accuracy on 6 test trajectories cannot support a general claim of perfect detection, particularly when the same appendix states that larger trajectory datasets are needed for non-trivial train-test splits. This is load-bearing because the multi-turn extension is the paper's main novelty. Please provide a substantially larger multi-turn evaluation with calibrated thresholds, error bars, and a clear statement of the test-set size and leakage-turn distribution.
  2. [§4.2, Eqs. (8)–(9)] The activation-velocity model assumes that adversarial steering is readable as a linearly accumulating displacement in activation space, computed from a single fixed token position at each turn, and that the cumulative statistic C_t = Σ_k ⟨v_k, w_vel⟩ is a valid detector. This is an asserted modeling assumption, not a derived or systematically validated one. The current evidence base (the small CMPL and mosaic test sets) cannot rule out failure modes where per-turn changes are nonlinear, vary with token position, or are not aligned with the velocity-probe direction. Please report sensitivity analyses for token position, layer choice, trajectory length, and threshold calibration on a broader set of attack trajectories, and temper the 'perfect accuracy' wording accordingly.
  3. [Table 1, §6.1.1] Table 1 reports multi-turn r_bypass=0 and UT=0 for NeuroFilter as exact zeros. Given that the underlying multi-turn evaluation uses only 20 trajectories per scenario (with 14 in the training set and 6 in the test set), these exact-zero values are not statistically meaningful as stated. The table should include sample sizes and, ideally, confidence intervals or a per-trajectory breakdown. This also affects the comparison with the agentic network firewall, where a 5% bypass rate is one trajectory out of twenty.
minor comments (4)
  1. [§6.2.3 / Fig. 18] The text says the test set comprises 'trajectories 14-19', which matches a 70:30 split of 20 trajectories, but Figure 18 labels the rows 'Information Subject' rather than trajectory indices. Please align the notation to avoid confusion.
  2. [Appendix A] The limitation that thresholds are set to τ=0 or heuristically is important and should be stated in the main text near the multi-turn results, not only in the appendix. This would also make the 'perfect accuracy' claims appropriately qualified.
  3. [§5 / Datasets] For PrivacyLens, the text reports '98600 combinations'; it would help to state explicitly how many total prompts are used after balancing and whether any are reused across train/test to avoid duplication leakage.
  4. [§6.3 / Fig. 11] The Pearson correlation of 0.9958 is reported for maximum distance versus aspect ratio, but no uncertainty or number of points is stated. With only four model sizes, this is a very small sample; please provide a scatter plot with all points and avoid overclaiming a strong architectural law.

Circularity Check

0 steps flagged

No circular derivation; the core pipeline is a standard supervised probe trained on externally labeled prompts/trajectories and evaluated on held-out data. The only self-citation supplies a benchmark, not the load-bearing linearity claim.

full rationale

The paper's derivation chain is a standard supervised probing procedure. Ground truth is defined by the contextual-integrity directive ψ (Eq. 2), independent of the filter's parameters. Single-turn detection fits logistic-regression weights w^ℓ on labeled activations (Eq. 6) and classifies via the projection score s^ℓ(p)=⟨a^ℓ(p),w^ℓ⟩ (Eq. 7). Multi-turn detection trains a velocity-probe weight on turn-to-turn activation differences and accumulates drift C_t=Σ_k⟨v^ℓ_k,w^ℓ_vel⟩ (Eqs. 8-9). Test accuracy is reported on 30% held-out prompts/trajectories, so the reported 'predictions' are not equal to the training fit by construction. The only in-house citation is CMPL [7], co-authored by the present authors, used to generate attack trajectories ('For multi-turn probing, we use the CMPL benchmark [7], using the framework provided therein to generate 20 adversarial and benign prompt trajectories each'); this is a benchmarking dependency, not a derivation step, and the linear-probe mechanism does not reduce to the truth of [7]. The paper itself flags the main weaknesses in Appendix A: 'The cumulative drift thresholds used to distinguish benign from malicious trajectories were determined either by default (choosing threshold τ=0) or heuristically for the evaluated benchmarks' and 'obtain larger datasets (than the ones used in the paper) with a number of prompt trajectories sufficient for non-trivial train-test splits to ensure robust safety and utility guarantees.' These are statistical robustness concerns (small test sets, heuristic thresholds, possible threshold overfitting), not circularity: no equation or cited result is equivalent by construction to the claimed output. The linear representation hypothesis is imported from external sources [8,21], not from a self-citation chain. Therefore the derivation is self-contained and no circular step is exhibited.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 1 invented entities

NeuroFilter is a supervised classifier, so the probe weights and thresholds are fitted quantities, not derived quantities. The conceptual axioms are the linear representation hypothesis and contextual-integrity labeling. The new construct 'activation velocity' is a definition rather than an independently evidenced entity; its only support is the paper's own small benchmark evaluation. Layer choice and threshold calibration are the main fitted parameters not explicitly emphasized as such.

free parameters (4)
  • Probe weight vector w^ℓ and bias b^ℓ = Learned by logistic regression on labeled activations
    The single-turn filter's decision rule s^ℓ(p)=⟨a^ℓ(p), w^ℓ⟩+b^ℓ is entirely determined by these fitted parameters; all accuracy claims depend on them.
  • Velocity-probe weight vector w^ℓ_vel = Learned by logistic regression on activation-velocity vectors
    Multi-turn cumulative drift C_t is defined as sum of ⟨v_k, w_vel⟩; without this fit there is no velocity signal.
  • Decision threshold τ (single-turn and cumulative-drift) = 0 by default or heuristic/validation-selected
    Appendix A states thresholds are set to 0 or heuristically; the 'zero false positive' result depends on this choice.
  • Layer index ℓ = Selected via validation ('predetermined layer')
    Probe accuracy varies sharply by layer; picking the best layer on validation affects reported performance.
axioms (5)
  • domain assumption Linear representation hypothesis: privacy-violating intent is (approximately) linearly separable in activation space at some layer.
    Adopted from [8,21] and used to justify logistic probe in §4.1; if false, the entire filtering mechanism fails.
  • domain assumption Contextual integrity is the correct normative model of privacy for this task, and the privacy directive ψ is available and correctly specified.
    Labels are generated from ψ; all evaluations presuppose ψ captures the privacy norm. §3.1.
  • domain assumption The user role ρU is available or inferable to define ψ and to label trajectories.
    §3.1: 'the paper assumes that the agent has access to (or can infer) the user role.'
  • ad hoc to paper A fixed token position in the residual stream at turn t captures the conversation state relevant to privacy intent.
    §3.2 defines a^ℓ_t as activation at 'a designated token position'; the paper never specifies the position or validates the choice.
  • ad hoc to paper Cumulative linear drift C_t = Σ_k ⟨v_k, w_vel⟩ is a valid detector of adversarial steering.
    §4.2 postulates this statistic; it is an untested modeling assumption beyond the small benchmark evaluation.
invented entities (1)
  • Activation velocity v^ℓ_t and cumulative drift C_t no independent evidence
    purpose: A per-turn activation displacement statistic meant to expose multi-turn adversarial intent that individual prompts hide.
    Defined in §4.2; the only evidence is the paper's own benchmark results, which are not yet independently replicated and rest on a tiny test set.

pith-pipeline@v1.3.0-alltime-deepseek · 23215 in / 15030 out tokens · 152515 ms · 2026-08-03T09:05:59.011688+00:00 · methodology

0 comments
read the original abstract

Agentic Large Language Models (LLMs) are models able to reason, plan, and execute tools over unstructured data. These abilities are enabling transformative applications in domains spanning from personal assistant, financial, and legal domains. While these systems can substantially improve productivity and service quality, effective agency typically requires access to sensitive personal or organizational information. However, this access introduces critical inference-time privacy risks, specifically regarding contextually appropriate information disclosure. While recent studies highlight the inability of agentic LLMs to consistently adhere to privacy norms, existing defenses often rely on auxiliary LLM-based monitors. However, these defenses are expensive and offer limited protection against attacks that are robust to semantic censorship. To contrast this background, this paper proposes a notion of privacy filters based on activation probing. We show that these filters are both computationally efficient and effective for both single-turn and multi-turn conversational settings. Furthermore, this work provides the first systematic investigation into probing model internals across a conversation trajectory, moving beyond static, single-prompt analysis to capture the evolving state of privacy-sensitive interactions.

Figures

Figures reproduced from arXiv: 2601.14660 by Ferdinando Fioretto, Saswat Das.

Figure 1
Figure 1. Figure 1: Overview of the threat landscape. User inputs (left) pass through the LLM agent (center) which provides a response (right). Four scenarios are considered: (0) Benign Authorized Use, where disclosure of a potentially sensitive attribute is contextually appropriate (e.g., patient details to an authorized doctor), (1) Direct Jailbreaks, where single-turn malicious prompts attempt to bypass safety filters to i… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the NeuroFilter framework: probe training process (left) and inference-time deployment (right). [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: CMPL Insurance (Qwen 2.5 7B Instruct): Comparing probing accuracies for SAEs vs. linear probes. we demonstrate that training SAEs is significantly more expensive than training linear probes in Sec￾tion 6.1.2, while offering a relatively poorer utility tradeoff than the latter. Comparison with Llama Guard Llama Guard, while including privacy violation in the list of haz￾ards it is supposed to filter against… view at source ↗
Figure 4
Figure 4. Figure 4: CMPL Insurance (Single-Turn): Test accuracies (left) and projection scores (right) for GPT OSS 20B. malicious intent regardless of the realization of the privacy risk, these probes provide strong privacy guarantees by not only filtering out prompts that yield leakage, but all that intend to, regardless of their success. Additionally, these probes also succeed in detecting malicious prompts even when the ad… view at source ↗
Figure 5
Figure 5. Figure 5: Cosine similarities between privacy violation direc [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Probing accuracy for probe obtained via su [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: CMPL Insurance: Probing test accuracy (left) and cumulative activation drift (right) for [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Fractured SORRY Bench (Mosaic Attacks): Probing test accuracy (left) and cumulative activation drift (right) [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Probe test accuracies (left) and projection scores (right) for generic harmfulness probe (trained on Wildjailbreak [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: CMPL Insurance: Ablation over different precisions/levels of quantization for [PITH_FULL_IMAGE:figures/full_fig_p014_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: CMPL Insurance: Mean and Max Distances from [PITH_FULL_IMAGE:figures/full_fig_p014_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Train/Test Loss Curves for CMPL Insurance: For [PITH_FULL_IMAGE:figures/full_fig_p020_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Single-Turn Probing: Test accuracies (top) and projection scores (bottom) for CMPL Schedule benchmark for GPT OSS 20B (left), Qwen 2.5 32B Instruct (center), Llama 3.3 70B Instruct (right) 0 10 20 30 40 50 60 Model Layer 0.5 0.6 0.7 0.8 0.9 1.0 Test Accuracy 0 5 10 15 20 25 Model Layer 0.5 0.6 0.7 0.8 0.9 1.0 Test Accuracy 0 10 20 30 40 50 60 70 80 Model Layer 0.5 0.6 0.7 0.8 0.9 1.0 Test Accuracy [PITH_… view at source ↗
Figure 14
Figure 14. Figure 14: PrivacyLens: Linear Probe Accuracies for Qwen 2.5 32B Instruct (left), GPT OSS 20B (center), Llama 3.3 [PITH_FULL_IMAGE:figures/full_fig_p021_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: CMPL Insurance (Qwen 2.5 32B Instruct): Linear probing accuracies (left) and projection scores (right) when model output is modified with ROT3 and not semantically censorable. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Single-Turn Probing: Test accuracies (top) and projection scores (bottom) for CMPL Insurance benchmark for GPT OSS 20B (left), Qwen 2.5 32B Instruct (center), Llama 3.3 70B Instruct (right) 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0 Conversation Turn 0.5 0.6 0.7 0.8 0.9 1.0 Test Accuracy 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0 Conversation Turn 0.5 0.6 0.7 0.8 0.9 1.0 Test Accuracy 2.5 5.0 7.5 10.0 12.5 15.0 17.5 … view at source ↗
Figure 17
Figure 17. Figure 17: CMPL Scheduling: Probing Test Accuracy (top) and Cumulative Drift (bottom) for GPT OSS 20B (left), [PITH_FULL_IMAGE:figures/full_fig_p022_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Trajectories in the CMPL Insurance (left) and Scheduling (right) benchmarks used for activation velocity [PITH_FULL_IMAGE:figures/full_fig_p023_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: CMPL Insurance (Single Turn) - Robustness to Finetuning: Test accuracies (left) and projection scores [PITH_FULL_IMAGE:figures/full_fig_p023_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: CMPL Insurance (Single Turn) - Robustness to Finetuning: Applying Probes Trained on [PITH_FULL_IMAGE:figures/full_fig_p024_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: CMPL Scheduling (Single Turn) - Robustness to Finetuning: Applying Probes Trained on [PITH_FULL_IMAGE:figures/full_fig_p024_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: CMPL Insurance with full AutoDAN Cross-and-Evolve Pipeline: test accuracies (top) and projection scores [PITH_FULL_IMAGE:figures/full_fig_p025_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: CMPL Scheduling with full AutoDAN Cross-and-Evolve Pipeline: test accuracies (top) and projection [PITH_FULL_IMAGE:figures/full_fig_p025_23.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Do Activation Monitors Survive Model Updates? Benchmarking, Predicting, and Repairing Activation-Monitor Staleness

    cs.LG 2026-06 unverdicted novelty 8.0

    Fine-tuning updates frequently stale activation monitors for language model safety while quantization does not, with degradation predictable and repairable via label-free realignment.

  2. MemPrivacy: Privacy-Preserving Personalized Memory Management for Edge-Cloud Agents

    cs.CR 2026-05 unverdicted novelty 6.0

    MemPrivacy replaces privacy-sensitive spans with structured placeholders on edge devices to enable effective cloud memory management while limiting utility loss to 1.6% and outperforming general models on privacy extraction.

  3. MemPrivacy: Privacy-Preserving Personalized Memory Management for Edge-Cloud Agents

    cs.CR 2026-05 unverdicted novelty 6.0

    MemPrivacy uses edge detection of sensitive spans and type-aware placeholders to enable cloud-side memory management for LLM agents without exposing private data, achieving under 1.6% utility loss.

  4. MemPrivacy: Privacy-Preserving Personalized Memory Management for Edge-Cloud Agents

    cs.CR 2026-05 unverdicted novelty 6.0

    MemPrivacy uses edge-side privacy span detection and semantic placeholders to enable cloud memory management for LLM agents while limiting utility loss to 1.6% and outperforming masking baselines.

  5. ClawLess: A Security Model of AI Agents

    cs.CR 2026-04 unverdicted novelty 5.0

    ClawLess introduces a formal fine-grained security model for AI agents with runtime-adaptive policies enforced via user-space kernel and BPF syscall interception.

Reference graph

Works this paper leans on

29 extracted references · 3 linked inside Pith · cited by 3 Pith papers

  1. [1]

    Abdelnabi, A

    S. Abdelnabi, A. Gomaa, E. Bagdasarian, P. O. Kristensson, and R. Shokri. Firewalls to secure dynamic llm agentic networks, 2025

    2025

  2. [2]

    Bagdasaryan, R

    E. Bagdasaryan, R. Yi, S. Ghalebikesabi, P. Kairouz, M. Gruteser, S. Oh, B. Balle, and D. Ramage. Air gap: Protecting privacy-conscious conversational agents.arXiv preprint arXiv:2405.05175, 2024

    Pith/arXiv arXiv 2024

  3. [3]

    Bloom, C

    J. Bloom, C. Tigges, A. Duong, and D. Chanin. Saelens. https://github.com/decoderesearch/SAELens, 2024

    2024

  4. [4]

    Braun, J

    D. Braun, J. Taylor, N. Goldowsky-Dill, and L. Sharkey. Identifying functionally important features with end-to-end sparse dictionary learning, 2024

    2024

  5. [5]

    P. Chao, E. Debenedetti, A. Robey, M. Andriushchenko, F. Croce, V . Sehwag, E. Dobriban, N. Flammarion, G. J. Pappas, F. Tramer, H. Hassani, and E. Wong. Jailbreakbench: An open robustness benchmark for jailbreaking large language models, 2024

    2024

  6. [6]

    Cyberey, Y

    H. Cyberey, Y . Ji, and D. Evans. Unsupervised concept vector extraction for bias control in llms, 2025

    2025

  7. [7]

    S. Das, J. Sandler, and F. Fioretto. Disclosure audits for llm agents, 2025

    2025

  8. [8]

    Elhage, T

    N. Elhage, T. Hume, C. Olsson, N. Schiefer, T. Henighan, S. Kravec, Z. Hatfield-Dodds, R. Lasenby, D. Drain, C. Chen, R. Grosse, S. McCandlish, J. Kaplan, D. Amodei, M. Wattenberg, and C. Olah. Toy models of superposition, 2022

    2022

  9. [9]

    Ghalebikesabi, E

    S. Ghalebikesabi, E. Bagdasaryan, R. Yi, I. Yona, I. Shumailov, A. Pappu, C. Shi, L. Weidinger, R. Stanforth, L. Berrada, P. Kohli, P.-S. Huang, and B. Balle. Operationalizing contextual integrity in privacy-conscious assistants.ArXiv, abs/2408.02373, 2024. 15 APREPRINT

    Pith/arXiv arXiv 2024

  10. [10]

    Glukhov, Z

    D. Glukhov, Z. Han, I. Shumailov, V . Papyan, and N. Papernot. Breach by a thousand leaks: Unsafe information leakage in ’safe’ ai responses. InInternational Conference on Learning Representations, 2024

    2024

  11. [11]

    Y . Guo, Y . Li, and M. Kankanhalli. Involuntary jailbreak: On self-prompting attacks, 2025

    2025

  12. [12]

    Huben, H

    R. Huben, H. Cunningham, L. R. Smith, A. Ewart, and L. Sharkey. Sparse autoencoders find highly interpretable features in language models. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  13. [13]

    H. Inan, K. Upasani, J. Chi, R. Rungta, K. Iyer, Y . Mao, M. Tontchev, Q. Hu, B. Fuller, D. Testuggine, and M. Khabsa. Llama guard: Llm-based input-output safeguard for human-ai conversations, 2023

    2023

  14. [14]

    Jiang and N

    H. Jiang and N. Haghtalab. On surjectivity of neural networks: Can you elicit any behavior from your model?, 2025

    2025

  15. [15]

    Kantamneni, J

    S. Kantamneni, J. Engels, S. Rajamanoharan, M. Tegmark, and N. Nanda. Are sparse autoencoders useful? a case study in sparse probing. InForty-second International Conference on Machine Learning, 2025

    2025

  16. [16]

    Li and R

    Y . Li and R. Eldan. Tinystories: How small can language models be and still speak coherent english, 2024

    2024

  17. [17]

    X. Liu, N. Xu, M. Chen, and C. Xiao. Autodan: Generating stealthy jailbreak prompts on aligned large language models. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  18. [18]

    McKenzie, U

    A. McKenzie, U. Pawar, P. Blandfort, W. Bankes, D. Krueger, E. S. Lubana, and D. Krasheninnikov. Detecting high-stakes interactions with activation probes. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025

    2025

  19. [19]

    Mireshghallah, H

    N. Mireshghallah, H. Kim, X. Zhou, Y . Tsvetkov, M. Sap, R. Shokri, and Y . Choi. Can llms keep a secret? testing privacy implications of language models via contextual integrity theory, 2024

    2024

  20. [20]

    Nissenbaum

    H. Nissenbaum. Privacy as contextual integrity.Washington Law Review, 79(1):119–157, Feb. 2004

    2004

  21. [21]

    K. Park, Y . J. Choe, and V . Veitch. The linear representation hypothesis and the geometry of large language models. InProceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org, 2024

    2024

  22. [22]

    Perez, S

    E. Perez, S. Huang, F. Song, T. Cai, R. Ring, J. Aslanides, A. Glaese, N. McAleese, and G. Irving. Red teaming language models with language models. InConference on Empirical Methods in Natural Language Processing, 2022

    2022

  23. [23]

    Priyanshu and S

    A. Priyanshu and S. Vijay. Fractured-sorry-bench: Framework for revealing attacks in conversational turns undermining refusal efficacy and defenses over sorry-bench (automated multi-shot jailbreaks), 2024

    2024

  24. [24]

    Russinovich, A

    M. Russinovich, A. Salem, and R. Eldan. Great, now write an article about that: The crescendo multi-turn llm jailbreak attack, 2024

    2024

  25. [25]

    Saglam, P

    B. Saglam, P. Kassianik, B. Nelson, S. Weerawardhena, Y . Singer, and A. Karbasi. Large language models encode semantics in low-dimensional linear subspaces, 2025

    2025

  26. [26]

    Y . Shao, T. Li, W. Shi, Y . Liu, and D. Yang. Privacylens: Evaluating privacy norm awareness of language models in action.ArXiv, abs/2409.00138, 2024

    Pith/arXiv arXiv 2024

  27. [27]

    Templeton, T

    A. Templeton, T. Conerly, J. Marcus, J. Lindsey, T. Bricken, B. Chen, A. Pearce, C. Citro, E. Ameisen, A. Jones, H. Cunningham, N. L. Turner, C. McDougall, M. MacDiarmid, C. D. Freeman, T. R. Sumers, E. Rees, J. Batson, A. Jermyn, S. Carter, C. Olah, and T. Henighan. Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet.Transform...

    2024

  28. [28]

    T. Xie, X. Qi, Y . Zeng, Y . Huang, U. M. Sehwag, K. Huang, L. He, B. Wei, D. Li, Y . Sheng, R. Jia, B. Li, K. Li, D. Chen, P. Henderson, and P. Mittal. Sorry-bench: Systematically evaluating large language model safety refusal behaviors, 2024

    2024

  29. [29]

    J. Zhao, J. Huang, Z. Wu, D. Bau, and W. Shi. Llms encode harmfulness and refusal separately, 2025. 16 APREPRINT A Limitations and Future Work While NeuroFilter demonstrates robust efficacy across single-turn and multi-turn settings in filtering inference-time contextual privacy attacks, its reliance on specific model states presents challenges regarding ...

    2025