On the Limits of Support-Preserving Alignment and Bounded Filtering

Anupam Chattopadhyay; Aryan Dutt; Rui Mao

On the Limits of Support-Preserving Alignment and Bounded Filtering

T0 review · 4 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Bounded safety filters cannot eliminate all harmful outputs that a base model supports, the paper argues.

desk verdict Abstract promises a lower bound on filtering harmful outputs after support-preserving alignment, but the formal claim may collapse to a definitional existence result and the empirical 'floor' is measured on fixed benchmarks. read the letter →

arxiv 2607.18295 v2 pith:HZE3I452 submitted 2026-07-01 cs.LG

Aryan Dutt , Rui Mao , Anupam Chattopadhyay This is my paper

classification cs.LG

keywords alignmentsafetyfilteringharmfuloutputssupportpreservationboundedquerieslargelanguagemodelsharmfloor

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Real-world alignment pipelines aim to make large language models safe, but this paper asks whether they can actually drive harmful behavior to zero rather than just suppressing its most visible forms. It formalizes the setting as support-preserving alignment operators, which keep the set of possible outputs intact, followed by bounded filters that must work under a finite query budget. The central claim is that under these conditions, black-box, white-box, and statistical-query filters all fail to approximate an ideal eliminator that removes all harmful mass. Empirically, across modern open-weight and hosted models, the estimated harmful-output rate drops with more filtering compute but consistently plateaus above zero, indicating a persistent floor. If true, this means safety cannot be bolted on by filtering alone; the base model's support must be addressed.

What carries the argument

The key machinery is the formal pair of a support-preserving alignment operator — a transformation that reshapes output probabilities without changing the set of possible outputs — and a bounded filtering algorithm, defined by a finite query budget. The paper uses this formalization to define the 'ideal eliminator' as the unattainable target that removes all harmful probability mass, and then deploys computational and information-theoretic arguments to show that no bounded filter can approximate it. On the empirical side, the machinery is the estimation of harmful-output rate as a function of filter budget, which yields the observed plateau.

What would settle it

Extend the filter budget a hundredfold beyond the tested range on several models and measure the harmful-output rate; if it falls below the apparent plateau and approaches zero, the paper's empirical floor is falsified.

Watch

Extended reading notes

Core claim

The paper's central discovery is a formal and empirical case that support-preserving alignment plus bounded filtering cannot eliminate all harmful outputs supported by the base distribution. It defines an ideal eliminator as a filter that assigns zero probability to harmful outputs, and asks whether any bounded filtering algorithm — one with a finite query or compute budget — can approximate it under black-box, white-box, or statistical-query access to the model. The authors give computational and information-theoretic arguments that under these constraints, such filters will fail to remove every supported harmful output. They then test this prediction on a range of modern models, applying b

Load-bearing premise

The formal limit rests on assuming that practical alignment preserves the support of the output distribution and that filters have only a finite query budget; the empirical floor additionally assumes the adversarial prompt sets are representative of harmful outputs.

Editorial extensions

If this is right

If support-preserving alignment is the norm, safety filters alone cannot bring harmful-output probability to zero; harm reduction is bounded from below by the base model's support.
Increasing filter budgets yields diminishing returns, so deploying ever-more-compute filters will not close the gap; the floor persists across access models.
Safety must be achieved at training time, by altering the support itself (e.g., removing harmful outputs from the distribution) rather than by post-hoc filtering.
Evaluations that only measure a finite prompt set will see a plateau; that plateau is evidence of a floor, not a measurement artifact under the paper's assumptions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An implicit consequence the authors do not state: preference-based alignment methods that deliberately preserve support (e.g., KL-constrained RLHF) are unlikely to fully sanitize a model; the realistic objective is minimizing harmful mass, not eliminating it.
The framework suggests a trade-off between capability preservation and safety: the only way around the floor is to shrink support, which may reduce the model's general utility — a cost that safety engineering must make explicit.
A testable extension: measure whether support-shrinking interventions (unlearning, targeted fine-tuning, or inference-time constrained decoding) push the plateau to zero, whereas filtering-only pipelines do not.
Another extension: the formal argument implies a rate — the minimal budget needed to bring harm below a given threshold — which could be estimated empirically and compared across models, offering a concrete safety metric.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's my read on arXiv:2607.18295, based only on the abstract. The core question is a good one: after alignment that preserves the support of the base distribution, can a bounded safety filter actually drive harmful output probability to zero? The paper frames this cleanly, with three access models and an empirical plateau across models and filter classes. If the formal part is substantive, it would be a useful reframing of what safety filtering can and cannot promise.

What's genuinely new: the combination of support-preserving alignment with bounded-filter analysis under black-box, white-box, and statistical-query access goes beyond the cited persistence and worst-case-hardness results. The empirical plateau across OpenRouter models is a reasonable sanity check, though not a proof of a fundamental limit.

Where I get cautious: the abstract's own wording is hedged — 'may fail to eliminate' — and the stress-test note points out the risk that the formal result is just an existence claim. If it says 'there exists some bounded filter that misses some harmful output,' that's true by definition under support-preservation with finite queries. It's only interesting if it shows every filter in a natural bounded class leaves nonzero constant harm mass. The abstract doesn't confirm that. On the empirical side, a plateau on curated cybersecurity prompts and PKU-SafeRLHF is a statement about those benchmarks. It doesn't establish that novel harmful outputs can't be caught. Without adaptive or open-ended evaluation, calling it a 'persistent empirical harm floor' overreaches.

I also note there are no error bars or statistical methodology visible in the abstract. That's okay for an abstract, but it means the empirical claim is unverified.

Credit where due: the paper is clearly written at the abstract level, asks a meaningful question, and is honest about the conditional nature of its arguments. The authors are not overclaiming in the abstract as severely as the reader's strong claim suggests; the reader upgraded 'may fail' to 'cannot,' which the abstract does not support.

For a serious referee: I'd send this to review if the full paper proves a universal lower bound for a non-trivial filter class and reports robust experiments with uncertainty quantification. If the proof is just existence, the empirical part might still justify a workshop paper, but not a strong venue claim. I'd want to see the full text before deciding, but the question is important enough to deserve referee time.

Recommendation: engage with the work, but ask for the formal theorem statement and a sensitivity analysis of the plateau before treating it as a law.

Referee Report

4 major / 4 minor

Summary. The paper studies whether alignment schemes that preserve the support of a base model's output distribution, combined with safety filters that are bounded by a finite query/compute budget, can reduce the probability of harmful outputs to zero in LLMs. It formalizes support-preserving alignment operators and bounded filtering under black-box, white-box, and statistical-query access, and claims computational and information-theoretic arguments indicating that, under these constraints, bounded filtering may fail to eliminate all harmful outputs supported by the base distribution. It also reports an empirical evaluation across open-weight and hosted LLMs accessed via OpenRouter, using adversarial prompts from cybersecurity scenarios and PKU-SafeRLHF, and observes that estimated harmful-output rates plateau above zero across models, filter classes, and query budgets. This review is based on the abstract only, as the full text was not available.

Significance. If the formal lower-bound result is nontrivial and the empirical plateau is robust to distribution shift and filter optimization, the paper would constitute a significant negative result for LLM safety: it would suggest a fundamental limit to support-preserving alignment plus bounded filtering as a harm-elimination strategy. The paper's methodological framing has clear strengths: it distinguishes black-box, white-box, and statistical-query access; it couples a formal framework with a multi-model empirical evaluation; and it addresses a practically important gap between worst-case hardness results and the behavior of real pipelines. However, the significance is conditional on the quantifier structure of the claimed theorem and on the external validity of the empirical benchmarks, neither of which can be assessed from the abstract alone.

major comments (4)

[Abstract (formal claim)] The abstract states that 'bounded filtering may fail to eliminate all harmful outputs' under support-preserving alignment and bounded query budgets. This phrasing is existential and risks being tautological: if the support includes harmful outputs and the filter has a finite query budget, then any filter that does not inspect a particular harmful output will fail to eliminate it. To be substantive, the paper must prove a universal lower bound — i.e., that every filter in a natural bounded class (black-box, white-box, statistical-query) leaves a nonzero constant harm mass — and give an explicit lower bound that is independent of the benchmark or filter choice. The abstract does not indicate whether the theorem quantifies over all filters in the class or merely asserts existence of a failing filter. This distinction is load-bearing for the central claim.
[Abstract (empirical evaluation)] The reported 'persistent empirical harm floor' is based on a finite, curated prompt set drawn from cybersecurity scenarios and PKU-SafeRLHF. A plateau on these benchmarks is consistent with the filters failing to generalize to those specific test items, not with a fundamental inability to eliminate novel harmful outputs. To support the strong interpretation of a 'floor,' the paper needs an adaptive or open-ended evaluation — for example, prompt generation that iterates against the filter — and/or a demonstration that the plateau persists under distribution shift. The abstract provides no sample sizes, confidence intervals, or statistical testing, so the plateau claim is currently not statistically grounded.
[Abstract (modeling assumptions)] The formal impossibility result is conditional on two modeling choices: support-preserving alignment operators and bounded filtering with finite query/compute budgets. The abstract does not justify these as faithful models of real deployed alignment. If deployed alignment pipelines substantially shrink the support of harmful outputs, or if safety filtering is effectively unbounded (e.g., via scalable human review or iterative refinement), the impossibility need not apply. The paper should at least discuss evidence that support preservation is characteristic of actual alignment pipelines and that bounded query budgets are the correct formalization of practical filtering. Without this, the formal result may be correct but narrowly scoped.
[Abstract (availability of technical content)] The central technical content is summarized only as 'computational and information-theoretic arguments indicating' the failure result. The abstract provides no theorem statements, proof sketches, or definitions of the operators and access models. Because the validity of the whole paper hinges on the quantifier structure and the tightness of the lower bound, the absence of these details in the available manuscript prevents verification. The referee report is necessarily provisional; if the full text contains the missing details, this concern may be resolved, but as presented, the formal claim cannot be evaluated.

minor comments (4)

[Abstract] The phrase 'may fail' is weaker than the later statement of a 'persistent empirical harm floor.' Clarify whether the formal theorem asserts impossibility for all bounded filters or merely non-guarantee for some; the modal language obscures the logical strength.
[Abstract] The terms 'support-preserving alignment operators,' 'bounded filtering algorithms,' and 'estimated harmful-output rate' are not defined in the abstract. Since the paper is targeting a broad ML audience, a one-sentence formal definition of each would help.
[Abstract] The mention of 'hosted LLMs accessed via OpenRouter' raises questions about black-box versus white-box access: clarify how white-box and statistical-query access were implemented for hosted models, and whether API-side filtering or rate limits affected the results.
[Abstract] The harmful-output classification threshold is a free parameter; its sensitivity should be reported. A floor at one threshold may disappear at another.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found; conditional formal result and empirical plateau are not definitionally forced or fitted-as-prediction.

full rationale

The abstract-only text does not exhibit a circular derivation. The formal setup (support-preserving alignment, bounded filtering, black-box/white-box/statistical-query access) is a conditional modeling framework, not a conclusion smuggled into the assumptions. The conclusion that 'bounded filtering may fail to eliminate all harmful outputs supported by the base model's distribution' is not shown to be a restatement of the setup: a bounded filter could in principle implement a general harmful-output predicate without enumerating the entire support, so the finite query budget alone does not definitionally force failure. The abstract says the paper provides 'computational and information-theoretic arguments' for the failure, which is the substantive content; there are no equations in the abstract showing that the conclusion is assumed. The empirical claim that 'the estimated harmful-output rate decreases with additional filtering compute but consistently plateaus above zero' is a measurement over curated prompts, not a parameter fitted to those same data and then renamed as a prediction. A plateau on a fixed benchmark could be an artifact or a generalization failure, but that is an external-validity concern, not circularity. There are no self-citations in the abstract, and the strongest claim in the provided 'reader's take' upgrades the abstract's hedged 'may fail' to 'cannot,' but that overstatement is not a circularity defect in the paper itself. Therefore no circular step is identifiable from the available text.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

No explicit fitted parameters are disclosed in the abstract; the two listed are modeling/measurement choices that shape the result and the empirical floor. The abstract introduces no new physical or causal entities; support-preserving operators and bounded filters are formal modeling devices, not invented entities. The load-bearing axioms are the support-preservation restriction and the bounded-query access model, both stated as the paper's setting.

free parameters (2)

Bounded filter query budget
The core setting assumes filtering algorithms operate under a finite query/compute budget, but the abstract does not state how budgets are chosen across experiments; the theoretical limits likely quantify over the budget rather than fitting it.
Harmful-output classification threshold
The empirical 'estimated harmful-output rate' requires a classifier or judge to label model outputs; its threshold and calibration are not given in the abstract and would influence where the plateau sits.

assumptions (3)

domain assumption Practical alignment pipelines are adequately modeled by support-preserving operators (operators that do not shrink the support of the base output distribution).
The negative result is stated for this class; if real alignment changes support, the conclusion need not transfer. This is introduced in the abstract's formalization sentence.
domain assumption Bounded filter algorithms under black-box, white-box, and statistical-query access capture the relevant practical filtering architectures.
All theoretical and empirical results are conditional on this access model; other filter architectures could behave differently.
standard math Standard computational/information-theoretic lower-bound toolkit (e.g., query-complexity or information inequalities) is valid.
The abstract claims computational and information-theoretic arguments, which rely on standard complexity/information-theory background not shown in the abstract.

how reviews work

0 comments

Cite this review

Pith. "Pith review of On the Limits of Support-Preserving Alignment and Bounded Filtering." pith.science (2026). https://pith.science/paper/HZE3I452

@misc{pith2026260718295,
  author       = {Pith},
  title        = {Pith review of: On the Limits of Support-Preserving Alignment and Bounded Filtering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HZE3I452}},
  note         = {Machine review of arXiv:2607.18295}
}

read the original abstract

We study whether alignment schemes that reshape a base model's output distribution, combined with bounded safety filters, can drive the probability of harmful behavior to zero in modern large language models. Recent research suggests that harmful behaviors can persist under preference-based alignment and that external filtering can be computationally hard in the worst case, but it remains unclear whether practical alignment pipelines that largely preserve internal representations can eliminate harmful behavior entirely rather than merely suppressing its most visible forms. We formalize this setting using support-preserving alignment operators together with bounded filtering algorithms under black-box, white-box, and statistical-query access, and analyze their ability to approximate an ideal eliminator that removes all harmful mass. Building on this framework, we provide computational and information-theoretic arguments indicating that, under these constraints, bounded filtering may fail to eliminate all harmful outputs supported by the base model's distribution. To evaluate these limits empirically, we analyze a range of state-of-the-art open-weight and hosted LLMs accessed via OpenRouter under bounded black-box, white-box, and statistical-query filters on adversarial prompts drawn from curated cybersecurity scenarios and PKU-SafeRLHF. Across models, filter classes, and query budgets, the estimated harmful-output rate decreases with additional filtering compute but consistently plateaus above zero, suggesting a persistent empirical harm floor.