{"id":"f57ce07d-e9d3-4ffc-b4b0-6771ac61b3fb","arxiv_id":"2507.20061","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A theoretical framework that formalizes the trade-off between curbing distorted content and preserving free speech, with learnability and hardness results and a penalty-based heuristic.","lead":"This paper models content moderation as a game in which users strategically tweak their posts to stay visible while platforms tune a filter to limit harmful trends. It proves the filter-design problem is learnable from finite data but NP-hard to solve exactly, and offers a penalty-based heuristic.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The offline data required by OP(8) are unobservable: true intent x and per-user manipulation cost c are not recoverable from published content, so the practical and sample-complexity claims rest on an unstated observability assumption.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the framework requires knowing each user's true expressive intent and manipulation cost, which the platform does not observe. My stress-test sharpens this from a practical inconvenience to a formal identifiability failure: the map from hidden (x, c) to observable published content is non-injective even with a known incumbent moderator and infinite data. This is more than a missing baseline or an implementation detail; it undermines the paper's central practical claim that finite offline data suffice to approximate the optimal moderator. The theoretical results themselves appear internally coherent conditional on access to oracle (x, c) data: Theorem 1 is a standard uniform-convergence argument, Proposition 3's dimension bounds are plausible, and Theorem 2's reduction, while notationally dense, follows the MAX-FLS hardness template. I did not find a clear internal contradiction in those proofs, though the piecewise-linear class in Proposition 3 uses an OR (union) definition while the derivation assumes projection onto a convex (essentially AND) region; this appears to be a typo or a latent convexity assumption and is secondary to the observability issue. Because the reader already marked the paper CONDITIONAL on this assumption, my finding does not change the verdict; it reinforces it. The concrete analytic check is simple and would settle whether the concern lands: if the authors can exhibit an additional assumption that makes (x, c) identifiable from published posts, or if they restrict the claim to settings where ground-truth intents are available (e.g., drafted but unpublished content, controlled experiments), then the practical claim would need to be scoped accordingly. As written, the paper's 'finite offline data' language promises more than the model supports.","tokens_in":24019,"tokens_out":30777,"duration_ms":380204,"concrete_test":"Analytic check: fix a known linear incumbent moderator f_0(w_0, b_0) and trend direction e. For any observed unfiltered post z* with f_0(z*) ≤ 0, choose any c' > 0 and set x' = z* - e/(2c'). Verify that Δ(x', c'; e, f_0) = z*, since z* + e/(2c') - e/(2c') = z* and f_0(z*) ≤ 0. This demonstrates that the best-response map is non-injective on (x, c), so the inputs to OP(8) cannot be identified from published posts. A complementary computational check: simulate users with known (x_i, c_i), publish z* under f_0, then solve Section 5.2 using z* as a proxy for x and an imputed cost from |z* - x_guess|/|e|. Compare the resulting moderator's true distortion-mitigation and filtered fraction against the oracle optimum computed from the true (x_i, c_i). If the gap is material, the observability concern lands.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central optimization (OP(7) and OP(8)) and all downstream results (Theorem 1, Theorem 2, and the Section 5.2 heuristic) operate on an offline dataset S = {(x_i, c_i)} where x_i is a user's original expressive intent and c_i is that user's manipulation cost. In the model, these are pre-moderation, private quantities. The only observable output is the published post z* = Δ(x_i, c_i; e, f_incumbent), which is the best response to whatever moderator is currently deployed. The paper gives no estimation procedure, no identifiability analysis, and no assumption that connects S to observable posts. In fact, the map from (x, c) to z* is non-injective. For a linear incumbent moderator f_0 and an unfiltered user, the best response is z* = x + e/(2c); hence for any c' > 0, the pair (x', c') with x' = z* - e/(2c') yields exactly the same published z*. Thus x and c are not recoverable from observed content even with infinite data and a known incumbent moderator. The sample-complexity guarantee in Theorem 1 therefore proves learnability from an oracle dataset of true intentions, not from data a platform can actually collect. Consequently, the practical claim that finite offline data suffice to approximate the optimal moderator is unsupported as stated. This does not invalidate the conditional theoretical results, but it makes the contribution to real content moderation depend on an additional, unstated observability assumption that the paper neither states nor defends.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies content moderation as a Stackelberg game in which users strategically adapt content toward a known social trend e subject to a moderation filter, balancing social distortion mitigation (DM) against freedom-of-speech preservation. It defines an optimization problem over moderator classes (OP(7)/OP(8)), proves a generalization/sample-complexity bound for the empirical DM objective (Theorem 1) with bounds for linear and piecewise-linear classes (Proposition 3), proves NP-hardness of exactly optimizing linear moderators (Theorem 2), and proposes a penalty-based heuristic solved by projected gradient descent with a smoothed surrogate loss, evaluated on synthetic data. The paper also draws a conceptual conclusion that welfare-maximizing moderation places the decision boundary where content concentrates.","tokens_in":24420,"tokens_out":8966,"duration_ms":104526,"significance":"If the technical results are repaired, the paper would make a useful contribution to strategic machine learning and welfare-oriented moderation: it formalizes a non-accuracy-based welfare objective for moderation, gives a conditional statistical learning guarantee for linear and piecewise-linear filters, and identifies a nontrivial computational hardness result. The model and closed-form best-response characterization (Proposition 1) are clean, and the appendix contains substantial proof sketches. However, the current version has several load-bearing gaps: the offline data assumption is not justified observably, the main sample-complexity theorem has an undefined range parameter and unstated boundedness assumptions, the NP-hardness reduction has an incomplete converse direction, and the empirical evaluation does not establish approximation quality. These issues are fixable within the manuscript's scope, but they need to be addressed before the central claims can be accepted.","major_comments":[{"comment":"The offline dataset S consists of pairs (x_i, c_i) where x_i is a user's original expressive intent and c_i is a private manipulation cost. These are pre-moderation, unobservable quantities. The only observable quantity is the published best response z*, and the map (x,c) -> z* is non-injective (for an unfiltered user and a linear incumbent, z* = x + e/(2c), so any c' > 0 with x' = z* - e/(2c') yields the same z*). The paper gives no estimation procedure, no identifiability analysis, and no assumption connecting S to observable posts. Consequently, Theorem 1 proves learnability from an oracle dataset of true intentions, not from data a platform can actually collect. The claims that finite offline data suffice for practical content moderation are unsupported unless the paper explicitly states and defends an observability assumption or reframes the result as conditional on access to such oracle data.","section":"Section 3, OP(8); Section 4, Theorem 1"},{"comment":"Theorem 1 states a sample complexity of O(1/epsilon^2 (H^2(PDim(H_F)+ln(1/delta))+VCDim(F))) but H is never defined. The range of h(f;x,c) is also not bounded a priori: D(bot; (x,c),e) = ||e/(2c)||^2, which is unbounded as c approaches 0. The paper assumes X is convex and compact but gives no condition on C beyond drawing c from a distribution. Without an assumption such as c in [c_min, c_max] with c_min > 0, and without defining H as a uniform upper bound on |h| over f in F, the stated bound is vacuous or at best incomplete. This is central because Theorem 1 is the paper's main statistical learnability claim.","section":"Section 4, Theorem 1; Section 2"},{"comment":"The proof of NP-hardness of epsilon-MAX-FLS with mandatory constraints has a gap in the converse direction. After assuming a solution x satisfies at least 3n+m optional constraints and violates at most q=2n+2m mandatory constraints, the text states that x must satisfy all constraints of type (45) and exactly m constraints among (46) and (47). This does not follow: an assignment could satisfy fewer than 3n coverage constraints in (45) while satisfying more than m constraints among (46) and (47) (for example, if x_{m+1} is nonzero and several x_j are zero, both (46) and (47) cannot both hold, but the counting still needs proof). The reduction to Exact 3-Set Cover is therefore incomplete as written, and the computational hardness result of Theorem 2 needs a repaired argument.","section":"Appendix D.2, Theorem 2"},{"comment":"The binary-search method for finding the penalty strength lambda assumes that each instance of OP(14) is solved to global optimality, since Proposition 4 is stated for 'any solution' and the monotonicity comparison uses optimal values of the penalized objective. However, OP(14) is a nonconvex indefinite quadratic program, and the paper's solver is projected gradient descent on a smoothed surrogate. No guarantee is provided that the local solution returned by PGD is a global optimum, so the claim that the platform can 'identify the optimal lambda' by solving O(log n + log(1/delta)) instances is not justified. The monotonicity result is mathematically sound, but its algorithmic application requires either exact solving or a stated approximation guarantee.","section":"Section 5.2, Proposition 4 and binary search"},{"comment":"The experiments do not validate the quality of the heuristic. The paper reports only the heuristic's own objective values and freedom-of-speech index as lambda varies, with no comparison against exact optima on small instances, no random or no-moderation baseline, and no quantification of the gap to the optimal value of OP(15). Additionally, the surrogate loss in Eq. (56) does not agree with the true loss for y > a (filtered users), and no consistency or approximation guarantee is given for minimizing the surrogate. The statement that the proposed technique 'can effectively approximate a solution' is therefore unsupported by the current evidence. The authors should add a small-scale exact or grid-search comparison, or at least report the surrogate-vs-true objective gap on the generated data.","section":"Section 5.3 and Appendix E.1"}],"minor_comments":[{"comment":"There is a sign inconsistency in the definition of H1: h1 is introduced as -(w^T x + b)^2 + (w^T e)^2 tilde{x}_{d+1}^2, but Eq. (24) and the expansion in Eq. (25) write it as +(w^T x + b)^2 - w_{d+1}^2 tilde{x}_{d+1}^2. Since pseudo-dimension is invariant to negating functions when thresholds range over a symmetric interval, this may not change the final bound, but the notation should be corrected for consistency.","section":"Appendix C.1, Eq. (24)-(25)"},{"comment":"The 'key insight' that moderation is most effective when the boundary lies near the content mass is essentially a restatement of the definition of DM in Eq. (5) and the closed form in Eq. (19): the objective explicitly rewards users with -w^T e/(2c) < w^T x + b <= 0. It would be more appropriate to present this as an interpretation of the model rather than a discovered design principle.","section":"Section 1.1 and Definition 2"},{"comment":"The symbol U is used both for the distribution over X x C and for the training sample ('any distribution U on X x C, a training sample U'), which is confusing. Rename the distribution to D or the sample to S.","section":"Section 4, Theorem 1"},{"comment":"The notation E_{(x,c) ~ U} appears as E(x,c)~U in the displayed equation; please fix the typesetting.","section":"Section 3, OP(7)"},{"comment":"The word 'compupted' in the caption is a typo for 'computed'.","section":"Figure 3 caption"}],"recommendation":"major_revision","confidential_remarks":"The paper's central modeling idea is attractive and the theoretical claims are plausible in spirit, but the current version asks the reader to accept several unstated premises: an oracle dataset of true intentions, a bounded range for the DM objective, and a heuristic whose approximation quality is unmeasured. The observability issue is the most important one for the paper's stated practical relevance; if the authors cannot provide an estimation procedure, they should clearly reframe the contribution as conditional mechanism design from oracle data. The proof gap in Theorem 2's reduction and the missing baseline in Section 5.3 are also likely to be raised by other reviewers. I would support publication after these points are addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this paper has a real formal contribution and a real gap. The gap is the one the stress-test note identifies — the offline dataset is an oracle of (x,c), not something a platform can observe. That doesn't kill the theory, but it means the paper's practical claims outrun its model.\n\nWhat's new and worth credit: the social-distortion-mitigation objective (Eq. 5) with the hard free-speech constraint (OP 7) is a genuinely new welfare framing for content moderation, distinct from accuracy-based strategic classification. The sample-complexity bound in Theorem 1 and the NP-hardness for linear moderators in Theorem 2 are the kinds of results that will be useful to people building theory in this area. The closed-form derivation for linear filters (Lemma 1) is clean, and the appendix contains real proof sketches. If those proofs check out, the paper delivers.\n\nNow the soft spots, in order of size. The observability issue is real. OP(8) and Theorem 1 assume samples from the joint distribution of true intent x and manipulation cost c. The platform observes best-response posts z*, and the mapping from (x,c) to z* is non-injective: for any c' you can find an x' that produces the same z*. The paper says 'we assume access to un-manipulated examples' but that's not the same as having x and c. No estimation procedure or identifiability analysis is offered. So the guarantee in Theorem 1 is about generalization from an oracle dataset, not from data a platform can actually collect. This should be stated in the paper, and if the authors want to claim practical relevance they need to add an estimation layer or weaken the claim.\n\nMinor gaps: Theorem 1 uses an unspecified range H and never bounds C, so the bound isn't fully pinned down. The experiments compare λ values but no baseline or exact optimum, and the surrogate loss is justified by a plot rather than analysis. No code. None of these are fatal, but they add up.\n\nOne more caution: the 'optimize by putting the boundary where content clusters' insight follows fairly directly from the DM definition. It's a good practical heuristic, not a surprising theorem.\n\nBottom line: this is a worthwhile theory submission for strategic ML and mechanism design readers. The formal core deserves a serious referee, even though the practical framing needs work. I'd send it to review and ask the authors to confront the observability gap head-on. And I'd take it to a reading group, though I'd want to pair it with the stress-test note.","headline":"New welfare framing for moderation with a solid hardness result, but the offline-data story runs into an observability gap the paper doesn't confront.","tokens_in":24869,"tokens_out":3375,"would_cite":false,"duration_ms":38322,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper frames content moderation as a mechanism-design problem and shows that, for linear and other bounded-complexity filters, a polynomial number of offline samples suffices to approximate the optimal trade-off between reducing…","keywords":["content moderation","strategic classification","mechanism design","social distortion","freedom of speech","sample complexity","pseudo-dimension","NP-hardness"],"falsifier":"Train the model on a dataset where each user's true intent $x$ and manipulation cost $c$ are known, deploy the recommended linear filter, and measure edited posts against the prediction of Proposition 1: benign ideal points $x+e/(2c)$ should appear unchanged, and filtered ideal points should appear at their $\\ell_2$ projection onto the boundary. Systematic mismatch, such as users whose posts are filtered quitting the platform instead of projecting, would invalidate the distortion model. As a purely computational check, brute-force search over small random instances should never beat the relaxed problem's claimed optimum by more than numerical error.","tokens_in":23844,"feed_emoji":"⚖️","tokens_out":8527,"duration_ms":93267,"temperature":0.7,"pith_summary":"The paper tries to establish that the fight between content moderation and user manipulation can be treated as a mechanism-design problem with a well-defined welfare objective. It models each user as moving their post toward a harmful trend by an amount set by their manipulation cost, unless the moderator blocks that move, in which case they stop at the decision boundary. On this model, the best filter balances social distortion (how far users are pulled from their true intent) against freedom of speech (how many users get filtered), and the paper proves that this balance can be approximated from a polynomial number of offline samples for linear and other bounded-complexity filters. It also proves that finding the exact optimal linear filter under a free-speech constraint is NP-hard, motivating the penalty-based approximation the paper implements and tests on synthetic data. A sympathetic reader would care because the result turns a policy debate into a concrete optimization with measurable guarantees and explicit limits.","feed_headline":"Optimal content filters are learnable—and NP-hard to find","feed_subtitle":"Bounded samples approximate the best speech-distortion trade-off; exact linear rules stay NP-hard.","key_machinery":"The load-bearing object is the strategic best response $\\Delta(x,c;e,f)$ of Eq. (2), whose closed form (Proposition 1) converts a policy question into a constrained empirical optimization over distortion-mitigation functions $h(f;x,c)$. Sample efficiency is carried by the pseudo-dimension of the induced class $\\mathcal{H}_{\\mathcal{F}}$ together with the VC-dimension of the filter class $\\mathcal{F}$, bounded via composition lemmas for linear, piecewise-linear, and feature-mapped linear filters; the hardness result is carried by a reduction from maximum feasible linear subsystems with mandatory constraints.","core_discovery":"The central discovery is that the moderation problem has a closed-form user response: each user moves from $x$ to $x + e/(2c)$ if that point is benign, and otherwise projects it onto the decision boundary. The paper then writes the welfare objective as expected distortion mitigation $h(f;x,c)$ subject to a hard free-speech constraint, proves uniform-convergence sample bounds in terms of $\\mathrm{VCDim}(\\mathcal{F})$ and $\\mathrm{PDim}(\\mathcal{H}_{\\mathcal{F}})$, and shows NP-hardness for exact optimal linear filters. Its design conclusion is that the welfare-maximizing moderator places the decision boundary where content concentrates, rather than simply suppressing more content.","pith_inferences":["If manipulation costs must be inferred from observed edits rather than given, the sample-complexity bound would need an extra estimation-error term; the paper assumes costs are known inputs.","A different free-speech metric, such as counting filtered original posts rather than filtered idealized manipulated posts, would change the optimization landscape and could change both the hardness and the design principle.","The \"boundary near content mass\" principle predicts a measurable pattern: incremental shifts of a moderation boundary toward content density should reduce trend-following edits before filter rates rise sharply.","The hardness construction uses homogeneous costs across users in the reduction; heterogeneous costs or low-dimensional structure might create tractable subcases that the paper does not explore."],"forward_implications":["For linear filters, $O(d^2/\\varepsilon^2)$ offline samples (up to log factors) suffice to approximate the optimal distortion/speech trade-off within additive error $\\varepsilon$ with high probability.","No polynomial-time algorithm can find the exact optimal linear moderator under the hard free-speech constraint unless P = NP, even when the constraint is relaxed by a slack $\\epsilon$.","A penalty-based method with $\\lambda$ chosen by binary search produces a family of moderators spanning the trade-off; increasing $\\lambda$ shifts the boundary toward the margin, preserving more speech at the cost of less distortion mitigation.","Distortion mitigation is maximized when the decision boundary sits where content is concentrated, while the free-speech constraint pulls it toward the margin, yielding the inverted-U trade-off seen in the toy example and experiments.","For piecewise-linear filters, learnability degrades exponentially in the number of linear pieces, so the guarantees are practical only for small $m$."],"supporting_citations":[{"why":"Introduces the strategic classification game between a mechanism designer and best-responding agents, the model this paper adapts to content moderation.","marker":"[Hardt et al., 2016]"},{"why":"Provides the PAC-learning formulation of strategic classification and the agent utility model that Eq. (1) specializes.","marker":"[Sundaram et al., 2023]"},{"why":"Supplies the uniform-convergence bound for real-valued function classes on which Theorem 1's sample-complexity guarantee rests.","marker":"[Pollard, 1984]"},{"why":"Sauer-Shelah lemma used in Proposition 3 to bound the pseudo-dimension of composed function classes.","marker":"[Sauer, 1972]"},{"why":"Independent Sauer-Shelah lemma used in the same composition bound.","marker":"[Shelah, 1972]"},{"why":"Gives the pseudo-dimension bound for the linear function class used to upper-bound $\\mathrm{PDim}(\\mathcal{H}_{\\mathcal{F}})$.","marker":"[Anthony et al., 1999]"},{"why":"Establishes NP-hardness of maximum feasible linear subsystems, the problem reduced to in Theorem 2's proof.","marker":"[Amaldi and Kann, 1995]"},{"why":"Provides the surrogate-loss smoothing technique used to make the empirical objective differentiable for gradient descent.","marker":"[Levanon and Rosenfeld, 2021]"},{"why":"Provides the exact penalty function framework used in the approximation heuristic.","marker":"[Han and Mangasarian, 1979]"}],"fun_headline_variants":["Moderation's free speech vs distortion trade-off: NP-hard, but learnable","Free speech vs distortion: exact filtering NP-hard, approximate learnable","Optimal content moderation: NP-hard exactly, learnable approximately","Learn the best moderation filter, despite NP-hard exact rules","Moderators: trade-off is NP-hard, but finite data learns it"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole optimization assumes the platform knows the global trend direction $e$ and each user's manipulation cost $c_i$, and that every user best-responds exactly according to the utility in Eq. (1); if real users' costs or unmanipulated intentions are invisible or their behavior differs, the computed moderator no longer optimizes the stated trade-off.","fun_headline_variants_meta":{"raw":{"variants":["Moderation's free speech vs distortion trade-off: NP-hard, but learnable","Free speech vs distortion: exact filtering NP-hard, approximate learnable","Optimal content moderation: NP-hard exactly, learnable approximately","Learn the best moderation filter, despite NP-hard exact rules","Moderators: trade-off is NP-hard, but finite data learns it"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000359,"raw_usage":{"total_tokens":1895,"prompt_tokens":846,"completion_tokens":1049,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":462,"completion_tokens_details":{"reasoning_tokens":956}},"tokens_in":462,"tokens_out":1049,"duration_ms":10860,"temperature":1.0,"reasoning_tokens":956,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T13:50:34.210844+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the model on a dataset where each user's true intent $x$ and manipulation cost $c$ are known, deploy the recommended linear filter, and measure edited posts against the prediction of Proposition 1: benign ideal points $x+e/(2c)$ should appear unchanged, and filtered ideal points should appear at their $\\ell_2$ projection onto the boundary. Systematic mismatch, such as users whose posts are filtered quitting the platform instead of projecting, would invalidate the distortion model. As a purely computational check, brute-force search over small random instances should never beat the relaxed problem's claimed optimum by more than numerical error.","supporting_citations":[{"cited_title":"Strategic classification","cited_arxiv_id":null,"evidence_quote":"Introduces the strategic classification game between a mechanism designer and best-responding agents, the model this paper adapts to content moderation."},{"cited_title":"Pac-learning for strategic classification","cited_arxiv_id":null,"evidence_quote":"Provides the PAC-learning formulation of strategic classification and the agent utility model that Eq. (1) specializes."},{"cited_title":"On the density of families of sets","cited_arxiv_id":null,"evidence_quote":"Sauer-Shelah lemma used in Proposition 3 to bound the pseudo-dimension of composed function classes."},{"cited_title":"A combinatorial problem; stability and order for models and theories in infinitary languages","cited_arxiv_id":null,"evidence_quote":"Independent Sauer-Shelah lemma used in the same composition bound."},{"cited_title":"Neural network learning: Theoretical foundations, volume 9","cited_arxiv_id":null,"evidence_quote":"Gives the pseudo-dimension bound for the linear function class used to upper-bound $\\mathrm{PDim}(\\mathcal{H}_{\\mathcal{F}})$."},{"cited_title":"The complexity and approximability of finding maximum feasible subsystems of linear relations","cited_arxiv_id":null,"evidence_quote":"Establishes NP-hardness of maximum feasible linear subsystems, the problem reduced to in Theorem 2's proof."},{"cited_title":"Strategic classification made practical","cited_arxiv_id":null,"evidence_quote":"Provides the surrogate-loss smoothing technique used to make the empirical objective differentiable for gradient descent."},{"cited_title":"Exact penalty functions in nonlinear programming","cited_arxiv_id":null,"evidence_quote":"Provides the exact penalty function framework used in the approximation heuristic."}],"review_version":1}