{"id":"6ee4a184-06cc-44f0-9906-6cd89df64a55","arxiv_id":"2509.05627","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"For demographic parity and binary cross-entropy loss, the paper gives a closed-form upper bound for the loss-fairness Pareto frontier, used as a scaling law to audit large models without training them.","lead":"This paper proposes a low-resource method for showing whether a less discriminatory AI model exists in anti-discrimination cases. It derives a closed-form bound for the loss-fairness frontier and uses it to extrapolate from small models to the large contested model.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 4.3's proof uses Lemma 4.1's E_p[S]=E_q[S] condition, which Assumption 4.2 does not state or imply; without it, Equation (4) is not derived.","rationale":"The reader's weakest-assumption analysis identifies the same gap: Lemma 4.1's condition E_p[S] = E_q[S] is required to derive equation (4) through Theorem 4.2, but Assumption 4.2 neither states nor entails it. I checked the proof structure in Appendices C and D and the statement of Assumption 4.2 in Section 4.2. The first condition of Assumption 4.2 is exactly E_q[S] = c1(F,D), which is not the same as E_p[S] = E_q[S]. The second condition is about group-conditional means of fhat and q, and does not control the pointwise log-ratio S under p. Since the proof uses Lemma 4.1 to decompose the loss and the decomposition only holds under E_p[S] = E_q[S], Theorem 4.3 is not proven as stated. This is the central mathematical claim underpinning the paper's scaling-law procedure, so it is load-bearing. The paper's empirical stress tests show that the fitted curve approximates the empirical Pareto frontier, but they do not compare E_p[S] and E_q[S] for the trained models, nor do they test the equality directly; therefore they do not close the proof gap. A repair is conceivable by strengthening Assumption 4.2 and justifying it, but that is a substantive change to the main result rather than a minor clarification, so the appropriate verdict remains rejection of the current manuscript.","tokens_in":33857,"tokens_out":6192,"duration_ms":57075,"concrete_test":"Re-derive Equation (4) from Appendix D while tracking Lemma 4.1's condition, and check whether Assumption 4.2's two stated conditions imply E_p[S] = E_q[S]. Concretely, instantiate the Theorem 4.3 DGP with a simple g (e.g., linear), let F be a small parametric family (such as logistic regression on g(X) with a group offset), train the loss-minimizing fhat on q for two values of ζ, and compute E_p[S] and E_q[S] on a fixed p. If any such fhat satisfies Assumption 4.2's conditions but has E_p[S] ≠ E_q[S], then the proof of Theorem 4.3 as written is invalid; the authors would need to add the missing condition to Assumption 4.2 and justify it separately.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Lemma 4.1 requires E_p[S] = E_q[S] for S(x,a,y) = y log(q(1|x,a)/fhat(x,a)) + (1-y) log((1-q(1|x,a))/(1-fhat(x,a))). This equality is what cancels the distribution-shift terms (5)+(7) in the proof of Lemma 4.1 (Step 2, Appendix C), and Theorem 4.3 invokes Lemma 4.1 at Step 1 of Appendix D. Assumption 4.2 states only two conditions: (i) L(q, fhat) - L(q, q_Bayes) = c1(F,D), constant in q and fhat, and (ii) E[fhat|A=0] - E[q|A=0] = E[fhat|A=1] - E[q|A=1]. Condition (i) pins down E_q[S] but not E_p[S]; condition (ii) is a mean-offset equality between groups and does not constrain the p-weighted log-ratio S. Because p and q differ by the tilt parameter ζ, E_p[S] and E_q[S] are generally unequal even when both conditions of Assumption 4.2 hold. The paper's statement that Theorem 4.2 'implicitly asks that this condition hold' (Section 4.1) is not a substitute for including it in the assumption. Without E_p[S] = E_q[S], equation (4) does not follow from the proof as written. Since the scaling-law application in Section 5 extrapolates from small to large models using exactly this upper bound, the gap is load-bearing, not a cosmetic omission. The synthetic experiments in Section 6 fit curves and do not verify this equality for the trained models, so they do not rescue the theorem as stated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript addresses the resource-information asymmetry confronting claimants who must prove the existence of a less discriminatory alternative (LDA) in US anti-discrimination litigation. The authors recast the LDA inquiry as locating the performance-fairness Pareto frontier (PF) and contribute a closed-form upper bound (Theorem 4.3, Eq. (4)) for the binary cross-entropy loss of Pareto-optimal classifiers as a function of the demographic-parity gap, under a specific data-generating process (A~Ber(pi), X independent of A, Y|X,A ~ Ber(sigma(g(X)-zeta A))) and a model-misspecification-symmetry assumption (Assumption 4.2). They propose using this bound as a scaling law: fit the constants c, c', c'' (claimed to depend only on the test distribution) and B(F,D) on small models, then extrapolate the PF to the contested large model using B(F,D) = C1 + C2/(N^-)^C3 + C2/(D^-)^C4, a form borrowed from language-model scaling laws. Synthetic experiments (Section 6.2, Figures 6 and 10-13) fit the closed form to empirical frontiers of MLPs of four sizes and report that the scaling law holds even when X perpendicular to A is relaxed. The paper is clearly written and unusually candid about its limitations. The central question is whether Theorem 4.3 is proven from the stated assumptions; on reading Appendix D together with Lemma 4.1, I find that the stress-test concern is real: the lemma's condition E_p[S] = E_q[S] is not implied by Assumption 4.2, so Eq. (4) does not follow as written.","tokens_in":34431,"tokens_out":28455,"duration_ms":218250,"significance":"If Theorem 4.3 could be established under a defensible assumption, the contribution would be significant: it would provide, to the best of my knowledge, the first closed-form performance-fairness Pareto frontier for the BCE/demographic-parity setting, and the claim that the shape constants c, c', c'' are independent of model class and dataset size is a sharp, falsifiable prediction that the experiments are designed to test. The legal framing is careful, the idea of replacing 'train an LDA' with 'locate the PF via a scaling law' is genuinely useful, and Section 8's limitations list is honest about the borrowed B-form, the synthetic-only experiments, and the upper-bound nature of the result. These strengths are real. However, the proof gap in Theorem 4.3 and the uncontrolled Taylor remainder in Lemma B.3 mean the central claim is not established as written, and the experiment section as currently reported does not convincingly demonstrate the extrapolation protocol because the fitted constants are not identifiable. Because all three problems are local and fixable within the manuscript's scope, the appropriate recommendation is major revision.","major_comments":[{"comment":"Theorem 4.3 invokes Lemma 4.1 in Step 1 of its proof, but the lemma's statement requires E_{pX,A,Y}[S] = E_{qX,A,Y}[S] for the specific classifier fhat under evaluation, and this condition is not implied by Assumption 4.2 as printed. The first clause of Assumption 4.2, L(q, fhat) - L(q, q_Bayes) = c1(F,D), is equivalent to E_q[S] = c1(F,D) for the classifier that minimizes L(q,·) for that q; applied with q = p it gives E_p[S] = c1(F,D) for the classifier trained on p, which is generally a different model from the Pareto-optimal classifier trained on the tilted distribution q_zeta. The second clause is a first-moment group-offset equality and does not constrain the p-weighted log-ratio S. Consequently E_p[S] = E_{q_zeta}[S] is neither stated nor proven; without it, the terms (5) and (7) in the proof of Lemma 4.1 need not cancel, so equation (4) does not follow from the stated assumptions. The sentence in Section 4.1 that Assumption 4.2 'implicitly asks that this condition hold' does not repair the gap, because the required equality must hold for the same model under two distributions, not merely be constant across the family of fitted models, and the synthetic experiments in Section 6.2 never check the equality for trained models. This is load-bearing, since the Section 5 extrapolation procedure uses exactly this bound.","section":"§4.2 / Appendix D; Assumption 4.2; §4.1"},{"comment":"The claimed remainder bound in Lemma B.3 is not justified. For f(z) = log(1-z), the third derivative f'''(z) = -2/(1-z)^3 is unbounded on (0,1), so the Lagrange remainder (Z-mu)^3/(3(1-xi)^3), with xi between mu and Z, cannot be bounded by a constant times (Z-mu)^3 uniformly; the analogous statement for E[log Z] fails near z = 0. The claimed O(E[(Z-E[Z])^3]) therefore requires an additional assumption such as Z bounded away from 0 and 1. In the DGP of Theorem 4.3, the role of Z is played by q_zeta(1|x,1) = sigma(g(x)-zeta), which is not bounded away from 1 unless g is bounded above, a condition the paper does not impose. As a result, the epsilon term in equation (4) is not established as stated.","section":"Appendix B, Lemma B.3"},{"comment":"The experiments do not demonstrate the small-to-large extrapolation protocol claimed in Section 5. Figures 6, 10, and 12 fit the constants to the empirical frontiers of all four model sizes jointly, which checks whether one set of shape constants explains every size but is not the proposed extrapolation; only Figures 11 and 13 fit to the small-model curve and extrapolate, and they do so qualitatively. The paper itself states that 'there were many possible fits,' and for the same (pi = 0.2, zeta = 0.5) data the reported constants differ materially (Figure 10: C2 = 55, C3 = 0.7, c = 0.16; Figure 11: C2 = 150, C3 = 0.8, c = 0.12; Figure 6: c = 0.0176). This non-identifiability is load-bearing: if the low-resource fits do not pin down the constants, the extrapolated PF at (N+, D+) is not well determined. In addition, the abstract's claim that 'as few as 7 (small) models' suffice to fit the scaling law is not tested anywhere, since the experiments train 300 models per model size.","section":"§6.2, Figures 6, 10-13; §5"}],"minor_comments":[{"comment":"The abstract contains 'need totraina less discriminatory yet high-performing model'; 'totraina' is missing a space.","section":"Abstract"},{"comment":"'Kullback-Liebler' should be 'Kullback-Leibler'.","section":"§4.1"},{"comment":"Assumption 4.2 is repeatedly referred to as 'Theorem 4.2' (Theorem 4.3 statement, Appendix D Step 1, Section 6.2, and the paragraph immediately after the assumption); the numbering should be made consistent.","section":"Throughout"},{"comment":"The scaling-law formula uses C2 for both the N^{-C3} and D^{-C4} terms, whereas the preceding text justifies the form Theta(1/N^alpha + 1/D^beta), which requires independent coefficients; if the shared coefficient is intentional, it should be justified.","section":"§5, Step 3"},{"comment":"Items from Appendix B are called 'Theorem B.1' through 'Theorem B.4' in the proofs of Lemma 4.1 and Theorem 4.3, but they are stated as lemmas.","section":"Appendices C and D"},{"comment":"The proof of Lemma 4.1 uses H to denote the BCE loss, while the rest of the paper uses L; the notation should be unified.","section":"Appendix C"},{"comment":"The caption appears garbled: 'for fixed values of c'' = C7 = 0.9 and c'' = 0.8 while varying c' repeats c'' twice; the captions of Figures 7-9 should be checked.","section":"Figure 7 caption"},{"comment":"The empirical PF is traced with linear scalarization L_BCE + lambda * L_DP, while the theory is built on tilting the distribution q_zeta; the correspondence between lambda and zeta is never established, so the connection between the experimental points and the theorem's assumptions remains indirect.","section":"§6.2"},{"comment":"The discussion states that Assumption 4.2 implies E_q[S] is constant, but the lemma's condition is E_p[S] = E_q[S] for the same classifier; the text should address the stronger condition explicitly (related to major comment 1).","section":"§4.1, 'Understanding the condition on fhat'"}],"recommendation":"major_revision","confidential_remarks":"The main issue is the proof gap in Theorem 4.3; I would ask the authors to either add the E_p[S] = E_q[S] condition to Assumption 4.2 with a plausible justification or modify Lemma 4.1/Theorem 4.3 to carry the residual as an explicit term. The non-identifiability of the fitted constants is more damaging to the paper's practical claim than the current text suggests, and the authors should address it with an identifiability analysis (e.g., bootstrap confidence regions for constants fit only on small models) or temper the extrapolation claims. The Assumption/Theorem 4.2 naming inconsistency and the garbled Figure 7 caption suggest a final polish pass is needed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The problem is real and the framing is smart: the paper recasts the LDA burden as finding the loss-fairness Pareto frontier, then proposes a closed-form bound that a claimant could fit on small models and extrapolate to a large contested model. That would change the resource equation in a practical way. The catch is that the main theorem, as written, is not proven. The proof of Theorem 4.3 invokes Lemma 4.1, which requires E_p[S] = E_q[S] for the tilted distributions q_ζ. Assumption 4.2 gives a constant misspecification loss and a group-offset equality, but neither implies the p/q equality on the log-likelihood ratio S. The paper says the condition is \"implicitly asked\" but it never appears in the assumption. Without it, the distribution-shift terms in the decomposition do not cancel, so equation (4) does not follow from the proof. This is not a cosmetic omission—it is load-bearing.\n\nWhat is genuinely new: to my knowledge no one has given a closed-form expression for a loss-fairness Pareto frontier, and the idea of using it as a scaling law with as few as seven small models is original. The proof techniques—tilting to traverse the frontier, Chebyshev's association inequality for the covariance bound, a Taylor expansion for the KL term—are interesting and mostly coherent. Credit is also due for the legal framing, which is more careful than most ML papers touching Title VII.\n\nWhere the soft spots are: (1) the missing condition in Assumption 4.2, which is the main issue; (2) the scaling law borrows B(F,D) ~ C1 + C2/N^C3 + C2/D^C4 from LLM literature without much justification for classification frontiers; (3) the validation is synthetic only, no code, and the authors admit the fitted constants are not unique, which weakens the \"seven models\" claim. The experiments also do not check whether the missing E_p[S]=E_q[S] equality holds, so they don't rescue the theorem.\n\nWho should read it: researchers in fair ML auditing and people on the legal side of disparate impact. It deserves a serious referee because the idea is novel and the problem matters, but the current version needs major revision. I'd send it out, with the clear expectation that the authors either add the missing assumption honestly or provide a proof that it follows from their DGP.","headline":"A genuinely novel idea for low-resource LDA audits, but the main theorem as written has a load-bearing proof gap that needs fixing before this is citable.","tokens_in":34832,"tokens_out":4503,"would_cite":false,"duration_ms":38598,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper derives a closed-form upper bound on the loss–fairness Pareto frontier that claimants can fit with small models and extrapolate to a large contested model.","keywords":["less discriminatory alternative","Pareto frontier","scaling law","demographic parity","binary cross-entropy loss","AI audit","resource-information asymmetry","fairness-performance trade-off"],"falsifier":"Train a large model on a real dataset where the sensitive attribute correlates with features, trace the empirical Pareto frontier with linear scalarization, fit the seven constants using only small models and a data subsample, and check whether any empirically found Pareto-optimal point of the large model lies above the extrapolated curve; if it does, the claimed upper bound is violated.","tokens_in":33657,"feed_emoji":"⚖️","tokens_out":8450,"duration_ms":67403,"temperature":0.7,"pith_summary":"This paper tackles the burden of proof in \"less discriminatory alternative\" (LDA) claims, where a plaintiff must show that a less discriminatory model with comparable performance exists. It reframes that question as locating the loss–fairness Pareto frontier: if the contested model sits far from the frontier, such an alternative exists. The main result is a closed-form upper bound on the frontier as a function of the fairness gap, with constants that depend on model class, data size, and test distribution. The central move is to read the bound as a scaling law, so a claimant can fit it by training as few as seven small models on a small data subsample and extrapolate the frontier for the contested large model. A sympathetic reader would care because this gives a low-resource, low-information route to evidence in disputes where the model and training data are shielded as trade secrets.","feed_headline":"Seven small models can reveal a big model's fairness frontier","feed_subtitle":"Claimants fit the frontier on small models and scale it to a large contested model without training it.","key_machinery":"The machine is Lemma 4.1, the tilted-loss decomposition: $L(p,\\hat f)=(L(q,\\hat f)-L(q,q^{\\mathrm{Bayes}}))+\\mathbb{E}_{p}[\\mathrm{KL}(p_{Y|X,A}\\Vert q_{Y|X,A})]+L(p,p^{\\mathrm{Bayes}})$, valid when the expected log-likelihood ratio $\\mathbb{E}_p[S]$ equals $\\mathbb{E}_q[S]$. Assumption 4.2 converts the first term into a constant $c_1(F,D)$ and imposes a group-symmetric offset, so a Pareto-optimal classifier can be treated as the Bayes estimator of an artificial distribution $q_\\zeta$; then Chebyshev's association inequality makes the KL term an upper bound instead of an exact expression. The remaining constants $c,c',c''$ depend only on the test distribution, which is what permits the frontier to be fitted at small scale and extrapolated in model size and data.","core_discovery":"Under the data generating process $A\\sim\\mathrm{Ber}(\\pi)$, $X\\perp A$, and $Y|X,A\\sim\\mathrm{Ber}(\\sigma(g(X)-\\zeta A))$, with demographic parity as the fairness measure and binary cross-entropy as the loss, every Pareto-optimal classifier satisfies the upper bound $L(p,\\hat f)\\le B(F,D)-c c'\\log(1-c''+\\Delta)-c(1-c')\\log(c''-\\Delta)$ plus a variance-correction term and an $\\varepsilon$ remainder. The paper derives the bound by decomposing the loss with an artificial tilted distribution $q_\\zeta$, using a symmetry assumption on misspecification loss to replace each Pareto-optimal classifier by the Bayes-optimal classifier of the tilted distribution, and letting a non-negative covariance produce the inequality. Because only $B(F,D)$ depends on the model class and dataset size, and because established neural scaling laws give $B(F,D)\\sim C_1+C_2/N^{C_3}+C_2/D^{C_4}$, the claimant can fit the constants on small models and extrapolate to the contested model without training it. The synthetic experiments relax the independence assumption and still find the fitted curves track empirical Pareto frontiers, which the paper reads as stress-testing support for the scaling law.","pith_inferences":["The split between scale-dependent $B(F,D)$ and shape constants $c,c',c''$ suggests the shape of the loss-fairness frontier may transfer across similar tasks; if that holds, fitting the shape once and adjusting only the scale could cut audit costs further.","The derivation's core is the Lemma 4.1 decomposition, so a parallel treatment of other fairness metrics or loss functions is a plausible extension even though the paper only proves the demographic-parity, binary-cross-entropy case.","The experiments violate the theorem's independence condition yet still fit, hinting that the operative assumption is the constant-misspecification symmetry on Pareto-optimal models rather than $X\\perp A$; this is testable by comparing fits under manipulated covariate-group correlations.","A direct untested extension would apply the scaling law when train and test distributions differ, since the paper's experiments keep them identical; the behaviour of the bounds under distribution shift would decide how broadly the procedure works in practice."],"forward_implications":["A claimant can estimate the Pareto frontier for a large contested model using a small subsample of training data, test data, an estimate of the model's parameter count, and an estimate of its training-data size, without training any large model.","If the contested model's point lies at least $\\delta$ above the extrapolated frontier, an LDA that preserves loss and shrinks the fairness gap by at least $\\delta$ is feasible, turning the legal LDA step into a quantitative computation.","The same formula can be inverted to report the parameter count and data size the defendant would need to reach a given loss-fairness pair, so the claimant can argue that the alternative was within the defendant's reasonable reach.","The estimate is conservative by construction: empirical frontiers use only observed models and Theorem 4.3 is an upper bound, so a claim of $\\delta$-feasibility is a lower bound on the true distance from the frontier.","The procedure is plug-and-play: any improvement in algorithms that trace Pareto-optimal models directly sharpens the fitted frontier and strengthens the claimant's evidence."],"supporting_citations":[{"why":"Supplies Chebyshev's association inequality, which yields the non-negative covariance that makes the KL-bound an upper bound in the proof.","marker":"[13]"},{"why":"Provides the compute-optimal scaling-law form for cross-entropy loss that the paper adopts for the constant $B(F,D)$ as $C_1+C_2/N^{C_3}+C_2/D^{C_4}$.","marker":"[32]"},{"why":"Explains neural scaling laws and supports the $\\Theta(1/N^\\alpha + 1/D^\\beta)$ template for how $B(F,D)$ shrinks with model size and data.","marker":"[7]"},{"why":"Casts the search for less discriminatory alternatives as an optimization problem, the framing this paper extends by making the frontier estimate feasible for large models.","marker":"[24]"},{"why":"Shows fundamental limits and NP-hardness of directly searching for the least discriminatory alternative, motivating the bypass via a scaling-law extrapolation.","marker":"[40]"}],"fun_headline_variants":["Seven small models scale to a big model's fairness frontier","Scaling law for loss-fairness frontier enables low-resource audits","Fairness frontier extrapolated from small models without training target","Closed-form bound yields scaling law for less discriminatory alternatives","Audit without access: 7 small models reveal big model's Pareto frontier"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The bound collapses if a Pareto-optimal classifier does not have the same expected log-likelihood distance to every tilted Bayes-optimal distribution used along the frontier, because the proof requires $\\mathbb{E}_p[S]=\\mathbb{E}_q[S]$ and Assumption 4.2 only states constant misspecification loss and a group-offset equality.","fun_headline_variants_meta":{"raw":{"variants":["Seven small models scale to a big model's fairness frontier","Scaling law for loss-fairness frontier enables low-resource audits","Fairness frontier extrapolated from small models without training target","Closed-form bound yields scaling law for less discriminatory alternatives","Audit without access: 7 small models reveal big model's Pareto frontier"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000713,"raw_usage":{"total_tokens":3316,"prompt_tokens":1164,"completion_tokens":2152,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":780,"completion_tokens_details":{"reasoning_tokens":2065}},"tokens_in":780,"tokens_out":2152,"duration_ms":12049,"temperature":1.0,"reasoning_tokens":2065,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T16:22:36.260567+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a large model on a real dataset where the sensitive attribute correlates with features, trace the empirical Pareto frontier with linear scalarization, fit the seven constants using only small models and a data subsample, and check whether any empirically found Pareto-optimal point of the large model lies above the extrapolated curve; if it does, the claimed upper bound is violated.","supporting_citations":[{"cited_title":"Boucheron, G","cited_arxiv_id":null,"evidence_quote":"Supplies Chebyshev's association inequality, which yields the non-negative covariance that makes the KL-bound an upper bound in the proof."},{"cited_title":"Bahri, E","cited_arxiv_id":null,"evidence_quote":"Explains neural scaling laws and supports the $\\Theta(1/N^\\alpha + 1/D^\\beta)$ template for how $B(F,D)$ shrinks with model size and data."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Casts the search for less discriminatory alternatives as an optimization problem, the framing this paper extends by making the frontier estimate feasible for large models."}],"review_version":2}