{"id":"228463f7-e7fd-4c9c-ac73-5e34e78ba30d","arxiv_id":"2501.04527","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A chi-squared distributionally robust optimization reweighting scheme improves worst-class robustness in adversarially trained image classifiers.","lead":"This paper introduces a training method, CODAT, that reweights classes during adversarial training to improve the robust accuracy of the worst-class while keeping the average robust accuracy high. It matters because attackers can exploit uneven class-wise robustness, and CODAT offers a principled, DRO-based way to close that gap.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1's closed-form solution can assign negative or >1 probabilities; the non-negativity constraint is dropped in Appendix A, so Eq. (11) is not proven to equal the DRO objective.","rationale":"The reader's strongest claim is exactly the point I attacked: the deterministic equivalent objective Eq. (11) and its theoretical guarantee. The derivation is standard for chi-squared DRO up to the moment where the non-negativity constraint is dropped. Rewriting the DRO objective as E + sqrt(η Var) is valid only when the unconstrained Lagrange solution remains a probability vector, and the authors never state or verify this. Their Section 4.3 argument that η < K-1 prevents Dirac degeneracy does not prevent a single class from receiving negative probability; the relevant condition is much more restrictive. The numerical example in the attack shows the equality can fail badly, so this is an internal gap in the proof rather than a disagreement with external consensus. The reader's missing-class minibatch issue is valid as an implementation and evaluation concern, but it does not go to the core theorem, so I only partially agree with the reader's weakest-assumption choice. The correct fix is to add a feasibility condition on η or to derive the clamped/truncated solution when the non-negativity constraint binds, and then to verify that condition for every reported configuration. Because this is a fixable gap and the empirical evidence suggests the method may still work when the condition holds, I would keep the paper conditional rather than reject it outright, but the condition must explicitly include the non-negativity check.","tokens_in":24379,"tokens_out":9675,"duration_ms":104700,"concrete_test":"Take a fully trained adversarial model (e.g., the AT baseline or final CODAT model on CIFAR-10), compute per-class robust losses R_k on the full training set using the same PGD attack as in Algorithm 1, and evaluate Eq. (12) with the reported η=0.3. If min_k p_k < 0 or max_k p_k > 1, Eq. (11) cannot be the true DRO objective. Independently, solve the constrained inner maximization max_{p ≥ 0, Σ p = 1, χ²(p||1/K) ≤ η} Σ p_k R_k with a convex optimizer and compare its value to Eq. (10); any mismatch beyond numerical tolerance refutes Theorem 1 as stated.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that Eq. (11) is the exact deterministic equivalent of the min-max DRO objective. In Appendix A the authors introduce L(ξ) = p(ξ)/p0(ξ), impose L(ξ) ≥ 0 and E_P0[L] = 1, but then solve the Lagrangian while ignoring the inequality constraint. The resulting L* in Eq. (A17), and hence p*(ξ) in Eq. (12), is a valid probability distribution only if p*(ξ) ∈ [0,1] for every class. For the uniform P0 this requires -sqrt(Var/η) ≤ R_k - E_P0[R] ≤ (K-1)sqrt(Var/η) for all k. Nothing in the paper verifies this condition, and it can fail even when η is far below the K-1 Dirac threshold used in Section 4.3. For example, with K=3 and H=(0,1,1), η=0.9 gives p*(first class) < 0, while Eq. (10) predicts E_P0[H] + sqrt(η Var) > 1, which exceeds the maximum possible class risk. Thus Eq. (A21) is in general only an upper bound on the true DRO value; the claimed equality and the 'optimal class distribution' with theoretical guarantees are unproved. Section 4.3's no-regularizer argument only rules out a fully Dirac distribution, not the more common failure mode where a few classes receive negative or >1 weight. The reader's missing-class minibatch concern is a real implementation gap, but this non-negativity failure attacks the theoretical foundation itself.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CODAT, a class-wise reweighting scheme for adversarial training based on distributionally robust optimization over the class distribution with a chi-squared divergence ambiguity set. It derives a closed-form worst-case class distribution and a deterministic equivalent objective (Eq. 11) that is used to jointly train the model and class weights. It also introduces a Fairness Elasticity Coefficient (FEC) and reports experiments on CIFAR-10, CIFAR-100, SVHN, and STL-10, showing improved worst-class robust accuracy and favorable FEC values compared with several baselines.","tokens_in":24662,"tokens_out":7994,"duration_ms":78480,"significance":"If the deterministic equivalence were fully established, CODAT would be a meaningful advance: it replaces heuristic class-weight schedules with a principled DRO objective, supplies an explicit formula for the worst-case distribution, and provides experimental evidence of improved worst-class robustness under multiple attacks. The FEC is a reasonable auxiliary metric for the robustness-fairness trade-off, and the experimental comparison covers several datasets, architectures, and attacks. However, the central theoretical claim is currently unsupported because the proof in Appendix A discards the non-negativity constraint on the likelihood ratio, and the empirical evaluation has an unaddressed implementation gap for classes absent from a minibatch. These issues are load-bearing for the paper's main claims.","major_comments":[{"comment":"The proof of Theorem 1 imposes L(ξ) ≥ 0 and E[L]=1, but then solves the Lagrangian without the inequality constraint. The resulting L* in Eq. (A17) can be negative, so p* in Eqs. (12)/(A18) is not a probability distribution in general. For example, with K=3, uniform P0, and H=(0,1,1), η=0.9, one has p*(first class) = (1/3)(1 + sqrt(0.9/(2/9))·(-2/3)) ≈ -0.114, while Eq. (10) gives 2/3 + sqrt(0.9·2/9) ≈ 1.114, which exceeds the largest possible class risk of 1. Thus Eq. (A21) is at best an upper bound on the inner maximization value, not an equality. Section 4.3's argument only rules out a fully Dirac distribution, not the more common failure mode of negative or >1 weights. The theorem needs either an explicit condition on the range of H relative to η and Var, or a correct derivation that handles the non-negativity constraint; Assumption 1 should also exclude Var=0, since Eq. (8) divides by zero in that case.","section":"Appendix A, Eqs. (A17)-(A21)"},{"comment":"The algorithm computes R^rob_ξ as the average robust loss over the samples of class ξ in the minibatch. With batch size 128 and CIFAR-100, a random minibatch typically contains only about 72 of the 100 classes; the remaining classes have no samples, so R^rob_ξ is undefined for them. The manuscript does not state how these classes are handled, even though the empirical mean and variance in Eq. (11) are defined over the uniform class distribution P0. This ambiguity directly affects the validity of the CIFAR-100 results in Table 2 and of any claim that Eq. (11) is the training objective. Please specify the missing-class procedure (e.g., zero-fill, class-balanced minibatch sampling, or excluding absent classes) and, if necessary, rerun the affected experiments.","section":"Algorithm 1, lines 6-9; Sec. 5.1"}],"minor_comments":[{"comment":"The sentence 'As described in Section 4.3, we consider Eq. (4)...' is a self-reference to the current section; it should be reworded.","section":"Sec. 4.3"},{"comment":"The caption of Fig. D1 says 'under CW-30 attack' while the accompanying text says 'under CW-100 attack' in one place and 'under CW-30 attack' in another; these should be reconciled.","section":"Appendix D"},{"comment":"The sentence 'CFOL exhibits the highest FEC across all attack scenarios' is contradicted by Table 2, where CODAT has the highest FEC under PGD-100, CW-30, and AA; the surrounding discussion also switches to comparing with WAT without explaining the change.","section":"Sec. 5.2, CIFAR-100 paragraph"},{"comment":"There are several typographical issues, including 'probatility distribution' in Sec. 4.1 and 'Randon-Nikodym' in Appendix A, and the dataset name is inconsistently spaced as 'CIF AR-10'.","section":"Global"}],"recommendation":"major_revision","confidential_remarks":"The non-negativity gap is serious: the claimed deterministic equivalent is not exact and can exceed the trivial upper bound of 1. If the authors can provide a corrected theorem (e.g., under a checkable condition on class risks) or reframe the method as optimizing a principled upper bound, and fix the minibatch handling for missing classes, the paper may be salvageable. Otherwise, the central theoretical claim is not established."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The empirical story is coherent and the paper is readable, but the central theoretical claim is not proven. The proof of Theorem 1 in Appendix A explicitly defines L(ξ) with L ≥ 0, yet when solving the Lagrangian it drops that inequality and only enforces E[L] = 1. As a result, Eq. (12) can produce negative class weights, and Eq. (A21) is at best an upper bound on the true DRO value, not an exact deterministic equivalent. This is a load-bearing flaw, not a minor gap. For instance, with K=3 and H=(0,1,1), η=0.9 gives a negative weight for the first class, and Eq. (10) can exceed 1, which is impossible for class risk. Section 4.3 only rules out the Dirac extreme; it does not address this more common failure mode.\n\nWhat the paper does well: using chi-squared DRO for class reweighting in adversarial training is a natural extension of CFOL and FAAL, and the deterministic-equivalent objective gives a clean way to update weights and model jointly. The FEC metric is simple and reasonable for reporting the worst-class/average-accuracy trade-off. The experiments cover four datasets, two architectures, and strong attacks, and the worst-class improvements are consistent across settings.\n\nThe soft spots beyond the theorem: Algorithm 1 computes R^rob_ξ as the class-average loss within a minibatch, but with batch size 128 and CIFAR-100 many classes are absent from a minibatch, and the paper never says how those classes are handled. That makes the estimated mean and variance in Eq. (11) potentially different from the intended class-uniform expectations. There is also no released code, no error bars or multi-seed runs, and η appears to be tuned on evaluation sets. On CIFAR-100 the worst-class numbers are tiny (2-4%), so those comparisons are close to noise.\n\nWho this is for: researchers working on robust fairness in adversarial training, especially those interested in a DRO perspective on class reweighting. It deserves a serious referee, but the current version should not be accepted as is. I would ask the authors to fix or reframe Theorem 1, specify the missing-class rule, release code with multi-seed results, and clarify the validation protocol for η.","headline":"A plausible and readable DRO-based reweighting scheme for robust fairness, but the exact-equivalence theorem at the center is unproven because the proof drops the non-negativity constraint on the class weights.","tokens_in":25224,"tokens_out":3190,"would_cite":false,"duration_ms":36670,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Class-wise robust fairness becomes a closed-form distributionally robust objective.","keywords":["adversarial training","robust fairness","class-wise robustness","distributionally robust optimization","chi-squared divergence","worst-class robust accuracy","class reweighting","fairness elasticity coefficient"],"falsifier":"Train CODAT twice on CIFAR-100: once with the minibatch class-average exactly as written in Algorithm 1, and once with class risks estimated from a running full-batch average that is updated every step. If the worst-class robust accuracy advantage over the baselines disappears under the full-batch estimate, the reported gains come from the unstated handling of absent classes rather than from the deterministic equivalent objective in Eq. (11).","tokens_in":24163,"feed_emoji":"🛡️","tokens_out":11348,"duration_ms":102062,"temperature":0.7,"pith_summary":"Adversarially trained networks often protect some classes far better than others, and existing class-reweighting fixes choose their weights using heuristics, with no guarantee that the chosen weights are optimal and no way to update weights and model parameters in the same step. This paper proposes Class Optimal Distribution Adversarial Training (CODAT), which treats class weights as a probability distribution and searches for the worst-case distribution inside a chi-squared divergence ball around the uniform distribution. The inner maximization has a closed-form solution, so the whole training objective becomes a deterministic equivalent: minimize the mean class robust risk plus a square-root-scaled variance term. Training under this objective automatically assigns bigger weights to classes with high adversarial loss, and the paper reports that on CIFAR-10, CIFAR-100, SVHN, and STL-10 this raises worst-class robust accuracy while giving up only a little average robust accuracy. It also introduces a fairness elasticity coefficient to express how much fairness improvement each unit of average-robustness loss buys.","feed_headline":"Lifting worst-class robust accuracy with a closed-form penalty","feed_subtitle":"Adversarial training with a mean-plus-variance class risk improves fairness at small average cost.","key_machinery":"The load-bearing object is the chi-squared divergence ambiguity set $\\Delta$ around the uniform empirical distribution over classes, viewed through the Radon-Nikodym derivative $L(\\xi) = p(\\xi)/p_0(\\xi)$. Solving the convex inner problem by Lagrange multipliers yields the closed-form worst-case distribution and the deterministic equivalent objective $\\min_\\theta \\mathbb{E}_{P_0}[R^{\\mathrm{rob}}_\\xi] + \\sqrt{\\eta\\,\\mathrm{Var}_{P_0}(R^{\\mathrm{rob}}_\\xi)}$, which is what makes joint optimization of weights and parameters possible. The derivation also solves for the optimal Lagrange multiplier, a point the paper contrasts with earlier distributionally robust optimization treatments. A second mechanism is the fairness elasticity coefficient, which converts the worst-class improvement and the average-accuracy drop into a single comparison number for model selection.","core_discovery":"At the center is the min-max problem $\\min_\\theta \\max_{P\\in\\Delta} \\mathbb{E}_P[R^{\\mathrm{rob}}_\\xi]$, where $\\Delta = \\{P : D_{\\chi^2}(P\\|P_0) \\le \\eta\\}$ and $P_0$ is the uniform class distribution. The paper proves Theorem 1: for finite second moments, the worst-case distribution is $p^*(\\xi) = p_0(\\xi) + p_0(\\xi)\\sqrt{\\eta / \\mathrm{Var}_{P_0}(R^{\\mathrm{rob}}_\\xi)}\\,(R^{\\mathrm{rob}}_\\xi - \\mathbb{E}_{P_0}[R^{\\mathrm{rob}}_\\xi])$, and substituting it collapses the inner problem to the deterministic equivalent objective $\\min_\\theta \\mathbb{E}_{P_0}[R^{\\mathrm{rob}}_\\xi] + \\sqrt{\\eta\\,\\mathrm{Var}_{P_0}(R^{\\mathrm{rob}}_\\xi)}$. Because the closed form sits inside the loss, one gradient step moves both the weights and the parameters, which the paper argues is the missing consistency in heuristic reweighting methods. The experiments claim that this objective improves worst-class robust accuracy and lowers the variance of class-wise robust accuracy across datasets and architectures. The paper also defines the fairness elasticity coefficient $\\mathrm{FEC} = e^{\\Delta A_{\\mathrm{wc}}}/e^{\\Delta A}$ to measure the trade-off between worst-class gain and average-accuracy loss.","pith_inferences":["A running full-batch estimate of each class's robust risk would give a version of CODAT that exactly matches the theory; if it preserves the reported worst-class gains, the minibatch approximation in Algorithm 1 is harmless, and if not, the approximation is the active ingredient.","The same mean-plus-$\\sqrt{\\eta \\cdot \\mathrm{variance}}$ penalty is a generic distributionally robust objective, so it could be transplanted to other fairness axes, such as demographic groups, domains, or label subgroups, wherever per-group risk is the random variable being averaged.","Because the closed form is specific to the chi-squared divergence ball, comparing CODAT with the CVaR-based and KL-divergence-based DRO variants already in the paper would isolate how much of the gain comes from the divergence choice rather than from the joint-optimization principle.","The exponential form of FEC is asymmetric, so FEC should be paired with an explicit floor on average robust accuracy before being used to select among methods."],"forward_implications":["Class weights no longer need a separate heuristic: under the closed form, the weight of class $k$ is determined by how far that class's robust risk sits above the class-average risk, scaled by $\\eta$.","Training with Eq. (11) should reduce the variance of class-wise robust accuracy, not merely lift the weakest class; the paper reports the lowest per-class variance among compared methods on CIFAR-10 under PGD-100, CW-30, and AutoAttack.","The hyperparameter $\\eta$ controls the robustness-fairness trade-off monotonically: larger ambiguity sets make the model more conservative and lower average robust accuracy, so the paper chooses $\\eta = 0.3$ for ResNet-18 and $\\eta = 0.8$ for WideResNet-34-10.","The fairness elasticity coefficient gives a scale-free comparison across methods; values above 1 mean the worst-class improvement outruns the average-accuracy decline, and CODAT reports the largest FEC on most attack and dataset combinations.","The worst-class gains grow with model capacity: on WideResNet-34-10 the reported worst-class improvement over baselines is larger than on ResNet-18, consistent with extra capacity being directed to hard classes."],"supporting_citations":[{"why":"Defines the standard adversarial training min-max problem that CODAT extends with a class distribution.","marker":"[7]"},{"why":"Provides the TRADES baseline and the robustness-accuracy trade-off setting used for comparison.","marker":"[13]"},{"why":"Supplies the distributionally robust optimization background and ambiguity-set formulation.","marker":"[21]"},{"why":"Earlier distributionally robust optimization method whose closed-form solution CODAT extends by also solving for the Lagrange multiplier.","marker":"[31]"},{"why":"WAT baseline that focuses on worst-class robustness via the Hedge algorithm and is a direct comparison target.","marker":"[17]"},{"why":"FRL-RWRM baseline for robust fairness through reweighting and remargin.","marker":"[16]"},{"why":"CFOL baseline using distributionally robust optimization with a CVaR ambiguity set, compared against CODAT's chi-squared choice.","marker":"[18]"},{"why":"Documents the robustness-fairness trade-off that motivates the fairness elasticity coefficient.","marker":"[14]"}],"fun_headline_variants":["Closed-form penalty boosts worst-class robust accuracy","Adversarial training with provably optimal class weights","Chi-squared DRO gives closed-form fair robustness","Joint weight-parameter optimization via closed form","From min-max to deterministic: fair robust training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that each class's robust risk is reliably estimated during training, but Algorithm 1 averages losses inside a minibatch, leaving most classes absent on 100-class datasets and never stating how missing classes are handled.","fun_headline_variants_meta":{"raw":{"variants":["Closed-form penalty boosts worst-class robust accuracy","Adversarial training with provably optimal class weights","Chi-squared DRO gives closed-form fair robustness","Joint weight-parameter optimization via closed form","From min-max to deterministic: fair robust training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000209,"raw_usage":{"total_tokens":1494,"prompt_tokens":1117,"completion_tokens":377,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":733,"completion_tokens_details":{"reasoning_tokens":307}},"tokens_in":733,"tokens_out":377,"duration_ms":4270,"temperature":1.0,"reasoning_tokens":307,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:30:30.532523+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train CODAT twice on CIFAR-100: once with the minibatch class-average exactly as written in Algorithm 1, and once with class risks estimated from a running full-batch average that is updated every step. If the worst-class robust accuracy advantage over the baselines disappears under the full-batch estimate, the reported gains come from the unstated handling of absent classes rather than from the deterministic equivalent objective in Eq. (11).","supporting_citations":[{"cited_title":"International Conference on Machine Learn- ing,International Conference on Machine Learning (2019)","cited_arxiv_id":null,"evidence_quote":"Provides the TRADES baseline and the robustness-accuracy trade-off setting used for comparison."},{"cited_title":"In: Pro- ceedings of the AAAI Conference on Artificial Intelligence, vol","cited_arxiv_id":null,"evidence_quote":"WAT baseline that focuses on worst-class robustness via the Hedge algorithm and is a direct comparison target."},{"cited_title":"In: International Confer- ence on Machine Learning, pp","cited_arxiv_id":null,"evidence_quote":"FRL-RWRM baseline for robust fairness through reweighting and remargin."},{"cited_title":"Revisiting adversarial training for the worst-performing class","cited_arxiv_id":"2302.08872","evidence_quote":"CFOL baseline using distributionally robust optimization with a CVaR ambiguity set, compared against CODAT's chi-squared choice."},{"cited_title":"In: NeurIPS 2020 Workshop on Pre-registration in Machine Learning, pp","cited_arxiv_id":null,"evidence_quote":"Documents the robustness-fairness trade-off that motivates the fairness elasticity coefficient."}],"review_version":1}