Pith. sign in

REVIEW 4 major objections 7 minor 2 cited by

Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees

T0 review · 4 major / 7 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A convex smooth surrogate makes two-stage learning-to-defer provably robust to adversarial query-routing attacks, and the paper's SARD algorithm realizes this guarantee.

desk verdict Useful first empirical study of adversarial attacks on two-stage L2D routers, but the paper's headline Bayes-consistency claim for the smooth SARD objective is not supported by the proof as written. read the letter →

arxiv 2502.01027 v4 pith:RB5N6HD5 submitted 2025-02-03 stat.ML cs.LG

Yannis Montreuil , Axel Carlier , Lai Xing Ng , Wei Tsang Ooi This is my paper · ORCID
classification stat.MLcs.LG MSC 68T0568Q3262H30
keywords learning-to-deferadversarialrobustnessBayesconsistencyboundssurrogatelossesqueryallocationattacksmulti-tasklearningconvexoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that two-stage learning-to-defer, where a router decides between a fixed main model and offline experts, can be made robust against adversarial input perturbations that would otherwise misroute queries. It introduces two attack regimes that target the router: untargeted attacks that push a query away from the best agent, and targeted attacks that steer it toward a chosen agent, for instance to inflate a colluding expert's workload. To defend against these, it constructs a convex smooth surrogate family and a regularized empirical algorithm, SARD, and claims that the surrogate losses are both Bayes-consistent and $(\mathcal{R},\mathcal{G})$-consistent in classification, regression, and multi-task settings. If those guarantees hold, adversarial deferral excess risk can be driven to zero as the smooth surrogate excess and the main model's cost gap shrink. The empirical section reports that SARD preserves most clean performance while keeping accuracy or loss nearly unchanged under attacks that cut baseline performance sharply.

What carries the argument

The load-bearing object is the smooth adversarial deferral surrogate (SAD), a convex upper bound on the non-smooth adversarial margin surrogate, built by adding a Lipschitz penalty on score differences across the perturbation ball to a scaled cross-entropy comp-sum term. The inequality $\tilde{\Phi}^{\rho,u,j}_{01}\le \tilde{\Phi}^{\mathrm{smth},u}_{01}$ lets the smooth convex surrogate inherit the margin family's calibration role, while the proof machinery is the $(\mathcal{R},\mathcal{G})$-consistency bound of Lemma 5.9 and Theorem 5.10, with a symmetric, locally $\rho$-consistent rejector class standing in for the usual all-measurable-functions assumption. SARD minimizes this surrogate plus a regularizer, and its per-epoch cost is analysed as $n(1+|A|T_a)(C_{\mathrm{fwd}}+C_{\mathrm{bwd}})$ network traversals.

What would settle it

Take a two-agent problem with a small linear rejector class and compute the gap between $\inf_R \mathbb{E}\tilde{\Phi}^{\mathrm{smth},u}_{\mathrm{def}}(r)$ and $\inf_R \mathbb{E}\tilde{\Phi}^{\rho,u}_{\mathrm{def}}(r)$. If that gap is strictly positive, Corollary 5.11 does not deliver Bayes-consistency for SARD, and a direct training comparison should reveal a regime where the smooth minimizer's adversarial deferral excess stays bounded away from zero while the non-smooth minimizer's goes to zero.

Watch

Extended reading notes

Core claim

The central claim is that the worst-case deferral loss under per-agent perturbations is $\tilde{\ell}_{\mathrm{def}}(r,g,m,z)=\sum_{j=0}^{J}\tau_j(g(x),m(x),z)\,\tilde{\ell}^{j}_{01}(r,x,j)+(1-J)\sum_{j=0}^{J} c_j(g(x),m_j(x),z)$, i.e. the per-agent worst-case misclassification of the router, weighted by the complementary costs $\tau_j$. The paper then claims that the non-smooth adversarial margin surrogates $\tilde{\Phi}^{\rho,u}_{\mathrm{def}}$ are Bayes- and $(\mathcal{R},\mathcal{G})$-consistent for symmetric, locally $\rho$-consistent rejector classes, with a bound in which adversarial deferral excess risk is controlled by $\Psi_u(1)$ times the surrogate excess plus the main model's cost excess. For computation it replaces the non-convex margin surrogate by the smooth convex upper bound $\tilde{\Phi}^{\mathrm{smth},u}_{\mathrm{def}}(r,x,j)=\Phi^{u}_{01}(r/\rho,x,j)+\nu\sup_{x'_j\in B_p(x,\gamma)}\|\Delta r(x'_j,j)-\Delta r(x,j)\|_2$, and it asserts that the regularized empirical minimizer SARD retains the same consistency and minimizability guarantees under the same assumptions. The paper also notes that its non-smooth family fails the differentiability criterion of a general consistency theorem and therefore requires the dedicated analysis it provides.

Load-bearing premise

The conclusion that SARD is Bayes-consistent rests on an unstated equality of minima: the smoothed surrogate used in training and the non-smooth margin surrogate that the theory controls must have the same infimum, or the gap between them must vanish as training proceeds; the paper neither assumes nor proves this.

Editorial extensions

If this is right

  • If the consistency guarantees hold, adversarial training for learning-to-defer reduces to convex empirical risk minimization, so robustness gains do not require non-convex optimization.
  • Targeted routing attacks, such as flooding a colluding expert, are neutralized by the same defense as untargeted misrouting attacks, because both are captured by the per-agent adversarial loss.
  • The consistency bound gives a finite-risk certificate: adversarial deferral excess risk is bounded by a constant times surrogate excess plus the main model's cost gap, so clean and robust objectives do not conflict asymptotically.
  • Empirically, the defense keeps CIFAR-100 accuracy near 50-65 percent under attacks that drop the non-robust router to 13-17 percent, and keeps regression RMSE at 0.17-0.18 where baselines rise to 0.29-0.41.
  • SARD's training cost grows only linearly in the number of agents and PGD depth, so the theoretical robustness is purchasable at a predictable compute premium.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Corollary 5.11 as written leaves a gap: it bounds the smooth surrogate's excess using the non-smooth surrogate's infimum, and without assuming those two infima coincide, minimizing the smooth SAD objective does not by itself guarantee Bayes-consistency of SARD.
  • The attacks assume the adversary knows the clean costs $\tau_j(g(x),m(x),z)$ and router scores; a black-box adversary with only query access would likely be weaker, so the reported robustness numbers may represent an upper bound on attack difficulty.
  • Because SARD's loss is convex in the router's score differences, the same per-agent worst-case formulation could transfer to other routing problems with fixed downstream agents, such as expert selection in large language model pipelines.
  • A direct ablation comparing SARD against the non-smooth margin surrogate on a controlled distribution would reveal whether the smooth approximation loses the consistency edge that the theory attributes to the margin family.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper addresses adversarial robustness in two-stage learning-to-defer (L2D), where a rejector routes each input either to a fixed main model or to one of several frozen experts. The authors introduce untargeted and targeted attack strategies aimed at the rejector, and propose SARD, a convex algorithm based on a smooth adversarial surrogate family. The advertised theoretical contribution is that the proposed surrogate losses are Bayes-consistent and (R,G)-consistent across classification, regression, and multi-task settings; the central results are Theorem 5.10, which establishes an (R,G)-consistency bound for the non-smooth adversarial margin surrogate family, and Corollary 5.11, which claims to extend the guarantees to the smooth surrogate actually minimized by SARD. The experimental section compares SARD with clean-trained L2D baselines on CIFAR-100, California Housing, and Pascal VOC, reporting large robustness gains for SARD under both attack types.

Significance. The problem is timely, and the attack taxonomy (query misrouting and targeted overloading of an expert) is sensible and clearly motivated. The per-agent perturbation model of Lemma 5.2 is a useful formulation, and Proposition 5.13's explicit accounting of the PGD overhead is a welcome, reproducible detail. Theorem 5.10 is a plausible adaptation of the published comp-sum margin-consistency template to the deferral setting, modulo the definitional issues below. However, the headline claim—that the smooth surrogate actually minimized by SARD is provably Bayes-consistent—is not established by the presented argument; the gap is not a missing edge case but a positive constant in simple examples. The experiments are broad and honestly reported but lack any adversarially trained baseline, so they demonstrate vulnerability of clean routers and robustness of SARD relative to no defense, without isolating the contribution of the proposed surrogate family. Nothing in the paper is circular; the issue is a missing proof bridge between the margin surrogate theory and the smooth objective used in practice.

major comments (4)
  1. [Section 5.2, Corollary 5.11] The reviewer concern about this corollary is confirmed and is quantitative, not merely a missing detail. Corollary 5.11 bounds the deferral excess risk by Ψu(1)(E eΦsmth_def − E*eΦρ_def + U) plus a c0-term, but a learner minimizing the SAD objective controls E eΦsmth_def − E*eΦsmth_def, not E eΦsmth_def − E*eΦρ_def. The paper neither states nor proves that the two infima coincide. In fact they differ in general: for two agents with equal conditional costs (τ̄0 = τ̄1 = 1, u = 1, and a rejector constant on the perturbation ball so the Lipschitz term is zero), the pointwise infimum of eΦsmth_def over scores is attained at tied scores with value 2 log 2, while eΦρ_def has infimum log 2, attained in the one-hot limit. Thus at the smooth surrogate's minimizer the right-hand side of Corollary 5.11 retains the positive constant Ψu(1)·log 2, and the text's conclusion that 'SAD satisfies similar consistency guarantees' does not follow. The accompanying assertion in Section 5.2 that SARD 'retains the theoretical guarantees of eΦρ_def' is made without proof. What is needed is either a direct calibration inequality between the smooth surrogate excess and the true deferral excess (along the lines of the smooth-surrogate analysis of Awasthi et al. 2023), or an explicit limit argument showing how the gap vanishes as ρ and ν are sent to appropriate limits; neither is supplied.
  2. [Section 3 (Def. 3.2) and Section 5.1 (Def. 5.4)] The margin surrogates are printed with the score difference reversed: eΦρ,u_01(h,x,y) = Ψu(Σ_{y'≠y} Ψρ(h(x',y') − h(x',y))). Since Ψρ(v) = 1 for v ≤ 0, this surrogate is large when y is the argmax with a large margin, which inverts the intended role of the surrogate (small when j is the router's choice). The proof of Lemma 5.9 in Appendix D.5 uses the opposite convention, r(x',j) − r(x',j'), and Appendix D.3's evaluation 'Ψρ(v) = 0 for v ≥ ρ' applied to v = −Δr is inconsistent with the printed Ψρ. The definitions and proofs must be brought to a single, correct convention (this-minus-other); as printed, a literal implementation of Definition 5.4 would reward the wrong routing decision.
  3. [Section 5.2, Proposition 5.12] Proposition 5.12 defines SARD as a regularized empirical risk minimizer with fixed η > 0 and states that it 'recovers' the guarantees of Theorem 5.10 and Corollary 5.11, but no supporting analysis is given. There is no generalization bound in the sample size K, no statement of how η should scale with K, and no excess-risk decomposition involving the regularizer Ω. For fixed η > 0 the regularizer contributes an irreducible bias, so even a correct calibration inequality for the smooth surrogate would not by itself imply consistency of the SARD objective as executed in Algorithm 1. The authors should either prove a consistency statement for the regularized ERM (e.g., with η = η_K → 0 at a suitable rate) or explicitly restrict the consistency claims to the unregularized population objective and present SARD as an empirically motivated approximation.
  4. [Section 6] The defense experiments compare SARD only with clean-trained L2D baselines (Mao et al. 2023a, 2024d; Montreuil et al. 2025c). Because those baselines employ no adversarial training, the results establish that clean L2D routers are vulnerable and that SARD is more robust than no defense, but they do not measure the incremental value of the proposed smooth margin surrogate and regularizer over, say, PGD training of the standard deferral surrogate. Adding an adversarially trained variant of the baselines would materially strengthen the claim that SARD's specific formulation, rather than the mere presence of inner-maximization training, is responsible for the reported robustness gains.
minor comments (7)
  1. [Tables 2 and 7] The baseline untargeted RMSE is reported as 0.29 ± 0.3 in both the main text and the appendix; if the standard deviation is really 0.3, the degradation relative to clean (0.17 ± 0.01) is not statistically significant, and if the intended value is 0.03, the tables should be corrected.
  2. [Appendix D.5, Proof of Lemma 5.9] The proof is very hard to follow: it uses notations such as r(x, r(x)), {i}^{r*}_{ξ'}, and the sentence 'Then RHS is equal to the above' does not constitute a derivation step; the statement's phrase 'for any distribution P with probabilities p' leaves the role of x implicit in a pointwise inequality. Please rewrite the proof with a consistent ordering notation and explicit step-by-step inequalities.
  3. [Equation (2) and Appendix D.3] The two displays of Ψu disagree: Equation (2) has (1+v)^{1-u} while Appendix D.3 writes (1−v)^{1-u}; please standardize the formula and check all subsequent calculations for consistency with the chosen convention.
  4. [Lemma 5.2] The 'adversarial true deferral loss' contains the constant term (1−J)Σ_j c_j, which is negative for J ≥ 2; noting explicitly that the loss is defined up to an input-independent affine shift would prevent confusion about non-negativity.
  5. [Abstract and Tables 1–3] The abstract contains a typo ('attack strategie--'); additionally, the main-text Tables 1–3 omit the 'Targ. Model' columns that appear in the corresponding appendix tables (Tables 5, 7, 9), and the column naming should be aligned between main text and appendix.
  6. [Proposition 5.13] The peak-memory statement ('a single forward–backward pass plus the storage of one adversarial copy of each input') should be made precise with respect to the mini-batch, since activations are batch-dependent.
  7. [Abstract and Section 5.2] The theoretical claims are framed as holding 'across classification, regression, and multi-task settings,' but Theorem 5.10 is purely a multiclass-over-agents argument; regression enters only through the c0 excess term. The paper should state this reduction explicitly so that readers can judge the actual scope of the consistency guarantees.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the adversarial consistency argument adapts external theorems, and the main concern (Corollary 5.11's infimum mismatch) is a derivation gap, not a reduction of the conclusion to the inputs.

full rationale

The paper's central derivation is not circular. The adversarial margin deferral surrogate eΦρ,u_def and its smooth counterpart eΦsmth,u_def are constructed as upper bounds on the adversarial true deferral loss via standard comp-sum and Lipschitz arguments (Lemmas 5.5–5.7, D.2–D.4), rather than by assuming the consistency conclusion. Theorem 5.10 is proved self-containedly from Lemma 5.9, whose R-consistency bound is derived from the margin structure and the local ρ-consistency assumption; the proof does not presuppose the target guarantee. The only notable issue is Corollary 5.11, which bounds E eΦsmth,u_def − E* eΦρ,u_def rather than E eΦsmth,u_def − E* eΦsmth,u_def; if the two infima differ, minimizing the smooth surrogate can leave a constant gap, so Bayes-consistency of the actual SARD objective is not established. This is a missing condition or derivation gap, not a circular reduction: the bound is a valid inequality built on eΦsmth,u01 ≥ eΦρ,u,j01, but it is applied to the wrong infimum. Self-citations (Montreuil et al. 2025c, 2024, 2025a, 2025b) are used for the two-stage L2D formulation, baselines, and related work, but the load-bearing consistency theorems rely on external works (Awasthi et al. 2022, 2023; Mao et al. 2023b; Wang & Scott 2023), which do not assume the present result. No fitted parameter is renamed as a prediction; ρ, ν, and η are tunable hyperparameters and do not enter the consistency claim as fitted values. Overall, the derivation chain is independent of its inputs, with a genuine but non-circular theoretical gap in the smooth-surrogate guarantee.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central proof imports the multiclass H-consistency machinery from Awasthi et al. and Mao et al.; it adds a per-agent distributional weighting and a smooth regularization term. The load-bearing assumptions are the local rho-consistency and symmetry of the rejector class and an unstated equality of the infima of the smooth and non-smooth surrogate families. No new physical entities are introduced.

free parameters (4)
  • rho (margin hyperparameter) = 1 in all experiments
    Controls the margin in the comp-sum surrogate and appears in the consistency bound and in local rho-consistency; selected via cross-validation in the prior works it follows.
  • nu (smoothness weight) = 0.01 (CIFAR-100, Pascal VOC), 0.05 (California Housing)
    Balances clean allocation accuracy against the adversarial robustness penalty; selected via cross-validation.
  • eta (regularizer weight) = L2 weight decay 1e-4
    Regularizer in the SARD objective; reported in the experiments but not tied to the theoretical guarantees.
  • PGD steps Ta = 40 (CIFAR-100), 10 (California Housing), 20 (Pascal VOC)
    Inner maximization depth for evaluating the smooth adversarial term; chosen per task and affects the reported robustness.
assumptions (5)
  • domain assumption The rejector hypothesis class R is symmetric and locally rho-consistent (Definition 5.8).
    Needed for Lemma 5.9 and Theorem 5.10; without it the R-consistency bounds do not hold.
  • ad hoc to paper The adversarial true deferral loss may use a different perturbation for each agent j (Lemma 5.2).
    This per-agent supremum is an upper bound on a single-perturbation adversary; the attacks in Section 4 use one perturbation, so the defense is trained against a stronger, different threat model.
  • ad hoc to paper The infimum of the smooth surrogate equals the infimum of the non-smooth margin surrogate, or the gap vanishes at the minimizer.
    Corollary 5.11 transfers consistency from ePhi_rho to ePhi_smth only if E*Phi_smth - E*Phi_rho goes to zero; the paper does not state or prove this.
  • domain assumption Experts and the main model are frozen and their outputs and costs are unaffected by adversarial perturbations; only the router is attacked.
    This is the two-stage setting and is stated in Section 4; it excludes attacks that simultaneously corrupt expert inputs.
  • standard math Standard multiclass consistency results for comp-sum and rho-margin surrogates carry over to the deferral setting.
    Used in the proofs of Lemma 5.9 and Theorem 5.10, following Awasthi et al. (2022) and Mao et al. (2023b).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees." pith.science (2026). https://pith.science/paper/RB5N6HD5

@misc{pith2026250201027,
  author       = {Pith},
  title        = {Pith review of: Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RB5N6HD5}},
  note         = {Machine review of arXiv:2502.01027}
}
abstract

Two-stage Learning-to-Defer (L2D) enables optimal task delegation by assigning each input to either a fixed main model or one of several offline experts, supporting reliable decision-making in complex, multi-agent environments. However, existing L2D frameworks assume clean inputs and are vulnerable to adversarial perturbations that can manipulate query allocation--causing costly misrouting or expert overload. We present the first comprehensive study of adversarial robustness in two-stage L2D systems. We introduce two novel attack strategie--untargeted and targeted--which respectively disrupt optimal allocations or force queries to specific agents. To defend against such threats, we propose SARD, a convex learning algorithm built on a family of surrogate losses that are provably Bayes-consistent and $(\mathcal{R}, \mathcal{G})$-consistent. These guarantees hold across classification, regression, and multi-task settings. Empirical results demonstrate that SARD significantly improves robustness under adversarial attacks while maintaining strong clean performance, marking a critical step toward secure and trustworthy L2D deployment.

Figures

Figures reproduced from arXiv: 2502.01027 by the authors.

Figure 1
Figure 1. Learning-to-Defer in inference. The input x is routed to the most cost-effective agent in the system by the rejector r ∈ R: in this case, the human expert. B.2. Untargeted Attack Router Input Poor classifier Human expert Allocate to the poor classifier Malicious Attacker Adversarial input Patient undergoing X-ray [PITH_FULL_IMAGE:figures/full_fig_p015_1.png] view at source ↗
Figure 2
Figure 2. Untargeted Attack: The malicious attacker perturbs the input to increase the probability that the query is assigned to a less accurate expert, thereby maximizing classification errors. Rather than targeting a specific expert, the attack injects adversarial noise to disrupt the expert allocation process, leading to erroneous routing and degraded decision-making. B.3. Targeted Attack Input Poor classifier Adversarial … view at source ↗
Figure 3
Figure 3. Targeted Attack: The malicious attacker perturbs the input to increase the probability that the query is assigned to its associated agent. By manipulating the L2D system to systematically route queries to this associate, the adversary ensures that the associate receives a higher volume of queries, thereby increasing its earnings. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Adversarial Robustness in One-Stage Learning-to-Defer

    stat.ML 2025-10 unverdicted novelty 7.0 of 10

    New adversarial surrogate losses and claimed consistency guarantees for one-stage learning-to-defer in classification and regression, with experiments suggesting improved robustness.

  2. Universal Model Routing for Efficient LLM Inference

    cs.CL 2025-02 conditional novelty 5.0 of 10

    UniRoute represents each language model by its error rates on a few prompt clusters, letting a router choose among models it has never seen during training.

Reference graph

Works this paper leans on

63 extracted references · 53 canonical work pages · cited by 2 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    and Mian, A

    Akhtar, N. and Mian, A. Threat of adversarial attacks on deep learning in computer vision: A survey. Ieee Access, 6: 0 14410--14430, 2018

    work page 2018

  3. [3]

    Calibration and consistency of adversarial surrogate losses

    Awasthi, P., Frank, N., Mao, A., Mohri, M., and Zhong, Y. Calibration and consistency of adversarial surrogate losses. In Ranzato, M., Beygelzimer, A., Dauphin, Y. N., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 202...

    work page 2021

  4. [4]

    Multi-class h-consistency bounds

    Awasthi, P., Mao, A., Mohri, M., and Zhong, Y. Multi-class h-consistency bounds. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA, 2022. Curran Associates Inc. ISBN 9781713871088

    work page 2022

  5. [5]

    Theoretically grounded loss functions and algorithms for adversarial robustness

    Awasthi, P., Mao, A., Mohri, M., and Zhong, Y. Theoretically grounded loss functions and algorithms for adversarial robustness. In Ruiz, F. J. R., Dy, J. G., and van de Meent, J. (eds.), International Conference on Artificial Intelligence and Statistics, 25-27 April 2023, Palau de Congressos, Valencia, Spain, volume 206 of Proceedings of Machine Learning ...

    work page 2023

  6. [6]

    Calibrated surrogate losses for adversarially robust classification

    Bao, H., Scott, C., and Sugiyama, M. Calibrated surrogate losses for adversarially robust classification. In Abernethy, J. D. and Agarwal, S. (eds.), Conference on Learning Theory, COLT 2020, 9-12 July 2020, Virtual Event [Graz, Austria] , volume 125 of Proceedings of Machine Learning Research, pp.\ 408--451. PMLR , 2020

    work page 2020

  7. [7]

    Convexity, classification, and risk bounds

    Bartlett, P., Jordan, M., and McAuliffe, J. Convexity, classification, and risk bounds. Journal of the American Statistical Association, 101: 0 138--156, 2006. doi:10.1198/016214505000000907

    doi:10.1198/016214505000000907 2006

  8. [8]

    Bartlett, P. L. and Wegkamp, M. H. Classification with a reject option using a hinge loss. Journal of Machine Learning Research, 9 0 (8), 2008

    work page 2008

Show all 63 references
  1. [9]

    Evasion Attacks against Machine Learning at Test Time, pp.\ 387–402

    Biggio, B., Corona, I., Maiorca, D., Nelson, B., Šrndić, N., Laskov, P., Giacinto, G., and Roli, F. Evasion Attacks against Machine Learning at Test Time, pp.\ 387–402. Springer Berlin Heidelberg, 2013. ISBN 9783642387098. doi:10.1007/978-3-642-40994-3_25

    2013 doi

  2. [10]

    In defense of softmax parametrization for calibrated and consistent learning to defer

    Cao, Y., Mozannar, H., Feng, L., Wei, H., and An, B. In defense of softmax parametrization for calibrated and consistent learning to defer. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems 36...

    2023

  3. [11]

    A survey on adversarial attacks and defences

    Chakraborty, A., Alam, M., Dey, V., Chattopadhyay, A., and Mukhopadhyay, D. A survey on adversarial attacks and defences. CAAI Transactions on Intelligence Technology, 6 0 (1): 0 25--45, 2021

    2021

  4. [12]

    A., and Samadi, S

    Charusaie, M., Mozannar, H., Sontag, D. A., and Samadi, S. Sample efficient learning of predictors that complement humans. In Chaudhuri, K., Jegelka, S., Song, L., Szepesv \' a ri, C., Niu, G., and Sabato, S. (eds.), International Conference on Machine Learning, ICML 2022, 17-...

    2022

  5. [13]

    On optimum recognition error and reject tradeoff

    Chow, C. On optimum recognition error and reject tradeoff. IEEE Transactions on information theory, 16 0 (1): 0 41--46, 2003

    2003

  6. [14]

    Learning with rejection

    Cortes, C., DeSalvo, G., and Mohri, M. Learning with rejection. In Ortner, R., Simon, H. U., and Zilles, S. (eds.), Algorithmic Learning Theory, pp.\ 67--82, Cham, 2016. Springer International Publishing. ISBN 978-3-319-46379-7

    2016

  7. [15]

    Cardinality-aware set prediction and top- k classification

    Cortes, C., Mao, A., Mohri, C., Mohri, M., and Zhong, Y. Cardinality-aware set prediction and top- k classification. Advances in neural information processing systems, 37: 0 18265--18309, 2024

    2024

  8. [16]

    Balancing the scales: A theoretical and algorithmic framework for learning from imbalanced data

    Cortes, C., Mao, A., Mohri, M., and Zhong, Y. Balancing the scales: A theoretical and algorithmic framework for learning from imbalanced data. In Forty-second International Conference on Machine Learning, 2025

    2025

  9. [17]

    The pascal visual object classes (voc) challenge

    Everingham, M., Van Gool, L., Williams, C., Winn, J., and Zisserman, A. The pascal visual object classes (voc) challenge. International Journal of Computer Vision, 88: 0 303--338, 2010. doi:10.1007/s11263-009-0275-4

    2010 doi

  10. [18]

    and El - Yaniv, R

    Geifman, Y. and El - Yaniv, R. Selective classification for deep neural networks. In Guyon, I., von Luxburg, U., Bengio, S., Wallach, H. M., Fergus, R., Vishwanathan, S. V. N., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 30: Annual Conference on N...

    2017

  11. [19]

    Ghosh, A., Kumar, H., and Sastry, P. S. Robust loss functions under label noise for deep neural networks. In Singh, S. P. and Markovitch, S. (eds.), Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA ,...

    2017

  12. [20]

    J., Shlens, J., and Szegedy, C

    Goodfellow, I. J., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. In Bengio, Y. and LeCun, Y. (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings , 2015

    2015

  13. [21]

    A., and Kohli, P

    Gowal, S., Qin, C., Uesato, J., Mann, T. A., and Kohli, P. Uncovering the limits of adversarial training against norm-bounded adversarial examples. ArXiv preprint, abs/2010.03593, 2020

    2010 arXiv

  14. [22]

    Deep residual learning for image recognition

    He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. corr abs/1512.03385 (2015), 2015

    2015 arXiv

  15. [23]

    G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H

    Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. Mobilenets: Efficient convolutional neural networks for mobile vision applications, 2017

    2017

  16. [24]

    and Barry, R

    Kelley Pace , R. and Barry, R. Sparse spatial autoregressions. Statistics and Probability Letters, 33 0 (3): 0 291--297, 1997. ISSN 0167-7152. doi:https://doi.org/10.1016/S0167-7152(96)00140-X

    1997 doi

  17. [25]

    Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In Bengio, Y. and LeCun, Y. (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings , 2015

    2015

  18. [26]

    Learning multiple layers of features from tiny images, 2009

    Krizhevsky, A. Learning multiple layers of features from tiny images, 2009

    2009

  19. [27]

    Long, P. M. and Servedio, R. A. Consistency versus realizable h-consistency for multiclass classification. In Proceedings of the 30th International Conference on Machine Learning, ICML 2013, Atlanta, GA, USA, 16-21 June 2013 , volume 28 of JMLR Workshop and Conference Proceedi...

    2013

  20. [28]

    Madras, D., Pitassi, T., and Zemel, R. S. Predict responsibly: Improving fairness and accuracy by learning to defer. In Bengio, S., Wallach, H. M., Larochelle, H., Grauman, K., Cesa - Bianchi, N., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 31: An...

    2018

  21. [29]

    Towards deep learning models resistant to adversarial attacks

    Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings . ...

    2018

  22. [30]

    Theory and Algorithms for Learning with Multi-Class Abstention and Multi-Expert Deferral

    Mao, A. Theory and Algorithms for Learning with Multi-Class Abstention and Multi-Expert Deferral. PhD thesis, New York University, 2025

    2025

  23. [31]

    Two-stage learning to defer with multiple experts

    Mao, A., Mohri, C., Mohri, M., and Zhong, Y. Two-stage learning to defer with multiple experts. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Pr...

    2023

  24. [32]

    Cross-entropy loss functions: Theoretical analysis and applications

    Mao, A., Mohri, M., and Zhong, Y. Cross-entropy loss functions: Theoretical analysis and applications. In Krause, A., Brunskill, E., Cho, K., Engelhardt, B., Sabato, S., and Scarlett, J. (eds.), International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu...

    2023

  25. [33]

    H-consistency bounds: Characterization and extensions

    Mao, A., Mohri, M., and Zhong, Y. H-consistency bounds: Characterization and extensions. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processin...

    2023

  26. [34]

    h -consistency guarantees for regression

    Mao, A., Mohri, M., and Zhong, Y. h -consistency guarantees for regression. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F. (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Procee...

    2024

  27. [35]

    Multi-label learning with stronger consistency guarantees

    Mao, A., Mohri, M., and Zhong, Y. Multi-label learning with stronger consistency guarantees. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J. M., and Zhang, C. (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neura...

    2024

  28. [36]

    Realizable h-consistent and bayes-consistent loss functions for learning to defer

    Mao, A., Mohri, M., and Zhong, Y. Realizable h-consistent and bayes-consistent loss functions for learning to defer. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J. M., and Zhang, C. (eds.), Advances in Neural Information Processing Systems 38: An...

    2024

  29. [37]

    Regression with multi-expert deferral

    Mao, A., Mohri, M., and Zhong, Y. Regression with multi-expert deferral. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 . OpenReview.net, 2024 d

    2024

  30. [38]

    A universal growth rate for learning with smooth surrogate losses

    Mao, A., Mohri, M., and Zhong, Y. A universal growth rate for learning with smooth surrogate losses. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J. M., and Zhang, C. (eds.), Advances in Neural Information Processing Systems 38: Annual Conference ...

    2024

  31. [39]

    Mastering multiple-expert routing: Realizable \ h\ -consistency and strong guarantees for learning to defer

    Mao, A., Mohri, M., and Zhong, Y. Mastering multiple-expert routing: Realizable \ h\ -consistency and strong guarantees for learning to defer. In Forty-second International Conference on Machine Learning, 2025 a

    2025

  32. [40]

    Enhanced \ h\ -consistency bounds

    Mao, A., Mohri, M., and Zhong, Y. Enhanced \ h\ -consistency bounds. In 36th International Conference on Algorithmic Learning Theory, 2025 b

    2025

  33. [41]

    Principled algorithms for optimizing generalized metrics in binary classification

    Mao, A., Mohri, M., and Zhong, Y. Principled algorithms for optimizing generalized metrics in binary classification. In Forty-second International Conference on Machine Learning, 2025 c

    2025

  34. [42]

    Towards consistency in adversarial classification

    Meunier, L., Ettedgui, R., Pinot, R., Chevaleyre, Y., and Atif, J. Towards consistency in adversarial classification. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on N...

    2022

  35. [43]

    Foundations of machine learning

    Mohri, M., Rostamizadeh, A., and Talwalkar, A. Foundations of machine learning. MIT Press, 2012

    2012

  36. [44]

    H., Carlier, A., Ng, L

    Montreuil, Y., Yeo, S. H., Carlier, A., Ng, L. X., and Ooi, W. T. Optimal query allocation in extractive qa with llms: A learning-to-defer framework with theoretical guarantees, 2024

    2024

  37. [45]

    X., and Ooi, W

    Montreuil, Y., Carlier, A., Ng, L. X., and Ooi, W. T. Why ask one when you can ask k ? two-stage learning-to-defer to the top- k experts, 2025 a

    2025

  38. [46]

    X., and Ooi, W

    Montreuil, Y., Carlier, A., Ng, L. X., and Ooi, W. T. One-stage top- k learning-to-defer: Score-based surrogates with theoretical guarantees, 2025 b

    2025

  39. [47]

    S., Carlier, A., Ng, L

    Montreuil, Y., Heng, Y. S., Carlier, A., Ng, L. X., and Ooi, W. T. A two-stage learning-to-defer approach for multi-task learning. In Forty-second International Conference on Machine Learning, 2025 c

    2025

  40. [48]

    and Sontag, D

    Mozannar, H. and Sontag, D. A. Consistent estimators for learning to defer to an expert. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event , volume 119 of Proceedings of Machine Learning Research, pp.\ 7076--7087...

    2020

  41. [49]

    Mozannar, H., Lang, H., Wei, D., Sattigeri, P., Das, S., and Sontag, D. A. Who should predict? exact algorithms for learning to defer to humans. In Ruiz, F. J. R., Dy, J. G., and van de Meent, J. (eds.), International Conference on Artificial Intelligence and Statistics, 25-27...

    2023

  42. [50]

    K., Rawat, A

    Narasimhan, H., Jitkrittum, W., Menon, A. K., Rawat, A. S., and Kumar, S. Post-hoc estimators for learning to defer to an expert. In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), Advances in Neural Information Processing Systems 35: Annual Con...

    2022

  43. [51]

    Ohn Aldrich, R. A. Fisher and the making of maximum likelihood 1912-1922. Statistical Science, 12 0 (3): 0 162--179, 1997

    1912

  44. [52]

    B., and Sun, J

    Ren, S., He, K., Girshick, R. B., and Sun, J. Faster R-CNN: towards real-time object detection with region proposal networks. In Cortes, C., Lawrence, N. D., Lee, D. D., Sugiyama, M., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 28: Annual Conferen...

    2015

  45. [53]

    How to compare different loss functions and their risks

    Steinwart, I. How to compare different loss functions and their risks. Constructive Approximation, 26: 0 225--287, 2007

    2007

  46. [54]

    Towards human- AI collaboration in healthcare: Guided deferral systems with large language models

    Strong, J., Men, Q., and Noble, A. Towards human- AI collaboration in healthcare: Guided deferral systems with large language models. In ICML 2024 Workshop on LLMs and Cognition, 2024

    2024

  47. [55]

    J., and Fergus, R

    Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I. J., and Fergus, R. Intriguing properties of neural networks. In Bengio, Y. and LeCun, Y. (eds.), 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2...

    2014

  48. [56]

    and Bartlett, P

    Tewari, A. and Bartlett, P. L. On the consistency of multiclass classification methods. In Auer, P. and Meir, R. (eds.), Learning Theory, 18th Annual Conference on Learning Theory, COLT 2005, Bertinoro, Italy, June 27-30, 2005, Proceedings , volume 3559 of Lecture Notes in Com...

    2005 doi

  49. [57]

    Verma, R., Barrej \' o n, D., and Nalisnick, E. T. Learning to defer to multiple experts: Consistent surrogate losses, confidence calibration, and conformal ensembles. In Ruiz, F. J. R., Dy, J. G., and van de Meent, J. (eds.), International Conference on Artificial Intelligenc...

    2023

  50. [58]

    and Scott, C

    Wang, Y. and Scott, C. On classification-calibration of gamma-phi losses. In Neu, G. and Rosasco, L. (eds.), Proceedings of Thirty Sixth Conference on Learning Theory, volume 195 of Proceedings of Machine Learning Research, pp.\ 4929--4951. PMLR, 2023

    2023

  51. [59]

    and Watkins, C

    Weston, J. and Watkins, C. Multi-class support vector machines. Technical report, Citeseer, 1998

    1998

  52. [60]

    and Agarwal, S

    Zhang, M. and Agarwal, S. Bayes consistency vs. h-consistency: The interplay between surrogate loss functions and the scoring function class. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems 33: Annu...

    2020

  53. [61]

    Statistical behavior and consistency of classification methods based on convex risk minimization

    Zhang, T. Statistical behavior and consistency of classification methods based on convex risk minimization. Annals of Statistics, 32, 2002. doi:10.1214/aos/1079120130

    2002

  54. [62]

    and Sabuncu, M

    Zhang, Z. and Sabuncu, M. R. Generalized cross entropy loss for training deep neural networks with noisy labels. In Bengio, S., Wallach, H. M., Larochelle, H., Grauman, K., Cesa - Bianchi, N., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 31: Annual...

    2018

  55. [63]

    Fundamental Novel Consistency Theory: H-Consistency Bounds

    Zhong, Y. Fundamental Novel Consistency Theory: H-Consistency Bounds. New York University, 2025

    2025

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.