Pith. sign in

REVIEW 4 major objections 4 minor 121 references

Confidence Calibration of Deep Learning Systems

T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Noisy labels need not inflate conformal prediction sets

desk verdict Genuinely useful empirical chapters and a promising idea, but the k-independent coverage guarantee rests on a DKW step that does not apply; deserves a serious referee and major revision. read the letter →

arxiv 2608.12100 v1 pith:TTQM4VDI submitted 2026-08-12 cs.LG cs.AIstat.ML

Coby Penso This is my paper · ORCID
classification cs.LGcs.AIstat.ML
keywords confidencecalibrationconformalpredictionlabelnoisetemperaturescalingcoverageguaranteelocaldifferentialprivacyunsuperviseddomainadaptationsets
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The thesis claims that confidence calibration and conformal prediction can be made reliable when validation labels are noisy, by modeling the noise and reconstructing the clean-label behavior. Its central method, NACP, estimates clean coverage from noisy coverage and average set size, then chooses a threshold whose clean coverage equals the target. The key payoff is a finite-sample coverage guarantee whose correction term does not grow with the number of classes, so tasks with hundreds or thousands of classes can keep small prediction sets. The same noise-modeling idea is applied to temperature scaling for medical imaging, to locally differentially private calibration, and to unsupervised domain adaptation.

What carries the argument

The central identity is the noisy-coverage decomposition $F^n(q) = (1-\epsilon)F^c(q) + \epsilon F^r(q)$, where a validation label is either the true label or a uniformly random label. NACP estimates $F^n$ from noisy validation scores, estimates $F^r$ from the average prediction-set size $|C_q(x)|/k$, inverts the identity, and searches for the threshold satisfying the target clean coverage. The finite-sample guarantee is carried by a DKW-type uniform concentration bound on the estimated clean CDF, producing the correction $\Delta = \sqrt{\log(4/\delta)/(2nh^2)}$ with $h=(1-\epsilon)/(1+\epsilon)$, which is independent of the number of classes.

What would settle it

Run NACP on a synthetic validation set with 1000 classes where the noise probability is twice as high for low-confidence inputs; if empirical coverage falls below $1-\alpha-\Delta$, or if the prediction set size grows with the number of classes under this feature-dependent noise, then the uniform x-independent noise model is the failing premise.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that under uniform, input-independent label noise with rate $\epsilon$, the clean conformal coverage $F^c(q)$ can be reconstructed from the noisy coverage $F^n(q)$ and the random-label coverage $F^r(q)$ through $\hat{F}^c(q) = (\hat{F}^n(q) - \epsilon \hat{F}^r(q))/(1-\epsilon)$. Choosing the threshold where $\hat{F}^c(q) = 1-\alpha$ yields prediction sets whose size tracks the clean-label oracle, with a finite-sample coverage guarantee $p(y \in C_q(x)) \ge 1-\alpha-\Delta$ in which $\Delta$ depends on validation set size and noise rate but not on the number of classes. The thesis further argues that ordinary temperature scaling fails badly on noisy validation labels, that a confusion-matrix correction recovers clean calibration for general noise matrices, and that the same reconstruction identity turns local differential privacy mechanisms into a known noise channel for conformal prediction.

Load-bearing premise

The coverage guarantee assumes the label noise is uniform and independent of the input, with the noise rate known or well estimated; real annotator noise can violate this, and the paper's own CIFAR-10N experiment shows coverage falling below the 90 percent target on the aggregate set.

Editorial extensions

If this is right

  • For validation sets corrupted by uniform label noise, practitioners can set conformal thresholds using noisy labels and still obtain prediction sets close to clean-label size.
  • In many-class tasks with 100 to 1000 classes, methods whose correction terms scale with the number of classes become useless, while NACP's class-independent correction keeps them viable.
  • Temperature scaling applied to noisy validation sets can match clean-label calibration on medical imaging benchmarks instead of overheating the model.
  • Local differential privacy can be added to conformal prediction by treating the privacy mechanism as a known noise channel, with coverage adjusted by a finite-sample term.
  • Calibrating on unlabeled target data by rescaling source accuracy substantially improves calibration under domain shift compared with importance-weighted source calibration.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the class-independent correction holds only for uniform noise, a natural next test is replacing the scalar $\epsilon$ with per-input noise estimates to handle instance-dependent label corruption; the thesis does not prove such an extension.
  • The reconstruction identity also suggests a unified view of privacy and noise: any mechanism that acts as a known label-noise channel, beyond k-ary randomized response, could be plugged into the same threshold search with an appropriate correction term.
  • A testable extension is applying NACP in online or streaming conformal settings where noisy calibration labels arrive incrementally and the coverage estimate must be updated without re-searching the whole grid.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This PhD-thesis manuscript develops methods for confidence calibration and conformal prediction under label noise, under local differential privacy, and under unsupervised domain shift. The main technical contributions are: a noise-robust temperature scaling method for noisy validation labels (Chapter 3); two noisy-label conformal prediction methods, NRSCP and NACP, with NACP intended to provide a finite-sample coverage guarantee whose correction term is independent of the number of classes under uniform label noise (Chapter 4); two locally differentially private conformal prediction frameworks, LDP-CP-L and LDP-CP-S (Chapter 5); and a target-domain calibration method, UTDC, that rescales source-domain binwise accuracy by an estimated accuracy ratio (Chapter 6). The experiments are extensive and cover medical and natural image datasets, multiple architectures, several noise models, and both synthetic and real annotation noise.

Significance. If the central NACP guarantee were valid, the claim that the finite-sample correction is independent of the number of classes for uniform noise would be a substantial advance over the prior noisy-label conformal prediction bounds of ACNL and CRCP, whose correction terms grow with k. The empirical work is careful in many respects: results are averaged over 1000 splits for the conformal experiments, several model architectures are compared, and code links are provided. The value of NACP as an efficient practical method and the strength of the UTDC domain-adaptation results are not in question. However, the theoretical guarantee that is the paper's headline claim is supported by an invalid application of the DKW inequality, so the significance of the theoretical contribution is currently conditional.

major comments (4)
  1. [§4.3.2, Lemma 4.3.4] The DKW inequality is applied to \hat F^r(q) = (1/n) \sum_i |C_q(x_i)|/k, but this quantity is not an empirical CDF of n i.i.d. scalar observations. For a fixed x_i, the k terms S(x_i,j) share the same input and are dependent, and the function q \mapsto |C_q(x_i)|/k is a conditional CDF rather than an indicator of one observed scalar. A uniform-in-q deviation bound for such an average of random functions does not follow from DKW as stated; a correct argument would need empirical-process or VC-type machinery and would in general introduce a k-dependent constant. Therefore the k-independent form of \Delta in Theorem 4.3.5, which is the paper's main claim for many-class problems, is not established.
  2. [§4.3.2, Algorithm 4, and Theorem 4.3.5] There is a mismatch between the theorem and the implemented algorithm. Theorem 4.3.5 proves coverage at least 1-\alpha only when q is chosen so that \hat F^c(q) = 1 - \alpha + \Delta, but Algorithm 4, line 4, selects q satisfying \hat F^c(q) = 1 - \alpha, and the guarantee printed in Algorithm 4, line 6, states coverage at least 1 - \alpha - \Delta. The claim that the unadjusted procedure reaches nominal coverage in the experiments is an empirical observation and does not replace the missing proof. The same issue appears in Algorithm 7 and Theorem 5.4.1 in Chapter 5.
  3. [§5.4, Theorem 5.4.1] The proof of the LDP-CP-L coverage guarantee is not supplied. The text says that substituting \beta and n into Theorem 4.3.5 yields the bound, but Theorem 4.3.5 requires the condition \hat F^c(q) = 1 - \alpha + \Delta and gives a two-sided coverage statement 1 - \alpha \le p(y \in C_q(x)) \le 1 - \alpha + 2\Delta; neither the condition nor the claimed one-sided 1 - \alpha - \Delta form is verified for the binary-search output of Algorithm 7. Additionally, because the raw features x_i are sent unperturbed to the aggregator, k-RR on labels provides label-LDP, not \epsilon-LDP for the full data pair (x,y); the theorem statement should be aligned with the label-DP setting that Section 5.2 actually describes.
  4. [§5.3, §5.4, Theorem 5.4.2] The proof of Theorem 5.4.2 is too terse to establish the stated conformal coverage guarantee. The theorem gives a sample complexity for the private binary-search quantile estimator, but the proof does not show how an error in the estimated (1-\alpha)-quantile of conformity scores translates into the stated bound on p(y \in C_{\hat q}(x)). The argument also implicitly assumes that the scores are bounded in a known interval [Q_min, Q_max] and that the binary-search interval satisfies the \tau-condition, but these assumptions are not stated in the theorem. A complete derivation connecting the quantile estimator's error to conformal coverage is needed.
minor comments (4)
  1. [§2.4, Eq. (2.10)] Equation (2.10) as written is not a stochastic matrix: for the uniform noise model that flips to one of the remaining k-1 classes, the off-diagonal entry should be \epsilon/(k-1), not \epsilon/k, otherwise the row sum is 1 - \epsilon/k rather than 1. The text says the two uniform-noise conventions are used interchangeably, but the displayed matrix should be corrected or the normalization should be explained.
  2. [§4.3.1, proof of Lemma 4.3.1] The proof refers to "Lemma 3.1" when it means Lemma 4.3.1; the cross-reference should be fixed.
  3. [§4.5, Table 4.6] On CIFAR-10N-aggregate, NACP with rand-APS reports 88.49% coverage, below the 90% target. The authors do acknowledge that real annotator noise is not uniform, but this violation of the key assumption should be stated directly in the text near the table, since it shows that the coverage guarantee does not transfer to the real-noise experiments.
  4. [§6.2, Eq. (6.1)] The assumption that the source-to-target accuracy ratio is constant across confidence bins is validated only qualitatively and on a single task in Figure 6.4. Since this ratio rescaling is the core of UTDC, it would be helpful to report a quantitative measure of the binwise ratio variability across the different benchmark tasks.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: NACP and NTS reconstruct clean coverage/accuracy from explicit noise models, and fitted noise parameters are upstream inputs rather than renamed predictions.

full rationale

The paper's central derivations are self-contained reconstructions, not self-fulfilling predictions. In Chapter 3, the clean bin accuracy Â_i is obtained by inverting the explicit uniform-noise relation (3.1)-(3.3); the fitted temperature is evaluated on a clean test set, so no fitted quantity is being passed off as a prediction. In Chapter 4, Eq. (4.7) is an exact identity under the stated noise model, and Eq. (4.10) solves for F^c rather than assuming it; the threshold is then chosen by İF^c(q)=1-α, and the coverage guarantee in Theorem 4.3.5 follows a standard concentration-plus-CP argument. The fact that the same threshold is acknowledged to match the earlier works [86,13] is disclosed, and the claimed novelty is the k-independent finite-sample correction, which is an independent mathematical claim even if its DKW application to İF^r is technically questionable. Chapter 5 reuses NACP with the k-RR channel; this is legitimate composition of an internally proven result, not circular import of an unverified self-citation, and the privacy guarantee is supplied by standard LDP post-processing. Chapter 6 uses external target-accuracy estimators and source bin accuracies, then evaluates on held-out target data; the rescaling (6.1) is an estimation heuristic, not a definition that forces the reported adaECE improvement. The thesis does contain many self-citations because it collects the author's published papers, but these citations are not load-bearing in the sense that removing them would collapse the derivations into tautology. The most serious concern is a correctness gap, not circularity: Lemma 4.3.4 applies DKW separately to İF^n and İF^r, but İF^r(q)=(1/n)Σ_i|C_q(x_i)|/k is an average of conditional CDFs rather than an empirical CDF of n i.i.d. scalar observations, so the claimed k-independent Δ does not follow from the stated argument. That is a proof-soundness issue and should be weighed as correctness risk, not as circular reasoning. Accordingly, no specific circular step is identified; the score of 2 reflects the presence of self-citations and internal reuse while affirming that the core derivations have independent content.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

No new physical or metaphysical entities are introduced. The load-bearing assumptions are the noise model for labels (uniform or matrix), the applicability of DKW-type concentration to the proposed estimators, and the constant accuracy-ratio assumption for target-domain rescaling. The noise parameters are estimated from data or assumed known, and the temperature and conformal thresholds are fitted, which is standard for these methods.

free parameters (3)
  • noise level epsilon = 0.2 in main experiments; estimated via [52] in end-to-end runs
    Input to NTS and NACP formulas for reconstructing clean accuracy and coverage; errors propagate to the calibrated temperature and conformal threshold.
  • label noise matrix P = estimated during noisy-label training via [52]
    NTS and general-noise NACP invert P to recover clean confusion and coverage matrices (Eqs. 3.10 and 4.20).
  • target accuracy estimate = Meta [16], ATC [28], PN [109] outputs
    UTDC rescales source bin accuracies by estimated target accuracy divided by source accuracy (Eq. 6.1).
assumptions (5)
  • domain assumption p(tilde y | y) is known, invertible, and independent of x given y.
    Used in Eq. 3.8 and Eq. 4.17 to factor noisy confusion and coverage matrices; breaks on annotator-dependent or feature-dependent noise.
  • domain assumption Uniform noise model with known epsilon (Eq. 4.6) for NACP's class-independent bound.
    The main finite-sample guarantee relies on h = (1 - epsilon)/(1 + epsilon); real CIFAR-10N noise violates this uniformity.
  • standard math DKW inequality bounds sup_q |hat F^r(q) - F^r(q)|.
    Lemma 4.3.4 applies DKW to hat F^r, an average of conditional CDFs, which is not literally an empirical CDF of observed samples.
  • ad hoc to paper Target and source accuracy ratio is constant across confidence bins.
    UTDC's Eq. 6.1 scales every source bin by the same ratio; Figure 6.4 provides empirical support only.
  • standard math Calibration and test samples are exchangeable.
    Standard conformal prediction guarantees in Chapters 2, 4, and 5 require exchangeability.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Confidence Calibration of Deep Learning Systems." pith.science (2026). https://pith.science/paper/TTQM4VDI

@misc{pith2026260812100,
  author       = {Pith},
  title        = {Pith review of: Confidence Calibration of Deep Learning Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TTQM4VDI}},
  note         = {Machine review of arXiv:2608.12100}
}
read the original abstract

In high-stakes applications, reliable confidence estimates are as important as the predictions themselves. Confidence calibration ensures that predicted probabilities reflect the likelihood of correctness, making it essential for safe deployment of deep learning models. However, existing methods typically assume access to clean validation data, which is often unrealistic due to label noise and domain shifts. This thesis develops methods for improving calibration under these conditions. First, we address calibration under label noise. Standard methods can produce misleading confidence estimates when labels are unreliable. We propose a framework that uses an estimated noise model to reconstruct noise-free confidence estimates by modeling the relationship between noisy and clean label distributions. We extend this approach to Conformal Prediction (CP), which provides set-valued predictions with guaranteed coverage. Our noise-aware CP method estimates clean conformity scores despite label noise, enabling reliable uncertainty quantification. Next, we study calibration in unsupervised domain adaptation, where a model trained on a labeled source domain is adapted to an unlabeled target domain. Since labeled target data are unavailable, we estimate target-domain accuracy from source performance and domain discrepancies, enabling calibration without target labels. We also consider privacy-preserving settings in which user labels and model outputs must remain protected. We propose a locally differentially private conformal prediction framework that provides valid uncertainty quantification while maintaining privacy guarantees and balancing privacy, computational feasibility, and prediction reliability. Our results bridge calibration theory and practical deployment in safety-critical applications, contributing to reliable, privacy-preserving, and noise-resilient neural network predictions.

Figures

Figures reproduced from arXiv: 2608.12100 by the authors.

Figure 3.1
Figure 3.1. Schema of the proposed model that includes the full pipeline of network training and [PITH_FULL_IMAGE:figures/full_fig_p031_3_1.png] view at source ↗
Figure 3.2
Figure 3.2. Comparative calibration results on several datasets that were trained with ResNet-50. [PITH_FULL_IMAGE:figures/full_fig_p037_3_2.png] view at source ↗
Figure 3.3
Figure 3.3. Standard deviation of ECE scores on the test set after the NTS calibration as a function [PITH_FULL_IMAGE:figures/full_fig_p038_3_3.png] view at source ↗
Figures from the paper (14 more)
Figure 3.4
Figure 3.4. Figure 3.4: Calibration results for several noise transition matrices on ChestX-ray14-bal and ResNet [PITH_FULL_IMAGE:figures/full_fig_p041_3_4.png]
Figure 3.5
Figure 3.5. Figure 3.5: Calibration performance, measured by adaECE (left), and the corresponding tempera [PITH_FULL_IMAGE:figures/full_fig_p041_3_5.png]
Figure 4.1
Figure 4.1. Figure 4.1: Correction terms ∆ of NACP, ACNL and CRCP as a function of the validation set size n given ϵ = 0.2. We show results for 3 numbers of classes, 10, 100 and 1000. datasets, CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet. In all cases, we used 1−α = 0.9 and a noise lev…
Figure 4.2
Figure 4.2. Figure 4.2: Noisy labels conformal prediction on ImageNet with different calibration set sizes. (a) [PITH_FULL_IMAGE:figures/full_fig_p058_4_2.png]
Figure 5.1
Figure 5.1. Figure 5.1: Local Differential Private Conformal Prediction (LDP-CP-L) Pipeline (Best viewed in [PITH_FULL_IMAGE:figures/full_fig_p063_5_1.png]
Figure 5.2
Figure 5.2. Figure 5.2: Local Differential Private Conformal Prediction (LDP-CP-S) Pipeline (Best viewed in [PITH_FULL_IMAGE:figures/full_fig_p066_5_2.png]
Figure 5.3
Figure 5.3. Figure 5.3: Comparison of ∆l and ∆s as a function of the number of classes k and dataset size n, for ϵ = 2, 4, 8. (a) LDP-CP-L - ∆L (b) LDP-CP-S - ∆S [PITH_FULL_IMAGE:figures/full_fig_p071_5_3.png]
Figure 5.4
Figure 5.4. Figure 5.4: CP correction terms ∆L, ∆S as a function of ϵ privacy parameter across different dataset configurations of n and k without the shuffle model. (a) LDP-CP-L - ∆L (b) LDP-CP-S - ∆S [PITH_FULL_IMAGE:figures/full_fig_p071_5_4.png]
Figure 5.5
Figure 5.5. Figure 5.5: CP correction terms ∆L, ∆S as a function of ϵ eff privacy parameter across different dataset configurations of n and k with the shuffle model. Utility-Coverage Tradeoff. Theorems 5.4.1 and 5.4.2 state that LDP-CP-L and LDP-CP-S are ϵ-LDP with a conformal prediction c…
Figure 5.6
Figure 5.6. Figure 5.6: Size of prediction set (left) and coverage (right) as a function of the privacy [PITH_FULL_IMAGE:figures/full_fig_p073_5_6.png]
Figure 6.1
Figure 6.1. Figure 6.1: Average accuracy on Office-home tasks for the three UDA techniques (DANN, DANN+E, [PITH_FULL_IMAGE:figures/full_fig_p081_6_1.png]
Figure 6.2
Figure 6.2. Figure 6.2: adaECE results as a function of the correction ratio [PITH_FULL_IMAGE:figures/full_fig_p082_6_2.png]
Figure 6.3
Figure 6.3. Figure 6.3: Accuracy of k-th percentile source images based on their probability of being classified as target [97], compared to target accuracy (Office-home, A → C). (a) CDAN+E (b) DANN+E (c) DANN [PITH_FULL_IMAGE:figures/full_fig_p083_6_3.png]
Figure 6.4
Figure 6.4. Figure 6.4: Accuracy per bin for source and target images. The results are shown on the Office-home [PITH_FULL_IMAGE:figures/full_fig_p083_6_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

121 extracted references · 62 canonical work pages

  1. [1]

    Conformal prediction: A gentle introduction

    Anastasios N Angelopoulos, Stephen Bates, et al. Conformal prediction: A gentle introduction. Foundations and Trends in Machine Learning , 16(4):494–591, 2023

    2023

  2. [2]

    Uncertainty sets for image classifiers using conformal prediction

    AnastasiosN.Angelopoulos, StephenBates, JitendraMalik, andMichaelIJordan. Uncertainty sets for image classifiers using conformal prediction. International Conference on Learning Representations (ICLR), 2021

    2021

  3. [3]

    Private prediction sets

    Anastasios N Angelopoulos, Stephen Bates, Tijana Zrnic, and Michael I Jordan. Private prediction sets. arXiv preprint arXiv:2102.06202 , 2022

    arXiv 2022

  4. [4]

    Learning with privacy at scale, 2017

    Apple. Learning with privacy at scale, 2017. Accessed: [Insert Access Date]

    2017

  5. [5]

    Private learning and sanitization: Pure vs

    Amos Beimel, Kobbi Nissim, and Uri Stemmer. Private learning and sanitization: Pure vs. approximate differential privacy. InInternational Workshop on Approximation Algorithms for Combinatorial Optimization, pages 363–378. Springer, 2013

    2013

  6. [6]

    Training deep neural-networks based on unreli- able labels

    Alan Joseph Bekker and Jacob Goldberger. Training deep neural-networks based on unreli- able labels. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2682–2686, 2016

    2016

  7. [7]

    Verification of forecasts expressed in terms of probability.Monthly Weather Review, 78(1):1–3, 1950

    Glenn W Brier. Verification of forecasts expressed in terms of probability.Monthly Weather Review, 78(1):1–3, 1950

    1950

  8. [8]

    AnomMAN: Detect Anomaly on Multi-view Attributed Networks

    Ling-Hao Chen, He Li, and Wenhao Yang. Anomman: Detect anomaly on multi-view at- tributed networks. arXiv preprint arXiv:2201.02822 , 2022

    work page Pith review arXiv 2022

Show all 121 references
  1. [9]

    Noise against noise: stochastic label noise helps combat inherent label noise

    Pengfei Chen, Guangyong Chen, Junjie Ye, Pheng-Ann Heng, et al. Noise against noise: stochastic label noise helps combat inherent label noise. InInternational Conference on Learn- ing Representations (ICLR), 2021

    2021

  2. [10]

    Instance-dependent label-noise learning with manifold-regularized transition matrix estimation

    De Cheng, Tongliang Liu, Yixiong Ning, Nannan Wang, Bo Han, Gang Niu, Xinbo Gao, and Masashi Sugiyama. Instance-dependent label-noise learning with manifold-regularized transition matrix estimation. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CV...

    2022

  3. [11]

    Learning with instance-dependent label noise: A sample sieve approach

    Hao Cheng, Zhaowei Zhu, Xingyu Li, Yifei Gong, Xing Sun, and Yang Liu. Learning with instance-dependent label noise: A sample sieve approach. In International Conference on Learning Representations (ICLR), 2021. 75

    2021

  4. [12]

    Differential privacy in the shuffle model: A survey of separations, 2022

    Albert Cheu. Differential privacy in the shuffle model: A survey of separations, 2022

    2022

  5. [13]

    Split conformal prediction under data contamination

    Jase Clarkson, Wenkai Xu, Mihai i Cucuringu, and Gesine Reinert. Split conformal prediction under data contamination. In Proceedings of the Thirteenth Symposium on Conformal and Probabilistic Prediction with Applications, 2024

    2024

  6. [14]

    Towards discriminability and diversity: Batch nuclear-norm maximization under label insufficient situa- tions

    Shuhao Cui, Shuhui Wang, Junbao Zhuo, Liang Li, Qingming Huang, and Qi Tian. Towards discriminability and diversity: Batch nuclear-norm maximization under label insufficient situa- tions. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2020

    2020

  7. [15]

    Imagenet: A large- scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. InProc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 248–255. Ieee, 2009

    2009

  8. [16]

    Are labels always necessary for classifier accuracy evaluation? In Proc

    Weijian Deng and Liang Zheng. Are labels always necessary for classifier accuracy evaluation? In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2021

    2021

  9. [17]

    Training a neural network based on unreliable human annotation of medical images

    Yair Dgani, Hayit Greenspan, and Jacob Goldberger. Training a neural network based on unreliable human annotation of medical images. InThe IEEE International Symposium on Biomedical Imaging (ISBI), 2018

    2018

  10. [18]

    Dowson and B

    D.C. Dowson and B. V. Landau. The Fréchet distance between multivariate normal distribu- tions. Journal of Multivariate Analysis , 12(3):450–455, 1982

    1982

  11. [19]

    Local privacy and statistical minimax rates

    John C Duchi, Michael I Jordan, and Martin J Wainwright. Local privacy and statistical minimax rates. In 2013 IEEE 54th annual symposium on foundations of computer science , pages 429–438. IEEE, 2013

    2013

  12. [20]

    Differential privacy

    Cynthia Dwork. Differential privacy. InInternational colloquium on automata, languages, and programming, pages 1–12. Springer, 2006

    2006

  13. [21]

    Conformal prediction is robust to label noise

    Bat-Sheva Einbinder, Stephen Bates, Anastasios N Angelopoulos, Asaf Gendler, and Yaniv Romano. Conformal prediction is robust to label noise. arXiv preprint arXiv:2209.14295 , 2022

    2022 arXiv

  14. [22]

    Rappor: Randomized aggregatable privacy-preserving ordinal response

    Úlfar Erlingsson, Vasyl Pihur, and Aleksandra Korolova. Rappor: Randomized aggregatable privacy-preserving ordinal response. InProceedings of the 2014 ACM SIGSAC conference on computer and communications security , pages 1054–1067, 2014

    2014

  15. [23]

    K Ruwani M Fernando and Chris P Tsokos. Dynamically weighted balanced loss: class im- balanced learning and confidence calibration of deep neural networks.IEEE Transactions on Neural Networks and Learning Systems , 33(7):2940–2951, 2021. 76

    2021

  16. [24]

    The limits of distribution-free conditional predictive inference.Information and Inference: A Journal of the IMA, 10(2):455–482, 2021

    Rina Foygel Barber, Emmanuel J Candes, Aaditya Ramdas, and Ryan J Tibshirani. The limits of distribution-free conditional predictive inference.Information and Inference: A Journal of the IMA, 10(2):455–482, 2021

    2021

  17. [25]

    Calibration of medical imaging classification systems with weight scaling

    Lior Frenkel and Jacob Goldberger. Calibration of medical imaging classification systems with weight scaling. In International Conference on Medical Image Computing and Computer- Assisted Intervention (MICCAI), 2022

    2022

  18. [26]

    Locally private mean estimation: Z-test and tight confidence intervals, 2019

    Marco Gaboardi, Ryan Rogers, and Or Sheffet. Locally private mean estimation: Z-test and tight confidence intervals, 2019

    2019

  19. [27]

    Domain-adversarial training of neural networks

    Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. The Journal of Machine Learning Research , 17(1):2096–2030, 2016

    2016

  20. [28]

    Leveraging unlabeled data to predict out-of- distribution performance

    Saurabh Garg and Sivaraman Balakrishnan. Leveraging unlabeled data to predict out-of- distribution performance. International Conference on Learning Representations (ICLR) , 2022

    2022

  21. [29]

    Deep learning with label differential privacy.Advances in Neural Information Processing Systems (NeurIPs), 34:27131–27145, 2021

    Badih Ghazi, Noah Golowich, Ravi Kumar, Pasin Manurangsi, and Chiyuan Zhang. Deep learning with label differential privacy.Advances in Neural Information Processing Systems (NeurIPs), 34:27131–27145, 2021

    2021

  22. [30]

    Aritra Ghosh, Himanshu Kumar, and P. S. Sastry. Robust loss functions under label noise for deep neural networks. InProceedings of the AAAI Conference on Artificial Intelligence , pages 1919–1925, 2017

    1919

  23. [31]

    Training deep neural-networks using a noise adap- tation layer

    Jacob Goldberger and Ehud Ben-Reuven. Training deep neural-networks using a noise adap- tation layer. InInternational Conference on Learning Representations (ICLR) , 2017

    2017

  24. [32]

    Pre- dicting with confidence on unseen distributions

    Devin Guillory, Vaishaal Shankar, Sayna Ebrahimi, Trevor Darrell, and Ludwig Schmidt. Pre- dicting with confidence on unseen distributions. InProc. of the IEEE International Conference on Computer Vision (ICCV) , 2021

    2021

  25. [33]

    On calibration of modern neural networks

    Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. InInternational Conference on Machine Learning (ICML),pages1321–1330.PMLR, 2017

    2017

  26. [34]

    Deep self-learning from noisy labels

    Jiangfan Han, Ping Luo, and Xiaogang Wang. Deep self-learning from noisy labels. InProc. of the IEEE International Conference on Computer Vision (ICCV) , pages 5138–5147, 2019

    2019

  27. [35]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016. 77

    2016

  28. [36]

    Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem

    Matthias Hein, Maksym Andriushchenko, and Julian Bitterwolf. Why relu networks yield high-confidence predictions far away from the training data and how to mitigate the problem. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 41–50, 2019

    2019

  29. [37]

    Using trusted data to train deep networks on labels corrupted by severe noise

    Dan Hendrycks, Mantas Mazeika, Duncan Wilson, and Kevin Gimpel. Using trusted data to train deep networks on labels corrupted by severe noise. InAdvances in Neural Information Processing Systems (NeurIPs), pages 10477–10486, 2018

    2018

  30. [38]

    Simple and effective regularization methods for training on noisily labeled data with generalization guarantee

    Wei Hu, Zhiyuan Li, and Dingli Yu. Simple and effective regularization methods for training on noisily labeled data with generalization guarantee. InInternational Conference on Learning Representations (ICLR), 2020

    2020

  31. [39]

    Densely con- nected convolutional networks

    Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely con- nected convolutional networks. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4700–4708, 2017

    2017

  32. [40]

    O2u-net: Asimplenoisylabeldetection approachfordeepneuralnetworks

    JinchiHuang, LieQu, RongfeiJia, andBinqiangZhao. O2u-net: Asimplenoisylabeldetection approachfordeepneuralnetworks. In Proc. of the IEEE International Conference on Computer Vision (ICCV), pages 3326–3334, 2019

    2019

  33. [41]

    Uncertainty-aware learning against label noise on imbalanced datasets

    Yingsong Huang, Bing Bai, Shengwei Zhao, Kun Bai, and Fei Wang. Uncertainty-aware learning against label noise on imbalanced datasets. InProceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 6960–6969, 2022

    2022

  34. [42]

    Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison

    Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. InProceedings of the AAAI C...

    2019

  35. [43]

    Delving into sample loss curve to embrace noisy and imbalanced data

    Shenwang Jiang, Jianan Li, Ying Wang, Bo Huang, Zhang Zhang, and Tingfa Xu. Delving into sample loss curve to embrace noisy and imbalanced data. InProceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 7024–7032, 2022

    2022

  36. [44]

    Minimum class confusion for ver- satile domain adaptation

    Ying Jin, Ximei Wang, Mingsheng Long, and Jianmin Wang. Minimum class confusion for ver- satile domain adaptation. InProc. of the European Conference on Computer Vision (ECCV) , 2020

    2020

  37. [45]

    Discrete distribution estimation under local privacy

    Peter Kairouz, Keith Bonawitz, and Daniel Ramage. Discrete distribution estimation under local privacy. In International Conference on Machine Learning (ICML) , pages 2436–2444. PMLR, 2016

    2016

  38. [46]

    What can we learn privately?SIAM Journal on Computing , 40(3):793–826, 2011

    Shiva Prasad Kasiviswanathan, Homin K Lee, Kobbi Nissim, Sofya Raskhodnikova, and Adam Smith. What can we learn privately?SIAM Journal on Computing , 40(3):793–826, 2011. 78

    2011

  39. [47]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  40. [48]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, Department of Computer Science, University of Toronto, 2009

    2009

  41. [49]

    Simple and scalable pre- dictive uncertainty estimation using deep ensembles

    Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable pre- dictive uncertainty estimation using deep ensembles. Advances in Neural Information Pro- cessing Systems (NeurIPs) , 30, 2017

    2017

  42. [50]

    Coupled-view deep classifier learning from multiple noisy annotators

    Shikun Li, Shiming Ge, Yingying Hua, Chunhui Zhang, Hao Wen, Tengfei Liu, and Weiqiang Wang. Coupled-view deep classifier learning from multiple noisy annotators. InProceedings of the AAAI Conference on Artificial Intelligence , pages 4667–4674, 2020

    2020

  43. [51]

    Trustable co-label learning from multiple noisy annotators.IEEE Transactions on Multimedia , 25:1045–1057, 2021

    Shikun Li, Tongliang Liu, Jiyong Tan, Dan Zeng, and Shiming Ge. Trustable co-label learning from multiple noisy annotators.IEEE Transactions on Multimedia , 25:1045–1057, 2021

    2021

  44. [52]

    Provably end-to-end label-noise learning without anchor points

    Xuefeng Li, Tongliang Liu, Bo Han, Gang Niu, and Masashi Sugiyama. Provably end-to-end label-noise learning without anchor points. InInternational Conference on Machine Learning (ICML), pages 6403–6413. PMLR, 2021

    2021

  45. [53]

    Domain adaptation with auxiliary target domain- oriented classifier

    Jian Liang, Dapeng Hu, and Jiashi Feng. Domain adaptation with auxiliary target domain- oriented classifier. In Proc. of the IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR), 2021

    2021

  46. [54]

    A holistic view of label noise transition matrix in deep learning and beyond

    Yong Lin, Renjie Pi, Weizhong Zhang, Xiaobo Xia, Jiahui Gao, Xiao Zhou, Tongliang Liu, and Bo Han. A holistic view of label noise transition matrix in deep learning and beyond. In International Conference on Learning Representations (ICLR) , 2023

    2023

  47. [55]

    Classification with noisy labels by importance reweighting

    Tongliang Liu and Dacheng Tao. Classification with noisy labels by importance reweighting. IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI) , 38(3):447–461, 2015

    2015

  48. [56]

    Conditional adversarial domain adaptation

    Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. Conditional adversarial domain adaptation. Advances in Neural Information Processing Systems (NeurIPs) , 2018

    2018

  49. [57]

    Improving trustworthiness of AI disease severity rating in medical imaging with ordinal conformal prediction sets

    Charles Lu, Anastasios N Angelopoulos, and Stuart Pomerantz. Improving trustworthiness of AI disease severity rating in medical imaging with ordinal conformal prediction sets. In International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2022

    2022

  50. [58]

    Fair conformal predictors for applications in medical imaging

    Charles Lu, Andréanne Lemay, Ken Chang, Katharina Höbel, and Jayashree Kalpathy- Cramer. Fair conformal predictors for applications in medical imaging. In Proceedings of the AAAI Conference on Artificial Intelligence , 2022. 79

    2022

  51. [59]

    Label-noise learning with intrinsically long-tailed data.arXiv e-prints, pages arXiv–2208, 2022

    Yang Lu, Yiliang Zhang, Bo Han, Yiu-ming Cheung, and Hanzi Wang. Label-noise learning with intrinsically long-tailed data.arXiv e-prints, pages arXiv–2208, 2022

    2022

  52. [60]

    The tight constant in the Dvoretzky-Kiefer-Wolfowitz inequality.The Annals of Probability, pages 1269–1283, 1990

    Pascal Massart. The tight constant in the Dvoretzky-Kiefer-Wolfowitz inequality.The Annals of Probability, pages 1269–1283, 1990

    1990

  53. [61]

    Accuracy on the line: on the strong correlation between out-of-distribution and in-distribution generalization

    John P Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, and Ludwig Schmidt. Accuracy on the line: on the strong correlation between out-of-distribution and in-distribution generalization. In International Conferen...

    2021

  54. [62]

    Revisiting the calibration of modern neural net- works

    Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural net- works. Advances in Neural Information Processing Systems (NeurIPs) , 34:15682–15694, 2021

    2021

  55. [63]

    Calibrating deep neural networks using focal loss.Advances in Neural Information Processing Systems (NeurIPs), 33:15288–15299, 2020

    Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip Torr, and Puneet Dokania. Calibrating deep neural networks using focal loss.Advances in Neural Information Processing Systems (NeurIPs), 33:15288–15299, 2020

    2020

  56. [64]

    When does label smoothing help?arXiv preprint arXiv:1906.02629, 2019

    Rafael Müller, Simon Kornblith, and Geoffrey Hinton. When does label smoothing help?arXiv preprint arXiv:1906.02629, 2019

    1906 arXiv

  57. [65]

    Obtaining well calibrated probabilities using bayesian binning

    Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 29, 2015

    2015

  58. [66]

    Posterior calibration and exploratory analysis for natural language processing models.arXiv preprint arXiv:1508.05154 , 2015

    Khanh Nguyen and Brendan O’Connor. Posterior calibration and exploratory analysis for natural language processing models.arXiv preprint arXiv:1508.05154 , 2015

    2015 arXiv

  59. [67]

    Estimating diagnostic uncertainty in artificial intelligence assisted pathology using conformal prediction.Nature Communications, 13(1):7761, 2022

    Henrik Olsson, Kimmo Kartasalo, Nita Mulliqi, et al. Estimating diagnostic uncertainty in artificial intelligence assisted pathology using conformal prediction.Nature Communications, 13(1):7761, 2022

    2022

  60. [68]

    Unsupervised calibration under covariate shift.arXiv preprint arXiv:2006.16405, 2020

    Anusri Pampari and Stefano Ermon. Unsupervised calibration under covariate shift.arXiv preprint arXiv:2006.16405, 2020

    2006 arXiv

  61. [69]

    Calibrated prediction with covariate shift via unsupervised domain adaptation

    Sangdon Park, Osbert Bastani, James Weimer, and Insup Lee. Calibrated prediction with covariate shift via unsupervised domain adaptation. InInternational Conference on Artificial Intelligence and Statistics , 2020

    2020

  62. [70]

    Making deep neural networks robust to label noise: A loss correction approach

    Giorgio Patrini, Alessandro Rozza, Aditya Krishna Menon, Richard Nock, and Lizhen Qu. Making deep neural networks robust to label noise: A loss correction approach. InProc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 1944–1952, 2017. 80

    1944

  63. [71]

    Moment matching for multi-source domain adaptation

    Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. InProc. of the IEEE International Conference on Computer Vision (ICCV) , pages 1406–1415, 2019

    2019

  64. [72]

    Visda: The visual domain adaptation challenge.arXiv preprint arXiv:1710.06924 , 2017

    Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. Visda: The visual domain adaptation challenge.arXiv preprint arXiv:1710.06924 , 2017

    2017 arXiv

  65. [73]

    Privacy-preserving confor- mal prediction under local differential privacy

    Coby Penso, , Bar Mahpud, Jacob Goldberger, and Or Sheffet. Privacy-preserving confor- mal prediction under local differential privacy. Symposium on Conformal and Probabilistic Prediction with Applications (COPA 2025), 2025

    2025

  66. [74]

    Confidence calibration of a medical imaging classification system that is robust to label noise

    Coby Penso, Lior Frenkel, and Jacob Goldberger. Confidence calibration of a medical imaging classification system that is robust to label noise. IEEE Transactions on Medical Imaging , 43(6):2050–2060, 2024

    2024

  67. [75]

    Calibration of network confidence for unsupervised domain adaptation using estimated accuracy

    Coby Penso and Jacob Goldberger. Calibration of network confidence for unsupervised domain adaptation using estimated accuracy. InECCV, Uncertainty in Computer Vision Workshop , 2024

    2024

  68. [76]

    A conformal prediction score that is robust to label noise

    Coby Penso and Jacob Goldberger. A conformal prediction score that is robust to label noise. In MICCAI, Machine Learning for Medical Imaging Workshop , 2024

    2024

  69. [77]

    A joint training and confidence calibration procedure that is robust to label noise

    Coby Penso and Jacob Goldberger. A joint training and confidence calibration procedure that is robust to label noise. InThe IEEE International Symposium on Biomedical Imaging (ISBI) , 2024

    2024

  70. [78]

    Estimatingtheconformalpredictionthresh- old from noisy labels.arXiv preprint arXiv:2501.12749 , 2024

    CobyPenso, JacobGoldberger, andEthanFetaya. Estimatingtheconformalpredictionthresh- old from noisy labels.arXiv preprint arXiv:2501.12749 , 2024

    2024 arXiv

  71. [79]

    Conformal prediction of classifiers with many classes based on noisy labels

    Coby Penso, Jacob Goldberger, and Ethan Fetaya. Conformal prediction of classifiers with many classes based on noisy labels. Symposium on Conformal and Probabilistic Prediction with Applications (COPA 2025), 2025

    2025

  72. [80]

    Probabilistic outputs for support vector machines and comparisons to regu- larized likelihood methods

    John Platt et al. Probabilistic outputs for support vector machines and comparisons to regu- larized likelihood methods. Advances in large margin classifiers , 10(3):61–74, 1999

    1999

  73. [81]

    Learning to reweight examples for robust deep learning

    Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. Learning to reweight examples for robust deep learning. In International Conference on Machine Learning (ICML) , pages 4331–4340, 2018

    2018

  74. [82]

    Classification with valid and adaptive coverage

    Yaniv Romano, Matteo Sesia, and Emmanuel Candes. Classification with valid and adaptive coverage. Advances in Neural Information Processing Systems (NeurIPs) , 2020. 81

    2020

  75. [83]

    Post training uncertainty calibration of deep networks for medical image segmentation

    Axel-Jan Rousseau, Thijs Becker, Jeroen Bertels, Matthew B Blaschko, and Dirk Valkenborg. Post training uncertainty calibration of deep networks for medical image segmentation. InThe IEEE International Symposium on Biomedical Imaging (ISBI) , pages 1052–1056. IEEE, 2021

    2021

  76. [84]

    Adapting visual category models to new domains

    Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. InProc. of the European Conference on Computer Vision (ECCV) , 2010

    2010

  77. [85]

    Improved pre- dictive uncertainty using corruption-based calibration.Stat, 1050:7, 2021

    Tiago Salvador, Vikram Voleti, Alexander Iannantuono, and Adam Oberman. Improved pre- dictive uncertainty using corruption-based calibration.Stat, 1050:7, 2021

    2021

  78. [86]

    Adaptive conformal classification with noisy labels

    Matteo Sesia, YX Wang, and Xin Tong. Adaptive conformal classification with noisy labels. arXiv preprint arXiv:2309.05092 , 2023

    2023 arXiv

  79. [87]

    Meta- weight-net: Learning an explicit mapping for sample weighting

    Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Zongben Xu, and Deyu Meng. Meta- weight-net: Learning an explicit mapping for sample weighting. InAdvances in Neural Infor- mation Processing Systems (NeurIPs) , pages 1917–1928, 2019

    1917

  80. [88]

    Learning from noisy labels with deep neural networks: A survey.IEEE transactions on neural networks and learning systems, 34(11):8135–8153, 2022

    Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. Learning from noisy labels with deep neural networks: A survey.IEEE transactions on neural networks and learning systems, 34(11):8135–8153, 2022

    2022

  81. [89]

    Joint optimization framework for learning with noisy labels

    Daiki Tanaka, Daiki Ikami, Toshihiko Yamasaki, and Kiyoharu Aizawa. Joint optimization framework for learning with noisy labels. InProc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 5552–5560, 2018

    2018

  82. [90]

    Post-hoc uncertainty calibration for domain drift scenarios

    Christian Tomani, Sebastian Gruber, Muhammed Ebrar Erdem, Daniel Cremers, and Florian Buettner. Post-hoc uncertainty calibration for domain drift scenarios. InProc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2021

    2021

  83. [91]

    The HAM10000 dataset, a large collec- tion of multi-source dermatoscopic images of common pigmented skin lesions.Scientific data, 5(1):1–9, 2018

    Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. The HAM10000 dataset, a large collec- tion of multi-source dermatoscopic images of common pigmented skin lesions.Scientific data, 5(1):1–9, 2018

    2018

  84. [92]

    Deep hashing network for unsupervised domain adaptation

    Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. InProc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2017

    2017

  85. [93]

    Springer, 2005

    Vladimir Vovk, Alexander Gammerman, and Glenn Shafer.Algorithmic learning in a random world, volume 29. Springer, 2005

    2005

  86. [94]

    Graph structure estimation neural networks

    Ruijia Wang, Shuai Mou, Xiao Wang, Wanpeng Xiao, Qi Ju, Chuan Shi, and Xing Xie. Graph structure estimation neural networks. InWWW, pages 342–353, 2021. 82

    2021

  87. [95]

    Locally differentially private protocols for frequency estimation

    Tianhao Wang, Jeremiah Blocki, Ninghui Li, and Somesh Jha. Locally differentially private protocols for frequency estimation. In26th USENIX Security Symposium (USENIX Security 17), pages 729–745, 2017

    2017

  88. [96]

    Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases

    XiaosongWang, YifanPeng, LeLu, ZhiyongLu, MohammadhadiBagheri, andRonaldMSum- mers. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. InProc. of the IEEE Conference on Computer Vision...

    2017

  89. [97]

    Transferable calibra- tion with lower bias and variance in domain adaptation

    Ximei Wang, Mingsheng Long, Jianmin Wang, and Michael Jordan. Transferable calibra- tion with lower bias and variance in domain adaptation. InAdvances in Neural Information Processing Systems (NeurIPs), 2020

    2020

  90. [98]

    Randomized response: A survey technique for eliminating evasive answer bias

    Stanley L Warner. Randomized response: A survey technique for eliminating evasive answer bias. Journal of the American statistical association , 60(309):63–69, 1965

    1965

  91. [99]

    Learning with noisy labels revisited: A study using real-world human annotations

    Jiaheng Wei, Zhaowei Zhu, Hao Cheng, Tongliang Liu, Gang Niu, and Yang Liu. Learning with noisy labels revisited: A study using real-world human annotations. arXiv preprint arXiv:2110.12088, 2021

    2021 arXiv

  92. [100]

    Robust long-tailed learning under label noise

    Tong Wei, Jiang-Xin Shi, Wei-Wei Tu, and Yu-Feng Li. Robust long-tailed learning under label noise. arXiv preprint arXiv:2108.11569 , 2021

    2021 arXiv

  93. [101]

    In situ classification of cell types in human kidney tissue using 3D nuclear staining

    Andre Woloshuk, Suraj Khochare, Aljohara F Almulhim, Andrew T McNutt, Dawson Dean, Daria Barwinska, Michael J Ferkowicz, Michael T Eadon, Katherine J Kelly, Kenneth W Dunn, et al. In situ classification of cell types in human kidney tissue using 3D nuclear staining. Cytometry ...

    2021

  94. [102]

    Part-dependent label noise: Towards instance- dependent label noise

    Xiaobo Xia, Tongliang Liu, Bo Han, Nannan Wang, Mingming Gong, Haifeng Liu, Gang Niu, Dacheng Tao, and Masashi Sugiyama. Part-dependent label noise: Towards instance- dependent label noise. In Advances in Neural Information Processing Systems (NeurIPs) , 2020

    2020

  95. [103]

    Are anchor points really indispensable in label-noise learning? In Advances in Neural Information Processing Systems (NeurIPs) , pages 6835–6846, 2019

    Xiaobo Xia, Tongliang Liu, Nannan Wang, Bo Han, Chen Gong, Gang Niu, and Masashi Sugiyama. Are anchor points really indispensable in label-noise learning? In Advances in Neural Information Processing Systems (NeurIPs) , pages 6835–6846, 2019

    2019

  96. [104]

    Mou-Cheng Xu, Yukun Zhou, Chen Jin, Marius De Groot, Daniel C Alexander, Neil P Oxtoby, and Joseph Jacob. Mismatch: Calibrated segmentation via consistency on differential mor- phological feature perturbations with limited labels.IEEE transactions on medical imaging , 42(10):2...

    2023

  97. [105]

    Robust medical image classification from noisy labeled data with global and local representation guided co-training

    Cheng Xue, Lequan Yu, Pengfei Chen, Qi Dou, and Pheng-Ann Heng. Robust medical image classification from noisy labeled data with global and local representation guided co-training. IEEE Transactions on Medical Imaging , 41(6):1371–1382, 2022

    2022

  98. [106]

    Searching to exploit memorization effect in learning from corrupted labels

    Hansi Yang, Quanming Yao, Bo Han, Gang Niu, Hansi Yang, Bo Han, Gang Niu, and James Kwok. Searching to exploit memorization effect in learning from corrupted labels. InInterna- tional Conference on Machine Learning (ICML) , 2020

    2020

  99. [107]

    MedMNIST classification decathlon: A lightweight automl benchmark for medical image analysis

    Jiancheng Yang, Rui Shi, and Bingbing Ni. MedMNIST classification decathlon: A lightweight automl benchmark for medical image analysis. In The IEEE International Symposium on Biomedical Imaging (ISBI), 2021

    2021

  100. [108]

    Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification

    Jiancheng Yang, Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, and Bingbing Ni. Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data, 10(1):41, 2023

    2023

  101. [109]

    Predicting out-of- distribution error with the projection norm

    Yaodong Yu, Zitong Yang, Alexander Wei, Yi Ma, and Jacob Steinhardt. Predicting out-of- distribution error with the projection norm. InInternational Conference on Machine Learning (ICML), 2022

    2022

  102. [110]

    Transforming classifier scores into accurate multiclass probability estimates

    Bianca Zadrozny and Charles Elkan. Transforming classifier scores into accurate multiclass probability estimates. InInternational Conference on Knowledge Discovery and Data Mining (KDD), pages 694–699, 2002

    2002

  103. [111]

    Layer em- bedding analysis in convolutional neural networks for improved probability calibration and classification

    Fan Zhang, Nicha Dvornek, Junlin Yang, Julius Chapiro, and James Duncan. Layer em- bedding analysis in convolutional neural networks for improved probability calibration and classification. IEEE Transactions on Medical Imaging , 39(11):3331–3342, 2020

    2020

  104. [112]

    mixup: Beyond empirical risk minimization

    Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. InInternational Conference on Learning Representations (ICLR), 2018

    2018

  105. [113]

    Mix-n-match: Ensemble and composi- tional methods for uncertainty calibration in deep learning

    Jize Zhang, Bhavya Kailkhura, and T Yong-Jin Han. Mix-n-match: Ensemble and composi- tional methods for uncertainty calibration in deep learning. InInternational Conference on Machine Learning (ICML), pages 11117–11128. PMLR, 2020

    2020

  106. [114]

    Disentangling human error from ground truth in segmentation of medical images.Advances in Neural Information Processing Systems (NeurIPs) , 33:15750– 15762, 2020

    Le Zhang, Ryutaro Tanno, Mou-Cheng Xu, Chen Jin, Joseph Jacob, Olga Cicarrelli, Frederik Barkhof, and Daniel Alexander. Disentangling human error from ground truth in segmentation of medical images.Advances in Neural Information Processing Systems (NeurIPs) , 33:15750– 15762, 2020

    2020

  107. [115]

    When and how mixup improves calibration

    Linjun Zhang, Zhun Deng, Kenji Kawaguchi, and James Zou. When and how mixup improves calibration. In International Conference on Machine Learning (ICML) , pages 26135–26160. PMLR, 2022. 84

    2022

  108. [116]

    Learning with feature-dependent label noise: A progressive approach

    Yikai Zhang, Songzhu Zheng, Pengxiang Wu, Mayank Goswami, and Chao Chen. Learning with feature-dependent label noise: A progressive approach. InInternational Conference on Learning Representations (ICLR), 2021

    2021

  109. [117]

    Learning noise transition matrix from only noisy labels via total variation regularization

    Yivan Zhang, Gang Niu, and Masashi Sugiyama. Learning noise transition matrix from only noisy labels via total variation regularization. InInternational Conference on Machine Learn- ing (ICML), pages 12501–12512. PMLR, 2021

    2021

  110. [118]

    Generalized cross entropy loss for training deep neural net- works with noisy labels

    Zhilu Zhang and Mert Sabuncu. Generalized cross entropy loss for training deep neural net- works with noisy labels. In Advances in Neural Information Processing Systems (NeurIPs) , pages 8778–8788, 2018

    2018

  111. [119]

    Error-bounded correction of noisy labels

    Songzhu Zheng, Pengxiang Wu, Aman Goswami, Mayank Goswami, Dimitris Metaxas, and Chao Chen. Error-bounded correction of noisy labels. InInternational Conference on Machine Learning (ICML), pages 11447–11457, 2020

    2020

  112. [120]

    Confidence regularized self-training

    Yang Zou, Zhiding Yu, Xiaofeng Liu, BVK Kumar, and Jinsong Wang. Confidence regularized self-training. InProc. of the IEEE International Conference on Computer Vision (ICCV),2019

    2019

  113. [121]

    Adaptive calibrator ensemble: Navigating test set difficulty in out-of-distribution scenarios

    Yuli Zou, Weijian Deng, and Liang Zheng. Adaptive calibrator ensemble: Navigating test set difficulty in out-of-distribution scenarios. InProc. of the IEEE International Conference on Computer Vision (ICCV) , 2023. 85 תקציר ביישומים קריטיים כגון דימות רפואי, מידת האמון של מודל ...

    2023

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.