REVIEW 6 cited by
Learning Optimal Conformal Classifiers
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Modern deep learning based classifiers show very high accuracy on test data but this does not provide sufficient guarantees for safe deployment, especially in high-stake AI applications such as medical diagnosis. Usually, predictions are obtained without a reliable uncertainty estimate or a formal guarantee. Conformal prediction (CP) addresses these issues by using the classifier's predictions, e.g., its probability estimates, to predict confidence sets containing the true class with a user-specified probability. However, using CP as a separate processing step after training prevents the underlying model from adapting to the prediction of confidence sets. Thus, this paper explores strategies to differentiate through CP during training with the goal of training model with the conformal wrapper end-to-end. In our approach, conformal training (ConfTr), we specifically "simulate" conformalization on mini-batches during training. Compared to standard training, ConfTr reduces the average confidence set size (inefficiency) of state-of-the-art CP methods applied after training. Moreover, it allows to "shape" the confidence sets predicted at test time, which is difficult for standard CP. On experiments with several datasets, we show ConfTr can influence how inefficiency is distributed across classes, or guide the composition of confidence sets in terms of the included classes, while retaining the guarantees offered by CP.
Forward citations
Cited by 6 Pith papers
-
Improving Backward Conformal Prediction via Non-Conformity Score Transformation
ST-BCP tightens the coverage bound in Backward Conformal Prediction by applying a computable data-dependent transformation to nonconformity scores, reducing the average gap from 4.20% to 1.12% on benchmarks while prov...
-
Conformal Changepoint Localization and Root Cause Analysis with Corrupted Observations
Weighted conformal changepoint localization (W-CONCH) and root-cause analysis (W-CROC) downweight likely-corrupted observations via classifier uncertainty, preserving coverage and shrinking confidence sets.
-
Direct Prediction Set Minimization via Bilevel Conformal Classifier Training
DPSM reformulates conformal training as a bilevel problem with quantile regression in the lower level and claims an O(1/sqrt n) learning bound, cutting prediction set size by about 20% in experiments.
-
Test-time augmentation improves efficiency in conformal prediction
Applying learned test-time augmentation before conformal scoring reduces prediction set sizes by 10-14% with no loss of nominal coverage.
-
Dynamic Estimation Loss Control in Variational Quantum Sensing via Online Conformal Inference
A dynamic variational quantum sensing method using online conformal inference controls the long-term estimation loss at a user-specified level while updating circuit and estimator parameters.
-
Enhancing Adversarial Robustness with Conformal Prediction: A Framework for Guaranteed Model Reliability
An adversarial attack and defense that respectively enlarge and shrink conformal prediction sets, with experiments on CIFAR-10, CIFAR-100 and mini-ImageNet.
Discussion (0). Continue with ORCID to comment.