REVIEW 4 cited by
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Merging uncertainty sets via majority vote
read the original abstract
Given $K$ uncertainty sets that are arbitrarily dependent -- for example, confidence intervals for an unknown parameter obtained with $K$ different estimators, or prediction sets obtained via conformal prediction based on $K$ different algorithms on shared data -- we address the question of how to efficiently combine them in a black-box manner to produce a single uncertainty set. We present a simple and broadly applicable majority vote procedure that produces a merged set with nearly the same error guarantee as the input sets. We then extend this core idea in a few ways: we show that weighted averaging can be a powerful way to incorporate prior information, and a simple randomization trick produces strictly smaller merged sets without altering the coverage guarantee. Further improvements can be obtained if the sets are exchangeable. We also show that many modern methods, like split conformal prediction, median of means, HulC and cross-fitted ``double machine learning'', can be effectively derandomized using these ideas.
Forward citations
Cited by 4 Pith papers
-
Set-Preserving Calibration from Conformal P-Values to E-Values
Introduces a set-preserving P2E calibrator for conformal prediction that improves efficiency and supports exact 1-alpha coverage in cross-conformal prediction and conformal aggregation.
-
Conformal prediction after data-dependent model selection
Novel methods for valid conformal prediction after data-dependent model selection without additional sample splitting, with finite-sample guarantees and asymptotic optimality under regularity conditions.
-
Rank Intervals for Leaderboards: A Hierarchical Framework for Model Evaluation
A hierarchical framework generates statistically valid task-level rank confidence intervals via pairwise comparisons and leaderboard-level rank prediction intervals via conformal prediction.
-
Theoretical Foundations of Conformal Prediction
The book curates and presents proofs of important existing results in conformal prediction in a unified pedagogical format with illustrations.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.