{"paper":{"title":"On the Burden of Achieving Fairness in Conformal Prediction","license":"http://creativecommons.org/licenses/by-nc-nd/4.0/","headline":"Pooled calibration in conformal prediction creates irreducible coverage distortion across groups set by quantile heterogeneity.","cross_cats":["cs.LG"],"primary_cat":"stat.ML","authors_text":"Archer Yi Yang, Jesse C. Cresswell, Masoud Asgharian, Mouloud Belbahri, Pengqi Liu, Ziang Gao","submitted_at":"2026-05-14T02:02:06Z","abstract_excerpt":"Conformal prediction is often calibrated with a single pooled threshold, but this can hide cross-group heterogeneity in score distributions and distort group-wise coverage. We study this phenomenon through the population score distributions underlying split conformal calibration. First, we derive a conservation law and lower bound showing that pooled calibration incurs irreducible group-wise coverage distortion at a scale set by cross-group quantile heterogeneity. Second, we demonstrate that the two leading fairness definitions for conformal prediction, Equalized Coverage and Equalized Set Siz"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"pooled calibration incurs irreducible group-wise coverage distortion at a scale set by cross-group quantile heterogeneity","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"The derivations rely on population-level score distributions for each group being well-defined and accessible for analysis, which may not translate exactly to finite-sample or misspecified models.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"Pooled conformal calibration incurs irreducible group-wise coverage distortion set by cross-group quantile heterogeneity, and Equalized Coverage and Equalized Set Size are in fundamental tension.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Pooled calibration in conformal prediction creates irreducible coverage distortion across groups set by quantile heterogeneity.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"e3175bb8b1c31b227999087da95e989128460df1aaeac70f60b9162caedb83f7"},"source":{"id":"2605.14260","kind":"arxiv","version":1},"verdict":{"id":"dc8c7e8e-cdfe-4a77-8668-7f9a75d63f17","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-15T02:35:56.558120Z","strongest_claim":"pooled calibration incurs irreducible group-wise coverage distortion at a scale set by cross-group quantile heterogeneity","one_line_summary":"Pooled conformal calibration incurs irreducible group-wise coverage distortion set by cross-group quantile heterogeneity, and Equalized Coverage and Equalized Set Size are in fundamental tension.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"The derivations rely on population-level score distributions for each group being well-defined and accessible for analysis, which may not translate exactly to finite-sample or misspecified models.","pith_extraction_headline":"Pooled calibration in conformal prediction creates irreducible coverage distortion across groups set by quantile heterogeneity."},"references":{"count":35,"sample":[{"doi":"","year":2021,"title":"Uncer- tainty sets for image classifiers using conformal prediction","work_id":"9833ba7d-6380-4907-b6c8-f54ad6491e0d","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2025,"title":"arXiv preprint arXiv:2512.19142 , year=","work_id":"41a9661e-66ec-4df7-85c4-b9f00fc08446","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"10.1089/big.2016.0047","year":2017,"title":"Big Data 5, 153–163","work_id":"f0a60d95-ce28-48c7-afdf-32427a3ffe3c","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2024,"title":"Cresswell, Yi Sui, Bhargava Kumar, and Noël V ouitsis","work_id":"6a946e20-adf9-43b2-ac6a-2673a2efd1b0","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2025,"title":"Cresswell, Bhargava Kumar, Yi Sui, and Mouloud Belbahri","work_id":"90b27117-e1a9-4f2e-8073-12a83a5a4004","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":35,"snapshot_sha256":"df40524e9f57ca0a11afb983cef1c2cd106425f197636620b3aa6827f4611fb4","internal_anchors":1},"formal_canon":{"evidence_count":2,"snapshot_sha256":"1ad075475f0640f12c446c82a0644ac2b5cc18c1deaf1f6ad8cf55a486ad255d"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}