Pith. sign in

REVIEW 3 major objections 7 minor 1 cited by

Estimating prevalence with precision and accuracy

T0 review · 3 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A new Bayesian method, Precise Quantifier (PQ), yields tighter and equally well-calibrated prediction intervals for class prevalence than existing Bayesian and bootstrap quantifiers.

desk verdict A genuine new Bayesian quantifier with solid 50%-interval results, but the abstract overstates precision and the key prior-shift assumption is never stress-tested. read the letter →

arxiv 2507.06061 v1 pith:G7KGYBYE submitted 2025-07-08 stat.ML cs.LG

Aime Bienfait Igiraneza , Christophe Fraser , Robert Hinch This is my paper · ORCID
classification stat.MLcs.LG MSC 62F1568T05
keywords prevalenceestimationquantificationlearningBayesianpredictionintervalsuncertaintypriorprobabilityshiftbootstrapconfidenceclassifierscoredistributionsMCMC
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that uncertainty in class-prevalence estimates can be reported more tightly than standard practice allows. It introduces Precise Quantifier (PQ), a Bayesian method that models the classifier's score distribution for each class with bins and derives a posterior predictive distribution for the test-set prevalence. In experiments on simulated data and three real-world review datasets, PQ's prediction intervals were shorter than BayesianCC's intervals and shorter than bootstrap confidence intervals from PACC and HDy, while coverage stayed at the expected confidence level. The paper also identifies the factors that set interval width: classifier discriminatory power, validation-set size, and test-set size. If the claim holds, anyone estimating prevalence from unlabeled data -- epidemiologists, social scientists, or engineers -- could report narrower, trustworthy ranges instead of wider bootstrap intervals.

What carries the argument

PQ rests on the weak prior probability shift assumption, $P_V(f(X)|Y)=P_T(f(X)|Y)$, which says the classifier's score distributions for each class are the same in validation and test data even though prevalence differs. Classifier scores are sorted into $N_{\mathrm{bin}}$ equal-size bins per class; positive and negative validation sets contribute multinomial counts and the unlabeled test set contributes a mixture likelihood $\theta_{\mathrm{pr}} p^+_k+(1-\theta_{\mathrm{pr}})p^-_k$ per bin. With uniform and Dirichlet priors, the posterior over $\theta_{\mathrm{pr}}$ and the bin probabilities is sampled by MCMC, and each posterior draw yields one predictive prevalence via the bin-level binomial sum. This formulation propagates validation and test uncertainty together while sampling only $2N_{\mathrm{bin}}+1$ continuous parameters rather than one latent label per test point.

What would settle it

Run PQ on test sets drawn from a different time period, population, or domain than the validation set, with known prevalence, and compare the empirical coverage of nominal 50% intervals across many replications; if coverage drops systematically below 50% when score distributions shift, the central coverage claim fails.

Watch

Extended reading notes

Core claim

The paper's central discovery is that a multi-level Bayesian quantification model, PQ, gives prediction intervals for prevalence that are narrower than those of existing Bayesian and bootstrap quantifiers while keeping coverage calibrated. PQ treats the test prevalence $\theta$ as the normalized sum of bin-level binomial counts, giving the posterior predictive distribution $\theta | \theta_{\mathrm{pr}},\{p^+\},\{p^-\} \sim \frac{1}{n_T}\sum_{k} \mathrm{Binomial}(t_k, \theta_{\mathrm{pr}} p^+_k/(\theta_{\mathrm{pr}} p^+_k+(1-\theta_{\mathrm{pr}})p^-_k))$. In the paper's comparisons, PQ's central 50% intervals were on average the shortest among methods that reached 50% coverage, and its point estimates had bias comparable to the other well-calibrated methods. The experiments further show that stronger classifiers, larger validation sets, and larger test sets all narrow the intervals, with test-size gains largely realized by 500 samples.

Load-bearing premise

The load-bearing premise is that the classifier's score distribution for each class is the same in validation and test data; if that score distribution shifts between settings, PQ's interval coverage can no longer be expected to reach the nominal level.

Editorial extensions

If this is right

  • Practitioners can report an interval for prevalence rather than a point estimate, with the width automatically reflecting how informative the classifier is and how much labeled and unlabeled data are available.
  • Bayesian quantification of this kind beats bootstrap resampling of point estimates in precision under the tested conditions, so interval users gain without losing coverage.
  • EMQ should not be trusted for interval estimates by default: it matched or beat PQ's precision only on some small test sets, and its coverage fell below the nominal level.
  • Improving the classifier, enlarging the validation set, or enlarging the test set each buys narrower intervals, and most of the test-size benefit is reached around 500 samples.
  • The bin count matters: four bins balanced precision and coverage, and larger bin counts can degrade coverage when the validation set is small.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: if validation and test data come from different domains, the weak prior probability shift assumption is the only bridge between them, so PQ's coverage should degrade under covariate shift; this is directly testable and would bound the method's practical reach.
  • Inference: PQ's binning suggests a natural extension to a continuous Dirichlet-process or kernel-density model for class-conditional scores, which could remove the sensitivity to bin count while keeping the same posterior predictive mechanism.
  • Inference: extending PQ to multi-class would require a genuine multivariate model per bin rather than one-versus-all binarization, since the paper notes that one-versus-all can violate the probability shift assumption.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The manuscript proposes Precise Quantifier (PQ), a Bayesian aggregative method for binary prevalence estimation. PQ discretizes classifier scores into Nbin bins, models positive/negative bin counts with multinomial likelihoods on validation and test data (Eqs. 1–2), places a Bernoulli prior on individual test labels, and obtains a posterior predictive distribution for the test prevalence θ via MCMC (Eqs. 10–12). The paper compares PQ to BayesianCC, EMQ, PACC, and HDy on two simulated classification problems and three review datasets (hp, kindle, imdb), measuring PI/CI length and coverage. The authors report that PQ yields shorter intervals than the other methods while achieving nominal coverage, and they analyze how classifier strength, validation size, and test size affect precision.

Significance. If the results are reproducible, PQ is a useful addition to quantification uncertainty: it is computationally efficient (only 2Nbin+1 continuous parameters), is agnostic to classifier internals, and the paper provides open-source code. The simulation and real-data protocols are extensive, and the binning sensitivity analysis (Figure 3) is a good practice. The main claims, however, are established only under the weak prior probability shift assumption and only for 50% coverage, which tempers the significance until these gaps are addressed. The derivation itself is a standard Bayesian posterior predictive and I do not see a circularity problem.

major comments (3)
  1. [Section 2, Eqs. (1)–(3)] The likelihood and the multinomial model rely on the weak prior probability shift assumption P_V(f(X)|Y)=P_T(f(X)|Y), which is never violated in any experiment: the simulated data use identical class-conditional distributions for validation and test by construction, and the review datasets are drawn from the same QuaPy test split (Section 3.2). The paper's headline claim that PQ's coverage is well-calibrated is therefore conditional on an assumption that is always satisfied by design. Please add at least one experiment in which the test class-conditional score distributions differ from validation (e.g., a covariate shift or label-conditioned score shift) and report the resulting coverage; if the method degrades, temper the general claim and state the assumption explicitly in the abstract.
  2. [Section 4, Figures 1–2] All coverage results are reported only for central 50% prediction intervals. The text in Section 2 states that more posterior samples are needed for 95% intervals, but no 95% coverage evaluation is presented. Because 'well-calibrated coverage' is a central claim, the paper should demonstrate at least one other confidence level (e.g., 95%) to rule out calibration being specific to the 50% level.
  3. [Abstract] The claim that PQ is 'more precise than existing quantifiers' is too strong. Section 4 concedes that EMQ can be as precise or more precise than PQ on small test sets (Figure 1C), and EMQ is excluded from the ranking only because its coverage is unreliable. Please qualify the claim, e.g., 'more precise than existing quantifiers with well-calibrated coverage,' or modify the abstract and introduction to match the reported results. This is not a mathematical error but it affects the paper's leading claim.
minor comments (7)
  1. [Section 1] Typo: 'treatmant' should be 'treatment'.
  2. [Table 1 caption] Typo: 'datasests' should be 'datasets'.
  3. [Section 4] Typo: 'Not suprising' should be 'Not surprisingly'.
  4. [Section 2] The notation 'prior prevalence θpr' is confusing because θpr is estimated from the data (Eq. 8) rather than fixed a priori; consider renaming it 'label prevalence hyperparameter' or 'mixing proportion' to avoid implying it is a fixed prior on θ.
  5. [Section 3.2] It is unclear whether the validation set for quantifiers is disjoint from the early-stopping split of the BERT training data; please clarify the data splitting procedure.
  6. [Section 3.3] For coverage evaluation, specify whether the 'true prevalence' is the nominal prevalence value used to generate the test set or the empirical fraction of positives in the drawn sample; because sampling is without replacement from finite pools, these can differ slightly.
  7. [Section 4] When EMQ is excluded because coverage can be below 50%, report the actual coverage values or a statistical comparison rather than relying on the informal phrase 'coverage was not reliable.'

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PQ's posterior predictive intervals follow from a self-contained Bayesian model, and the weak prior shift assumption is explicit rather than a fitted input.

full rationale

The derivation chain in Section 2 is a standard multi-level Bayesian analysis. Equations (1)-(2) define multinomial models for binned validation counts; Eq. (3) expresses the per-item likelihood under the stated weak prior probability shift; Eqs. (6)-(8) form the likelihood and posterior; Eqs. (9)-(12) define the test prevalence as the normalized sum of latent labels and derive its posterior predictive distribution as a mixture of binomials integrated over the posterior. The quantity called the prediction interval is literally the central interval of this posterior predictive distribution, so there is no point estimate first fitted and then renamed a prediction. The hyper-parameter θpr is estimated from the same data, which is an empirical-Bayes element, but this does not make the interval equivalent to its inputs: the posterior still integrates over parameter uncertainty and the test counts enter the likelihood. The paper cites no load-bearing self-citations; references to BayesianCC, HDy, EMQ, and label-shift literature are external. The largest caveat, that empirical coverage is only demonstrated under the weak prior shift assumption and all experiments satisfy that assumption by construction, is a limitation on external validity or a robustness gap, not a circularity: the assumption is explicitly stated rather than encoded as the conclusion. The 'coverage reaching expected confidence level' claim is conditional on the model being correct, which is standard for Bayesian calibration and is not an instance of fitted input being called a prediction.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

PQ introduces no new physical entities; the model has one user-set hyperparameter (Nbin) and rests on standard i.i.d. sampling plus the weak prior probability shift assumption, with a non-parametric histogram representation of class-conditional scores.

free parameters (1)
  • Nbin (number of bins) = 4
    Hyperparameter for the granularity of the class-conditional histograms; the paper sets it to 4 by default and tests sensitivity, higher values can reduce coverage with small validation sets.
assumptions (4)
  • domain assumption Weak prior probability shift: P_V(f(X)|Y)=P_T(f(X)|Y)
    Central to PQ's likelihood (Eq. 3-7); violations would break calibration. Stated in Section 2.
  • domain assumption Samples in validation and test sets are independent
    Used to write the factored log-likelihood in Eq. 7.
  • domain assumption The trained classifier f(X) is fixed and its parameters are not part of the uncertainty model
    PQ treats f as a black-box input, so classifier parameter uncertainty is ignored.
  • standard math Multinomial/binomial models for binned counts
    Assumed in Eqs. 1, 2, and 10-12.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Estimating prevalence with precision and accuracy." pith.science (2026). https://pith.science/paper/G7KGYBYE

@misc{pith2026250706061,
  author       = {Pith},
  title        = {Pith review of: Estimating prevalence with precision and accuracy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G7KGYBYE}},
  note         = {Machine review of arXiv:2507.06061}
}
read the original abstract

Unlike classification, whose goal is to estimate the class of each data point in a dataset, prevalence estimation or quantification is a task that aims to estimate the distribution of classes in a dataset. The two main tasks in prevalence estimation are to adjust for bias, due to the prevalence in the training dataset, and to quantify the uncertainty in the estimate. The standard methods used to quantify uncertainty in prevalence estimates are bootstrapping and Bayesian quantification methods. It is not clear which approach is ideal in terms of precision (i.e. the width of confidence intervals) and coverage (i.e. the confidence intervals being well-calibrated). Here, we propose Precise Quantifier (PQ), a Bayesian quantifier that is more precise than existing quantifiers and with well-calibrated coverage. We discuss the theory behind PQ and present experiments based on simulated and real-world datasets. Through these experiments, we establish the factors which influence quantification precision: the discriminatory power of the underlying classifier; the size of the labeled dataset used to train the quantifier; and the size of the unlabeled dataset for which prevalence is estimated. Our analysis provides deep insights into uncertainty quantification for quantification learning.

Figures

Figures reproduced from arXiv: 2507.06061 by the authors.

Figure 1
Figure 1. PQ’s high precision on simulated datasets. Bars represent mean values. Rows A and C correspond to difficult classification (i.e. mean for the positive class was 1), while rows B and D correspond to easy classification (i.e. mean for the positive class was 2.5). In both cases, the mean for the negative class was 0, and all samples were drawn from normal distributions with unit variance. The validation sample size was… view at source ↗
Figure 2
Figure 2. PQ’s high precision on review datasets. Bars represent mean values. Rows A and D correspond to the hp dataset; rows B and E are for the kindle dataset; rows C and F correspond to the imdb dataset. The validation sample size was 1000 in A, B, and C, while it was 100 in D, E, F, with 50% class prevalence in all cases. Validation samples were drawn from QuaPy test data. Test samples were drawn from the remaining QuaPy … view at source ↗
Figure 3
Figure 3. The impact of binning. Bars represent mean values. The training sample size was 10000 with 90% prevalence. The number of used bins is shown as suffix. Rows A and C are for the difficult classification case while rows B and D are for the easy classification case. Error bars represent 95% bootstrap confidence intervals, calculated with 1000 samples. not suprising that precision was comparable across datasets with a sl… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Geometry-Aware Bayesian Quantification via Compositional Data Analysis

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Shrinkage-regularized Aitchison KDE on classifier posteriors yields competitive multiclass quantification and a practical Bayesian posterior over class prevalences under label shift.

Reference graph

Works this paper leans on

30 extracted references · 20 canonical work pages · cited by 1 Pith paper

  1. [1]

    Counting positives accurately despite inaccurate classification

    George Forman. Counting positives accurately despite inaccurate classification. In João Gama, Rui Camacho, Pavel B. Brazdil, Alípio Mário Jorge, and Luís Torgo, editors,Machine Learning: ECML 2005, pages 564–575. Springer, . ISBN 978-3-540-31692-3. doi: 10.1007/11564096_55

    work page doi:10.1007/11564096_55 2005

  2. [2]

    Quantifying counts and costs via classification

    George Forman. Quantifying counts and costs via classification. 17(2):164–206, . ISSN 1573-756X. doi: 10.1007/s10618-008-0097-y. URL https://doi.org/10.1007/ s10618-008-0097-y

    work page doi:10.1007/s10618-008-0097-y

  3. [3]

    Ag- gregative quantification for regression

    Antonio Bella, Cèsar Ferri, José Hernández-Orallo, and María José Ramírez-Quintana. Ag- gregative quantification for regression. 28(2):475–518, . ISSN 1573-756X. doi: 10.1007/ s10618-013-0308-z. URL https://doi.org/10.1007/s10618-013-0308-z . 9

    work page doi:10.1007/s10618-013-0308-z

  4. [4]

    Why is quantification an interesting learning problem? 6(1):53–58

    Pablo González, Jorge Díez, Nitesh Chawla, and Juan José del Coz. Why is quantification an interesting learning problem? 6(1):53–58. ISSN 2192-6360. doi: 10.1007/s13748-016-0103-3. URL https://doi.org/10.1007/s13748-016-0103-3

    work page doi:10.1007/s13748-016-0103-3

  5. [5]

    Learning to Quantify, volume 47 of The Information Retrieval Series

    Andrea Esuli, Alessandro Fabris, Alejandro Moreo, and Fabrizio Sebastiani. Learning to Quantify, volume 47 of The Information Retrieval Series. Springer International Publishing, . ISBN 978-3-031-20466-1 978-3-031-20467-8. doi: 10.1007/978-3-031-20467-8. URL https://link.springer.com/10.1007/978-3-031-20467-8

    doi:10.1007/978-3-031-20467-8

  6. [6]

    Confidence intervals for class prevalences under prior probability shift

    Dirk Tasche. Confidence intervals for class prevalences under prior probability shift. 1(3): 805–831, . ISSN 2504-4990. doi: 10.3390/make1030047. URL https://www.mdpi.com/ 2504-4990/1/3/47. Number: 3 Publisher: Multidisciplinary Digital Publishing Institute

    work page doi:10.3390/make1030047

  7. [7]

    When training and test sets are different: Characterizing learning transfer

    Storkey Amos. When training and test sets are different: Characterizing learning transfer. In Joaquin Quiñonero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D. Lawrence, editors, Dataset Shift in Machine Learning, page 0. The MIT Press. ISBN 978-0-262-17005-

    work page

  8. [8]

    URL https://doi.org/10.7551/ mitpress/9780262170055.003.0001

    doi: 10.7551/mitpress/9780262170055.003.0001. URL https://doi.org/10.7551/ mitpress/9780262170055.003.0001

    arXiv

Show all 30 references
  1. [9]

    Bayesian quantification with black-box estimators

    Albert Ziegler and Paweł Czy ˙z. Bayesian quantification with black-box estimators. ISSN 2835-8856. URL https://openreview.net/forum?id=Ft4kHrOawZ

  2. [10]

    Uncertainty-aware generative models for inferring document class prevalence

    Katherine Keith and Brendan O’Connor. Uncertainty-aware generative models for inferring document class prevalence. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii, editors, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Proces...

    2018 doi

  3. [11]

    Asif Naeem

    Benjamin Denham, Edmund M-K Lai, Roopak Sinha, and M. Asif Naeem. Gain-some- lose-some: Reliable quantification under general dataset shift. In 2021 IEEE International Conference on Data Mining (ICDM), pages 1048–1053. doi: 10.1109/ICDM51629.2021.00121. URL https://ieeexplore....

    2021

  4. [12]

    Generalized bayes quantifica- tion learning under dataset shift

    Jacob Fiksel, Abhirup Datta, Agbessi Amouzou, and Scott Zeger. Generalized bayes quantifica- tion learning under dataset shift. 117(540):2163–2181. ISSN 0162-1459. doi: 10.1080/ 01621459.2021.1909599. URL https://doi.org/10.1080/01621459.2021.1909599. Publisher: ASA Website _e...

    2021

  5. [13]

    Lipton, Yu-Xiang Wang, and Alex Smola

    Zachary C. Lipton, Yu-Xiang Wang, and Alex Smola. Detecting and correcting for label shift with black box predictors. URL http://arxiv.org/abs/1802.03916

    arXiv

  6. [14]

    QuaPy: A python-based framework for quantification

    Alejandro Moreo, Andrea Esuli, and Fabrizio Sebastiani. QuaPy: A python-based framework for quantification. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, CIKM ’21, pages 4534–4543. Association for Computing Machinery. ISBN 978-1...

  7. [15]

    BERT: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. URL http://arxiv.org/abs/ 1810.04805

    arXiv

  8. [16]

    The importance of the test set size in quantification assessment

    André Maletzke, Waqar Hassan, Denis dos Reis, and Gustavo Batista. The importance of the test set size in quantification assessment. volume 3, pages 2640–2646. doi: 10.24963/ijcai.2020/366. URL https://www.ijcai.org/proceedings/2020/366. ISSN: 1045-0823

    2020 doi

  9. [17]

    Quan- tification via probability estimators

    Antonio Bella, Cesar Ferri, José Hernández-Orallo, and María José Ramírez-Quintana. Quan- tification via probability estimators. In 2010 IEEE International Conference on Data Mining, pages 737–742, . doi: 10.1109/ICDM.2010.75. URL https://ieeexplore.ieee.org/ abstract/document...

    2010

  10. [18]

    Guzmán- Martínez, and Enrique Alegre

    Víctor González-Castro, Rocío Alaiz-Rodríguez, Laura Fernández-Robles, R. Guzmán- Martínez, and Enrique Alegre. Estimating class proportions in boar semen analysis using the hellinger distance. In Nicolás García-Pedrajas, Francisco Herrera, Colin Fyfe, José Manuel 10 Benítez, ...

    doi

  11. [19]

    Adjusting the outputs of a classifier to new a priori probabilities: A simple procedure

    Marco Saerens, Patrice Latinne, and Christine Decaestecker. Adjusting the outputs of a classifier to new a priori probabilities: A simple procedure. 14(1):21–41. ISSN 0899-7667. doi: 10.1162/089976602753284446. URL https://doi.org/10.1162/089976602753284446

    doi

  12. [20]

    A critical reassessment of the saerens- latinne-decaestecker algorithm for posterior probability adjustment

    Andrea Esuli, Alessio Molinari, and Fabrizio Sebastiani. A critical reassessment of the saerens- latinne-decaestecker algorithm for posterior probability adjustment. 39(2):19:1–19:34, . ISSN 1046-8188. doi: 10.1145/3433164. URL https://dl.acm.org/doi/10.1145/3433164

    doi

  13. [21]

    Maximum likelihood with bias- corrected calibration is hard-to-beat at label shift adaptation

    Amr Alexandari, Anshul Kundaje, and Avanti Shrikumar. Maximum likelihood with bias- corrected calibration is hard-to-beat at label shift adaptation. In Proceedings of the 37th International Conference on Machine Learning , pages 222–232. PMLR. URL https:// proceedings.mlr.pres...

  14. [22]

    Minimising quantifier variance under prior probability shift,

    Dirk Tasche. Minimising quantifier variance under prior probability shift, . URL http: //arxiv.org/abs/2107.08209

    arXiv

  15. [23]

    Quantification-oriented learning based on reliable classifiers

    Jose Barranquero, Jorge Díez, and Juan José del Coz. Quantification-oriented learning based on reliable classifiers. 48(2):591–604. ISSN 0031-3203. doi: 10.1016/j.patcog.2014.07.032. URL https://www.sciencedirect.com/science/article/pii/S003132031400291X

    2014 doi

  16. [24]

    A comparative evaluation of quantification methods

    Tobias Schumacher, Markus Strohmaier, and Florian Lemmerich. A comparative evaluation of quantification methods. URL http://arxiv.org/abs/2103.03223

    arXiv

  17. [25]

    Quantification under prior probability shift: the ratio estimator and its extensions

    Afonso Fernandes Vaz, Rafael Izbicki, and Rafael Bassi Stern. Quantification under prior probability shift: the ratio estimator and its extensions. URL http://arxiv.org/abs/1807. 03929

  18. [26]

    Zahra Donyavi, Adriane B. S. Serapião, and Gustavo Batista. MC-SQ and MC-MQ: Ensembles for multi-class quantification. 36(8):4007–4019. ISSN 1558-2191. doi: 10.1109/TKDE.2024. 3372011. URL https://ieeexplore.ieee.org/document/10457862. Conference Name: IEEE Transactions on Kno...

    2024

  19. [27]

    A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. 39(1):1–22. ISSN 0035-9246. doi: 10.1111/j.2517-6161.1977.tb01600.x. URL https://doi.org/10.1111/j.2517-6161.1977.tb01600.x

    1977

  20. [28]

    Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods

    John Platt. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. 10(3):61–74. Publisher: Cambridge, MA. 11 A Technical Appendices and Supplementary Material In this section, we present the main ideas behind existing quantificatio...

  21. [29]

    That is, TPR and FPR are the same for the validation set and the test set

    = PV ( ˆY = 1|Y = 0) [2]. That is, TPR and FPR are the same for the validation set and the test set. With this assumption applied to equation (14), ACC’s prevalence estimate becomes: ˆPT (Y = 1) = PT ( ˆY = 1) − PV ( ˆY = 1|Y = 0) PV ( ˆY = 1|Y = 1) − PV ( ˆY = 1|Y = 0) (15) B...

  22. [30]

    class 1 prevalence in our case) in the validation data

    proposed that ω be set to the true class prevalence (i.e. class 1 prevalence in our case) in the validation data. However, Alexandari et al. [20] pointed out that it may be more appropriate to set ω to the average of predicted class probabilities in the validation set, as long...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.