Pith. sign in

REVIEW 2 major objections 4 minor 78 references

Aleatoric and Epistemic Uncertainty Measures for Ordinal Classification through Binary Reduction

T0 review · 2 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read For ordinal classification, uncertainty measured by summing variances over order-consistent binary splits yields the best error-detection performance, beating entropy, plain variance, and label-wise measures on 23 benchmark datasets.

desk verdict A solid incremental extension of binary-reduction uncertainty to ordinal classification, undercut mainly by an abstract that overstates the statistical evidence. read the letter →

arxiv 2507.00733 v1 pith:IRJMZ6VZ submitted 2025-07-01 cs.LG

Stefan Haas , Eyke Hüllermeier This is my paper · ORCID
classification cs.LG MSC 68T0562H3062F15
keywords ordinalclassificationuncertaintyquantificationaleatoricepistemicbinaryreductionorder-consistentsplitselectivepredictionensemblemethods
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Ordinal classification, predicting ordered labels like credit ratings or disease severity, inherits uncertainty-quantification tools built for nominal labels, but those tools ignore the label order. This paper argues that the right way to measure aleatoric (irreducible randomness) and epistemic (lack of knowledge) uncertainty for ordered labels is to reduce the $K$-class problem to $K-1$ binary questions of the form "is the label above level $k$?", score each question with a base measure such as entropy or variance, and sum the scores. The central claim, supported by rejection-based experiments on 23 tabular benchmark datasets with gradient-boosted-tree and MLP ensembles, is that the variance-based version (ord-var) best captures the ordinal trade-off between exact hit rate and error distance, significantly outperforming standard entropy, plain variance, and label-wise decompositions in error detection. The paper also establishes that cross-entropy training serves uncertainty quantification better than dedicated ordinal losses, which bias predicted distributions toward compressed unimodal shapes. If these findings hold, ord-var is an immediate practical default for selective prediction on ordered labels.

What carries the argument

The order-consistent split (OCS) reduction is the mechanism: the $K$-class ordered problem is replaced by $K-1$ binary classifiers that answer "does the label exceed level $k$?", with success probability equal to the cumulative predicted mass above $k$. Any binary uncertainty measure $U$, Shannon entropy or variance, is applied twice: once to the ensemble-averaged binary distribution (total uncertainty) and once averaged over ensemble members (aleatoric uncertainty), with epistemic uncertainty as their difference. The overall ordinal measure is the sum of these binary measures over all $K-1$ splits (Eq. 16), which is what injects the label order into the score; the same construction with one-versus-rest splits ignores order entirely. The paper shows that, unlike plain entropy, this construction treats the extreme bimodal distribution (probability $1/2$ at each end class) as maximally uncertain, and that variance-based OCS lands between uniform-maximizing and dispersion-maximizing behavior on the simplex.

What would settle it

A controlled simulation can settle the central claim: generate query points with known ground-truth conditional label distributions (uniform, extreme-bimodal, and intermediate mixtures), and check which uncertainty score best orders the points by the expected mean absolute error of the optimal decision. If ord-var is not the best ordering score on this synthetic ground truth, the paper's "best trade-off" conclusion would need to be restricted to the particular distribution of real benchmark datasets rather than stated as a property of the measure.

Watch

Extended reading notes

Core claim

The paper's central claim is that aleatoric and epistemic uncertainty in ordinal classification should be measured through an order-consistent binary reduction: the $K$-class ordered label space is turned into $K-1$ binary questions of the form "does the label lie above level $k$?", each binary question is scored with a standard uncertainty measure (Shannon entropy or variance), and the scores are summed. With variance as the base measure, this construction (ord-var) significantly outperforms standard entropy, plain variance, and label-wise one-versus-rest decompositions in rejection-based error detection, when results are pooled over aleatoric, epistemic, and total uncertainty and over misclassification rate and mean absolute error across 23 ordinal benchmark datasets. Geometrically, ord-var sits between uniform-maximizing entropy and the extreme-bimodal-focused plain variance on the probability simplex, which the authors identify as the trade-off between exact hit rate and error distance that defines ordinal classification. The paper also argues that cross-entropy training is preferable to dedicated ordinal losses (EMD, QWK, unimodal soft labeling) whenever uncertainty quantification matters, because ordinal losses bias predictive distributions toward compressed unimodal shapes.

Load-bearing premise

Everything rests on the assumption that total ordinal uncertainty is an equally weighted sum of the $K-1$ binary split uncertainties, so if some "is the label above level $k$?" questions are redundant or carry more information than others, the flat sum could mis-rank instances even though every individual split is measured correctly.

Editorial extensions

If this is right

  • For selective prediction on ordered labels, ord-var supplies a ready-made confidence score: delegating the highest-uncertainty queries to a human expert improves both misclassification rate and mean absolute error more than entropy-based scores do.
  • Because the OCS reduction is defined on cumulative split probabilities, the variance version does not require assigning numerical values to the ordered categories, sidestepping the equal-spacing assumption flagged as debatable for direct variance on ordinal targets.
  • Cross-entropy-trained ensembles achieve higher prediction-rejection ratios than models trained with ordinal losses or unimodal soft labeling, so practitioners who need trustworthy uncertainty should prefer proper scoring rules even when ordinal metrics matter.
  • Out-of-distribution detection is the documented exception: entropy-based measures, especially ord-ent, keep the advantage there, so the choice of measure should depend on whether the deployment risk is in-distribution errors or out-of-distribution inputs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct untested extension is to weight the terms of Eq. (16) by split informativeness or class prevalence; ord-var's edge would be sharpened or shown to depend on the flat aggregation, depending on the outcome.
  • Since the OCS variance never needs numerical label encodings, ord-var transfers naturally to semi-quantitative scales, such as severity grades or survey ratings, where equal category spacing is indefensible; the paper does not make this transfer explicit.
  • The architecture-dependent OOD behavior the paper documents (tree ensembles report high aleatoric uncertainty on out-of-distribution inputs, MLPs report high epistemic uncertainty) implies that uncertainty-measure rankings may not transfer between architectures, so a neural-network deployment should not simply assume the tree-ensemble ranking.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes a new family of aleatoric and epistemic uncertainty measures for ordinal classification, built by reducing the ordinal problem to K-1 order-consistent binary splits and summing (or otherwise composing) binary entropy- and variance-based uncertainties. The authors compare these measures against standard multinomial entropy, plain variance, and label-wise one-versus-rest decompositions on 23 tabular datasets using ensembles of gradient-boosted trees and multi-layer perceptrons. Evaluation is performed through prediction-rejection ratios for error detection (MCR and MAE) and AUC-ROC for out-of-distribution detection. The paper concludes that the order-consistent variance-based measure (ord-var) best trades off exact hit-rate against error distance and significantly outperforms the other measures in error detection, while also being competitive in OOD detection.

Significance. The paper addresses a real gap: existing aleatoric/epistemic decomposition frameworks target nominal classification or regression, and the order-consistent split reduction is a natural and previously under-explored idea for ordinal labels. The proposed measures are defined without fitted parameters, the code is publicly released, and the experimental protocol (10-fold CV, 23 datasets, two model classes) is substantial for the field. If the empirical claims were fully supported, ord-var would be a practical default for selective prediction on ordered labels. The main weakness is that the headline significance claim rests on a statistically questionable pooled analysis, so the current manuscript overstates what is demonstrated by the experiments.

major comments (2)
  1. [Section 6.3, Figures 11a-c, Abstract] The claim in the abstract and Section 7 that ord-var 'significantly outperforms standard and label-wise entropy and variance-based measures in error detection, as indicated by misclassification rates and mean absolute error' is not supported by the per-metric analyses. Figure 11b shows no significant differences on MCR alone, and Figure 11c places ord-var, var, and ord-ent in the same nonsignificant group on MAE alone. The only analysis in which ord-var is uniquely superior is Figure 11a, which pools PRR values across the three uncertainty types (AU, EU, TU) and both metrics (MCR, MAE). This pooling violates the block structure assumed by the Friedman test and the paired Wilcoxon signed-rank test: each of the 23 datasets contributes up to six correlated observations of the same underlying measure (three uncertainty types times two metrics), inflating the effective sample size and the significance claim. Please re-run the statistical comparison with dataset as the blocking factor, separately for each uncertainty type and each metric (or use an appropriate repeated-measures / mixed-effects model), and adjust the abstract, conclusion, and Section 6.3 statements to reflect the results of those correctly blocked analyses.
  2. [Section 4, Eq. (16)] The ordinal uncertainty measure is defined as the unweighted sum of K-1 binary split uncertainties, but no justification is given for this additive, equal-weight aggregation, nor is a sensitivity analysis provided. This choice is load-bearing for the identity of ord-var and ord-ent that drive the paper's central claim. The manuscript should either supply a theoretical argument (e.g., an information-theoretic decomposition, consistency with the law of total variance or total entropy, or an axiomatic characterization) or demonstrate that the main empirical conclusions are robust to reasonable alternatives (e.g., using the maximum split uncertainty, weights based on split informativeness, or a different combination rule). Without such support, the claim that ord-var 'best captures the trade-off' is specific to one arbitrary parameterization.
minor comments (4)
  1. [Throughout] There are several typos that should be corrected: 'minimial' in the Abstract, 'uqantification' in Section 6.3, 'uncerainty' in the Figure 2 caption, 'T able' in the Table 1 heading and appendix tables, 'avaliable' in the footnote about the code repository, and 'inclduing' in Appendix E's heading.
  2. [Section 6.4 and Appendix E] The OOD detection protocol is described only briefly: for every in-distribution dataset, OOD data are sampled from Year MSD, but it is not explained how feature spaces are reconciled when the ID dataset has different features than Year MSD, nor whether categorical features are matched by name. Please provide the full preprocessing details for OOD construction, as this affects the interpretability of the AUC-ROC results in Tables E7 and E8.
  3. [Section 7] The sentence 'we were able to experimentally prove our hypothesis' is too strong for a statistical comparison on 23 datasets; 'provide evidence for' or 'support' would be more accurate and consistent with the level of certainty appropriate to the experimental design.
  4. [Section 6.3, Tables A1-A2] The description of how PRR values are aggregated for the statistical tests is incomplete: it should state explicitly whether the values used in the Friedman/Wilcoxon tests are per-fold PRRs, per-dataset means over folds, or per-dataset values from a pooled rejection curve. This detail is necessary for reproducibility and for assessing the independence assumptions of the tests.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the proposed measures are definitionally constructed and evaluated against external benchmark data.

full rationale

The paper's central construction, Eq. (16), defines ordinal uncertainty as a sum of per-split binary uncertainty measures. This is a definitional aggregation choice, not a quantity fitted to or derived from the performance outcome it is later used to predict. The headline claim that ord-var provides the best trade-off between exact hit-rate and error distance is supported by rejection-based experiments on 23 external ordinal benchmark datasets, with PRR values, accuracy-rejection curves, and Friedman/Holm-adjusted Wilcoxon tests, rather than by any parameter fitted to the evaluation data. The only notable self-citation is to Haas and Hüllermeier (2025), which is used transparently to motivate the OCS decomposition for total uncertainty and to criticize entropy in ordinal settings; that prior work is not the basis of the empirical superiority claim, which is tested directly against standard, label-wise, and ordinal baselines. The additivity assumption in Eq. (16) is a modeling assumption, not a circular step, since no measure is defined in terms of the very quantity it predicts and no fitted value is renamed as a prediction. Potential concerns about the pooled significance test in Figure 11a, or about the weighting of splits, are statistical or design issues, not circularity, and therefore do not raise the circularity score.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central measures contain no fitted parameters. The main modeling commitments are the integer encoding of ordinal labels for variance, the ensemble approximation of the posterior, and the additive aggregation of split-level uncertainties in Eq. (16); the last of these is the most ad hoc.

free parameters (2)
  • Ensemble size M = 10
    Number of models in the Monte Carlo ensemble (Section 6.1); chosen by the authors, not fitted, and it affects the quality of all uncertainty estimates.
  • Subsample rate = 0.5
    Used for GBT ensembles to induce stochasticity (Section 6.1); a hand-chosen hyperparameter inherited from Malinin et al. (2021).
assumptions (3)
  • domain assumption Ordinal labels can be encoded as integers 1..K and variance is computed on these values, implying equal distances between adjacent categories.
    Used in Equations (12)-(14) and in the binary variance reductions; the authors note the equal-distance assumption is disputable (Section 3.2).
  • domain assumption The posterior predictive distribution is approximated by averaging over a finite ensemble of M models (Eq. 5).
    Standard Monte Carlo approximation of Bayesian inference; required for all computed measures.
  • ad hoc to paper Overall ordinal uncertainty is the sum of K-1 binary split uncertainties (Eq. 16).
    Proposed without axiomatic justification; motivated by analogy to binary decomposition of classifiers. No derivation shows this sum is the correct or unique aggregation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Aleatoric and Epistemic Uncertainty Measures for Ordinal Classification through Binary Reduction." pith.science (2026). https://pith.science/paper/IRJMZ6VZ

@misc{pith2026250700733,
  author       = {Pith},
  title        = {Pith review of: Aleatoric and Epistemic Uncertainty Measures for Ordinal Classification through Binary Reduction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IRJMZ6VZ}},
  note         = {Machine review of arXiv:2507.00733}
}
read the original abstract

Ordinal classification problems, where labels exhibit a natural order, are prevalent in high-stakes fields such as medicine and finance. Accurate uncertainty quantification, including the decomposition into aleatoric (inherent variability) and epistemic (lack of knowledge) components, is crucial for reliable decision-making. However, existing research has primarily focused on nominal classification and regression. In this paper, we introduce a novel class of measures of aleatoric and epistemic uncertainty in ordinal classification, which is based on a suitable reduction to (entropy- and variance-based) measures for the binary case. These measures effectively capture the trade-off in ordinal classification between exact hit-rate and minimial error distances. We demonstrate the effectiveness of our approach on various tabular ordinal benchmark datasets using ensembles of gradient-boosted trees and multi-layer perceptrons for approximate Bayesian inference. Our method significantly outperforms standard and label-wise entropy and variance-based measures in error detection, as indicated by misclassification rates and mean absolute error. Additionally, the ordinal measures show competitive performance in out-of-distribution (OOD) detection. Our findings highlight the importance of considering the ordinal nature of classification problems when assessing uncertainty.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

78 extracted references · 46 canonical work pages

  1. [1]

    write newline

    " write newline " cite write " FUNCTION editor.postfix editor num.names #1 > "( )" "( )" if FUNCTION editor.trans.postfix editor num.names #1 > "( )" "( )" if FUNCTION trans.postfix translator num.names #1 > "( )" "( )" if FUNCTION authors.editors.reflist.apa5 'field := 'dot := field num.names 'numnames := numnames 'format.num.names := format.num.names na...

  2. [2]

    sn-aps.bst

    FUNCTION identify.aps.version "sn-aps.bst" " [2024/07/19 v1.1 APS bibliography style]" * top ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key keywords month note number organization pages publisher school series title type url volume year eprint archive archivePrefix primaryClass adsurl adsnote version lab...

    2024

  3. [3]

    write newline

    " write newline "" before.all 'output.state := FUNCTION if.digit duplicate "0" = swap duplicate "1" = swap duplicate "2" = swap duplicate "3" = swap duplicate "4" = swap duplicate "5" = swap duplicate "6" = swap duplicate "7" = swap duplicate "8" = swap "9" = or or or or or or or or or FUNCTION n.separate 't := "" #0 'numnames := t empty not t #-1 #1 subs...

  4. [4]

    , Schapire, R

    allw_rm01 APACrefauthors Allwein, E. , Schapire, R. Singer, Y. APACrefauthors \ 2001 . Reducing multiclass to binary: a unifying approach for margin classifiers Reducing multiclass to binary: a unifying approach for margin classifiers . The Journal of Machine Learning Research 1 113--141,

    work page 2001

  5. [5]

    APACrefauthors \ 1984

    anderson1984regression APACrefauthors Anderson, J.A. APACrefauthors \ 1984 . Regression and ordered categorical variables Regression and ordered categorical variables . Journal of the Royal Statistical Society: Series B (Methodological) 46 1 1--22,

    work page 1984

  6. [6]

    , Esuli, A

    DBLP:conf/isda/BaccianellaES09 APACrefauthors Baccianella, S. , Esuli, A. Sebastiani, F. APACrefauthors \ 2009 . Evaluation Measures for Ordinal Regression Evaluation measures for ordinal regression . Ninth International Conference on Intelligent Systems Design and Applications, ISDA 2009, Pisa, Italy , November 30-December 2, 2009 Ninth international con...

    2009

  7. [7]

    , Corani, G

    benavoli2016should APACrefauthors Benavoli, A. , Corani, G. Mangili, F. APACrefauthors \ 2016 . Should we really use post-hoc tests based on mean-ranks? Should we really use post-hoc tests based on mean-ranks? The Journal of Machine Learning Research 17 1 152--161,

    work page 2016

  8. [8]

    , Ayll \'o n-Gavil \'a n, R

    berchez2025dlordinal APACrefauthors B \'e rchez-Moreno, F. , Ayll \'o n-Gavil \'a n, R. , Vargas, V.M. , Guijo-Rubio, D. , Herv \'a s-Mart \' nez, C. , Fern \'a ndez, J.C. Guti \'e rrez, P.A. APACrefauthors \ 2025 . dlordinal: A Python package for deep ordinal classification dlordinal: A python package for deep ordinal classification . Neurocomputing 1293...

    arXiv 2025

Show all 78 references
  1. [9]

    APACrefauthors \ 1950

    brier1950verification APACrefauthors Brier, G.W. APACrefauthors \ 1950 . Verification of forecasts expressed in terms of probability Verification of forecasts expressed in terms of probability . Monthly weather review 78 1 1--3,

    1950

  2. [10]

    u lte, C. , Sale, Y. , L \

    bulte2025axiomatic APACrefauthors B \"u lte, C. , Sale, Y. , L \"o hr, T. , Hofman, P. , Kutyniok, G. H \"u llermeier, E. APACrefauthors \ 2025 . An Axiomatic Assessment of Entropy-and Variance-based Uncertainty Quantification in Regression An axiomatic assessment of entropy-a...

    2025 arXiv

  3. [11]

    , Mirjalili, V

    DBLP:journals/prl/CaoMR20 APACrefauthors Cao, W. , Mirjalili, V. Raschka, S. APACrefauthors \ 2020 . Rank consistent ordinal regression for neural networks with application to age estimation Rank consistent ordinal regression for neural networks with application to age estimat...

    2020 doi

  4. [12]

    \ Guestrin, C

    DBLP:conf/kdd/ChenG16 APACrefauthors Chen, T. \ Guestrin, C. APACrefauthors \ 2016 . XGBoost: A Scalable Tree Boosting System Xgboost: A scalable tree boosting system . Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Fran...

    2016

  5. [13]

    APACrefauthors \ 1960

    cohen1960coefficient APACrefauthors Cohen, J. APACrefauthors \ 1960 . A coefficient of agreement for nominal scales A coefficient of agreement for nominal scales . Educational and psychological measurement 20 1 37--46,

    1960

  6. [14]

    , Herv \' a s - Mart \' nez, C

    DBLP:journals/ijon/Cruz-RamirezHSG14 APACrefauthors Cruz - Ram \' rez, M. , Herv \' a s - Mart \' nez, C. , S \' a nchez - Monedero, J. Guti \' e rrez, P.A. APACrefauthors \ 2014 . Metrics to guide a multi-objective evolutionary algorithm for ordinal classification Metrics to ...

    2014 doi

  7. [15]

    , Puig, D

    DBLP:journals/prl/TorrePV18 APACrefauthors de La Torre, J. , Puig, D. Valls, A. APACrefauthors \ 2018 . Weighted kappa loss function for multi-class classification of ordinal data in deep learning Weighted kappa loss function for multi-class classification of ordinal data in d...

    2018 doi

  8. [16]

    , Song, H

    DBLP:journals/ml/FilhoSPSKF23 APACrefauthors de Menezes e Silva Filho, T. , Song, H. , Perell \' o - Nieto, M. , Santos - Rodr \' guez, R. , Kull, M. Flach, P.A. APACrefauthors \ 2023 . Classifier calibration: a survey on how to assess and improve predicted class probabilities...

    2023 doi

  9. [17]

    APACrefauthors \ 2006

    demsar2006statistical APACrefauthors Demsar, J. APACrefauthors \ 2006 . Statistical Comparisons of Classifiers over Multiple Data Sets Statistical comparisons of classifiers over multiple data sets . The Journal of Machine Learning Research 7 1--30,

    2006

  10. [18]

    , Hernandez-Lobato, J M

    pmlr-v80-depeweg18a APACrefauthors Depeweg, S. , Hernandez-Lobato, J M. , Doshi-Velez, F. Udluft, S. APACrefauthors \ 2018 10--15 Jul . Decomposition of Uncertainty in B ayesian Deep Learning for Efficient and Risk-sensitive Learning Decomposition of uncertainty in B ayesian d...

    2018

  11. [19]

    APACrefauthors \ 1969

    epstein1969scoring APACrefauthors Epstein, E.S. APACrefauthors \ 1969 . A scoring system for probability forecasts of ranked categories A scoring system for probability forecasts of ranked categories . Journal of Applied Meteorology (1962-1982) 8 6 985--987,

    1969

  12. [20]

    \ Hall, M.A

    DBLP:conf/ecml/FrankH01 APACrefauthors Frank, E. \ Hall, M.A. APACrefauthors \ 2001 . A Simple Approach to Ordinal Classification A simple approach to ordinal classification . Machine Learning: EMCL 2001, 12th European Conference on Machine Learning, Freiburg, Germany, Septemb...

    2001

  13. [21]

    APACrefauthors \ 2001

    friedman2001greedy APACrefauthors Friedman, J.H. APACrefauthors \ 2001 . Greedy function approximation: a gradient boosting machine Greedy function approximation: a gradient boosting machine . Annals of statistics 1189--1232,

    2001

  14. [22]

    \ Ghahramani, Z

    DBLP:conf/icml/GalG16 APACrefauthors Gal, Y. \ Ghahramani, Z. APACrefauthors \ 2016 . Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning Dropout as a bayesian approximation: Representing model uncertainty in deep learning . Proceedings of the ...

    2016

  15. [23]

    APACrefauthors \ 2023

    DBLP:conf/miccai/Galdran23 APACrefauthors Galdran, A. APACrefauthors \ 2023 . Performance Metrics for Probabilistic Ordinal Classifiers Performance metrics for probabilistic ordinal classifiers . Medical Image Computing and Computer Assisted Intervention - MICCAI 2023 - 26th I...

    2023

  16. [24]

    \ Japkowicz, N

    DBLP:conf/ai/GaudetteJ09 APACrefauthors Gaudette, L. \ Japkowicz, N. APACrefauthors \ 2009 . Evaluation Methods for Ordinal Classification Evaluation methods for ordinal classification . Advances in Artificial Intelligence, 22nd Canadian Conference on Artificial Intelligence, ...

    2009

  17. [25]

    \ El - Yaniv, R

    DBLP:conf/nips/GeifmanE17 APACrefauthors Geifman, Y. \ El - Yaniv, R. APACrefauthors \ 2017 . Selective Classification for Deep Neural Networks Selective classification for deep neural networks . Advances in Neural Information Processing Systems 30: Annual Conference on Neural...

    2017

  18. [26]

    \ Raftery, A.E

    gneiting2007strictly APACrefauthors Gneiting, T. \ Raftery, A.E. APACrefauthors \ 2007 . Strictly proper scoring rules, prediction, and estimation Strictly proper scoring rules, prediction, and estimation . Journal of the American Statistical Association 102 477 359--378,

    2007

  19. [27]

    , Oyallon, E

    DBLP:conf/nips/GrinsztajnOV22 APACrefauthors Grinsztajn, L. , Oyallon, E. Varoquaux, G. APACrefauthors \ 2022 . Why do tree-based models still outperform deep learning on typical tabular data? Why do tree-based models still outperform deep learning on typical tabular data? Adv...

    2022

  20. [28]

    , P \' e rez - Ortiz, M

    DBLP:journals/tkde/GutierrezPSFH16 APACrefauthors Guti \' e rrez, P.A. , P \' e rez - Ortiz, M. , S \' a nchez - Monedero, J. , Fern \' a ndez - Navarro, F. Herv \' a s - Mart \' nez, C. APACrefauthors \ 2016 . Ordinal Regression Methods: Survey and Experimental Study Ordinal ...

    2016

  21. [29]

    \ H \" u llermeier, E

    DBLP:conf/pkdd/HaasH22 APACrefauthors Haas, S. \ H \" u llermeier, E. APACrefauthors \ 2022 . A Prescriptive Machine Learning Approach for Assessing Goodwill in the Automotive Domain A prescriptive machine learning approach for assessing goodwill in the automotive domain . Mac...

    2022

  22. [30]

    \ H \" u llermeier, E

    DBLP:conf/pkdd/HaasH23 APACrefauthors Haas, S. \ H \" u llermeier, E. APACrefauthors \ 2023 . Rectifying Bias in Ordinal Observational Data Using Unimodal Label Smoothing Rectifying bias in ordinal observational data using unimodal label smoothing . Machine Learning and Knowle...

    2023

  23. [31]

    \ H \"u llermeier, E

    haas2024conformalized APACrefauthors Haas, S. \ H \"u llermeier, E. APACrefauthors \ 2024 . Conformalized prescriptive machine learning for uncertainty-aware automated decision making: the case of goodwill requests Conformalized prescriptive machine learning for uncertainty-aw...

    2024

  24. [32]

    \ Hüllermeier, E

    haas4965714uncertainty APACrefauthors Haas, S. \ Hüllermeier, E. APACrefauthors \ 2025 . Uncertainty Quantification in Ordinal Classification: A Comparison of Measures Uncertainty quantification in ordinal classification: A comparison of measures . International Journal of App...

    2025

  25. [33]

    , Perini, L

    DBLP:journals/ml/HendrickxPPMD24 APACrefauthors Hendrickx, K. , Perini, L. , der Plas, D.V. , Meert, W. Davis, J. APACrefauthors \ 2024 . Machine learning with a reject option: a survey Machine learning with a reject option: a survey . Mach. Learn. 113 5 3073--3110, APACrefDOI...

    2024 doi

  26. [34]

    \ Gimpel, K

    DBLP:conf/iclr/HendrycksG17 APACrefauthors Hendrycks, D. \ Gimpel, K. APACrefauthors \ 2017 . A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks A baseline for detecting misclassified and out-of-distribution examples in neural networks ....

    2017

  27. [35]

    DBLP:journals/corr/HouYS16 APACrefauthors Hou, L. , Yu, C. Samaras, D. APACrefauthors \ 2016 . Squared Earth Mover's Distance-based Loss for Training Deep Neural Networks Squared earth mover's distance-based loss for training deep neural networks . CoRR abs/1611.05916 , https:...

    2016 arXiv

  28. [36]

    \ H \" u llermeier, E

    DBLP:journals/ijdmmm/HuhnH08 APACrefauthors Huhn, J.C. \ H \" u llermeier, E. APACrefauthors \ 2008 . Is an ordinal class structure useful in classifier learning? Is an ordinal class structure useful in classifier learning? Int. J. Data Min. Model. Manag. 1 1 45--67, APACrefDO...

    2008 arXiv

  29. [37]

    \ H \" u llermeier, E

    DBLP:journals/tfs/HuhnH09 APACrefauthors Huhn, J.C. \ H \" u llermeier, E. APACrefauthors \ 2009 . FR3: A Fuzzy Rule Learner for Inducing Reliable Classifiers FR3: A fuzzy rule learner for inducing reliable classifiers . IEEE Trans. Fuzzy Syst. 17 1 138--149, APACrefDOI doi:10...

    2009

  30. [38]

    \ Waegeman, W

    DBLP:journals/ml/HullermeierW21 APACrefauthors H \" u llermeier, E. \ Waegeman, W. APACrefauthors \ 2021 . Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods Aleatoric and epistemic uncertainty in machine learning: an introduction ...

    2021 doi

  31. [39]

    , Goel, A

    DBLP:conf/acl/KasaGGRPBM24 APACrefauthors Kasa, S.R. , Goel, A. , Gupta, K. , Roychowdhury, S. , Priyatam, P. , Bhanushali, A. Murthy, P.S. APACrefauthors \ 2024 . Exploring Ordinality in Text Classification: A Comparative Study of Explicit and Implicit Techniques Exploring or...

    2024

  32. [40]

    , Meng, Q

    DBLP:conf/nips/KeMFWCMYL17 APACrefauthors Ke, G. , Meng, Q. , Finley, T. , Wang, T. , Chen, W. , Ma, W. Liu, T. APACrefauthors \ 2017 . LightGBM: A Highly Efficient Gradient Boosting Decision Tree Lightgbm: A highly efficient gradient boosting decision tree . Advances in Neura...

    2017

  33. [41]

    , Longjohn, R

    kelly2023uci APACrefauthors Kelly, M. , Longjohn, R. Nottingham, K. APACrefauthors \ 2023 . The UCI machine learning repository The uci machine learning repository . URL https://archive. ics. uci. edu ,

    2023

  34. [42]

    \ Gal, Y

    DBLP:conf/nips/KendallG17 APACrefauthors Kendall, A. \ Gal, Y. APACrefauthors \ 2017 . What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? What uncertainties do we need in bayesian deep learning for computer vision? Advances in Neural Information Proce...

    2017

  35. [43]

    , Gummadi, P.K

    DBLP:journals/ml/LahotiGW23 APACrefauthors Lahoti, P. , Gummadi, P.K. Weikum, G. APACrefauthors \ 2023 . Responsible model deployment via model-agnostic uncertainty learning Responsible model deployment via model-agnostic uncertainty learning . Mach. Learn. 112 3 939--970, APA...

    2023 doi

  36. [44]

    \ Lin, H

    DBLP:conf/nips/LiL06 APACrefauthors Li, L. \ Lin, H. APACrefauthors \ 2006 . Ordinal Regression by Extended Binary Classification Ordinal regression by extended binary classification . Advances in Neural Information Processing Systems 19, Proceedings of the Twentieth Annual Co...

    2006

  37. [45]

    , Wang, J

    DBLP:conf/cvpr/LiWYL0YWP22 APACrefauthors Li, Q. , Wang, J. , Yao, Z. , Li, Y. , Yang, P. , Yan, J. Pu, S. APACrefauthors \ 2022 . Unimodal-Concentrated Loss: Fully Adaptive Label Distribution Learning for Ordinal Regression Unimodal-concentrated loss: Fully adaptive label dis...

    2022

  38. [46]

    , Fan, F

    DBLP:journals/ijon/LiuFKDXLY20 APACrefauthors Liu, X. , Fan, F. , Kong, L. , Diao, Z. , Xie, W. , Lu, J. You, J. APACrefauthors \ 2020 . Unimodal regularized neuron stick-breaking for ordinal classification Unimodal regularized neuron stick-breaking for ordinal classification ...

    2020 doi

  39. [47]

    o hr, T. , Ingrisch, M. H \

    DBLP:conf/aime/LohrIH24 APACrefauthors L \" o hr, T. , Ingrisch, M. H \" u llermeier, E. APACrefauthors \ 2024 . Towards Aleatoric and Epistemic Uncertainty in Medical Image Classification Towards aleatoric and epistemic uncertainty in medical image classification . Artificial...

    2024

  40. [48]

    APACrefauthors \ 2019

    malinin2019uncertainty APACrefauthors Malinin, A. APACrefauthors \ 2019 . \ Uncertainty estimation in deep learning with application to spoken language assessment Uncertainty estimation in deep learning with application to spoken language assessment \

    2019

  41. [49]

    , Prokhorenkova, L

    DBLP:conf/iclr/MalininPU21 APACrefauthors Malinin, A. , Prokhorenkova, L. Ustimenko, A. APACrefauthors \ 2021 . Uncertainty in Gradient Boosting via Ensembles Uncertainty in gradient boosting via ensembles . 9th International Conference on Learning Representations, ICLR 2021, ...

    2021

  42. [50]

    , Nguyen, H.V

    DBLP:journals/corr/abs-1906-04569 APACrefauthors Mobiny, A. , Nguyen, H.V. , Moulik, S. , Garg, N. Wu, C.C. APACrefauthors \ 2019 . DropConnect Is Effective in Modeling Uncertainty of Bayesian Deep Networks Dropconnect is effective in modeling uncertainty of bayesian deep netw...

    1906 arXiv

  43. [51]

    , Kirchhof, M

    DBLP:journals/corr/abs-2402-19460 APACrefauthors Mucs \' a nyi, B. , Kirchhof, M. Oh, S.J. APACrefauthors \ 2024 . Benchmarking Uncertainty Disentanglement: Specialized Uncertainties for Specialized Tasks Benchmarking uncertainty disentanglement: Specialized uncertainties for ...

    2024 doi arXiv

  44. [52]

    , Zucker, J

    DBLP:journals/jmlr/NadeemZH10 APACrefauthors Nadeem, M.S.A. , Zucker, J. Hanczar, B. APACrefauthors \ 2010 . Accuracy-Rejection Curves (ARCs) for Comparing Classification Methods with a Reject Option Accuracy-rejection curves (arcs) for comparing classification methods with a ...

    2010

  45. [53]

    , Zhou, M

    DBLP:conf/cvpr/NiuZWGH16 APACrefauthors Niu, Z. , Zhou, M. , Wang, L. , Gao, X. Hua, G. APACrefauthors \ 2016 . Ordinal Regression with Multiple Output CNN for Age Estimation Ordinal regression with multiple output CNN for age estimation . 2016 IEEE Conference on Computer Visi...

    2016

  46. [54]

    , Varoquaux, G

    scikit-learn APACrefauthors Pedregosa, F. , Varoquaux, G. , Gramfort, A. , Michel, V. , Thirion, B. , Grisel, O. Duchesnay, E. APACrefauthors \ 2011 . Scikit-learn: Machine Learning in P ython Scikit-learn: Machine learning in P ython . Journal of Machine Learning Research 12 ...

    2011

  47. [55]

    , Gusev, G

    DBLP:conf/nips/ProkhorenkovaGV18 APACrefauthors Prokhorenkova, L.O. , Gusev, G. , Vorobev, A. , Dorogush, A.V. Gulin, A. APACrefauthors \ 2018 . CatBoost: unbiased boosting with categorical features Catboost: unbiased boosting with categorical features . Advances in Neural Inf...

    2018

  48. [56]

    , Islam, S.R

    RAFIQUE20214003 APACrefauthors Rafique, R. , Islam, S.R. Kazi, J.U. APACrefauthors \ 2021 . Machine learning in the prediction of cancer therapy Machine learning in the prediction of cancer therapy . Computational and Structural Biotechnology Journal 19 4003-4017, APACrefDOI d...

    2021 doi

  49. [57]

    \ Klautau, A

    DBLP:journals/jmlr/RifkinK03 APACrefauthors Rifkin, R.M. \ Klautau, A. APACrefauthors \ 2004 . In Defense of One-Vs-All Classification In defense of one-vs-all classification . J. Mach. Learn. Res. 5 101--141,

    2004

  50. [58]

    , Shaker, M.H

    HosseinIce2024 APACrefauthors Saberi, N. , Shaker, M.H. , Duguay, C. , Scott, K.A. Hüllermeier, E. APACrefauthors \ 2024 . Uncertainty Estimation of Lake Ice Cover Maps from A Random Forest Classifier using MODIS TOA Reflectance Data Uncertainty estimation of lake ice cover ma...

    2024

  51. [59]

    ohr, T. , Wimmer, L. , Nagler, T. H\

    labelwise APACrefauthors Sale, Y. , Hofman, P. , L\"ohr, T. , Wimmer, L. , Nagler, T. H\"ullermeier, E. APACrefauthors \ 2024 . Label-wise Aleatoric and Epistemic Uncertainty Quantification Label-wise aleatoric and epistemic uncertainty quantification . Proc.\ UAI, Conference ...

    2024

  52. [60]

    o sner, S. , Dembczynski, K. , Haasenritter, J. , Hirsch, O. , Donner - Banzhoff, N. H \

    DBLP:journals/isci/SengeBDHHDH14 APACrefauthors Senge, R. , B \" o sner, S. , Dembczynski, K. , Haasenritter, J. , Hirsch, O. , Donner - Banzhoff, N. H \" u llermeier, E. APACrefauthors \ 2014 . Reliable classification: Learning classifiers that distinguish aleatoric and epist...

    2014 doi

  53. [61]

    \ H \" u llermeier, E

    DBLP:conf/ida/ShakerH20 APACrefauthors Shaker, M.H. \ H \" u llermeier, E. APACrefauthors \ 2020 . Aleatoric and Epistemic Uncertainty with Random Forests Aleatoric and epistemic uncertainty with random forests . Advances in Intelligent Data Analysis XVIII - 18th International...

    2020

  54. [62]

    \ H \"u llermeier, E

    shaker2021ensemble APACrefauthors Shaker, M.H. \ H \"u llermeier, E. APACrefauthors \ 2021 . Ensemble-based uncertainty quantification: Bayesian versus credal inference Ensemble-based uncertainty quantification: Bayesian versus credal inference . Proceedings 31. Workshop Compu...

    2021

  55. [63]

    APACrefauthors \ 1948

    shannon1948mathematical APACrefauthors Shannon, C.E. APACrefauthors \ 1948 . A mathematical theory of communication A mathematical theory of communication . The Bell System Technical Journal 27 3 379--423,

    1948

  56. [64]

    \ Armon, A

    DBLP:journals/inffus/Shwartz-ZivA22 APACrefauthors Shwartz - Ziv, R. \ Armon, A. APACrefauthors \ 2022 . Tabular data: Deep learning is not all you need Tabular data: Deep learning is not all you need . Inf. Fusion 81 84--90, APACrefDOI doi:10.1016/J.INFFUS.2021.11.011 APACref...

    2022 doi

  57. [65]

    , Uddin Ahamed , M.K

    UDDIN2023327 APACrefauthors Uddin, N. , Uddin Ahamed , M.K. , Uddin, M.A. , Islam, M.M. , Talukder, M.A. Aryal, S. APACrefauthors \ 2023 . An ensemble machine learning based bank loan approval predictions system with a smart application An ensemble machine learning based bank ...

    2023 doi

  58. [66]

    , Van Rijn, J.N

    vanschoren2014openml APACrefauthors Vanschoren, J. , Van Rijn, J.N. , Bischl, B. Torgo, L. APACrefauthors \ 2014 . OpenML: networked science in machine learning Openml: networked science in machine learning . ACM SIGKDD Explorations Newsletter 15 2 49--60,

    2014

  59. [67]

    , Sale, Y

    DBLP:conf/uai/WimmerSHBH23 APACrefauthors Wimmer, L. , Sale, Y. , Hofman, P. , Bischl, B. H \" u llermeier, E. APACrefauthors \ 2023 . Quantifying aleatoric and epistemic uncertainty in machine learning: Are conditional entropy and mutual information appropriate measures? Quan...

    2023

  60. [68]

    , G \' o mez - Orellana, A.M

    DBLP:conf/caepia/YunGGBGH24 APACrefauthors Yun, V.M.V. , G \' o mez - Orellana, A.M. , Guijo - Rubio, D. , B \' e rchez - Moreno, F. , Guti \' e rrez, P.A. Herv \' a s - Mart \' nez, C. APACrefauthors \ 2024 . Age Estimation Using Soft Labelling Ordinal Classification Approach...

    2024

  61. [69]

    sn-basic.bst

    FUNCTION identify.basic.version "sn-basic.bst" " [2024/07/19 v1.1 bibliography style]" * top ENTRY address archive author booktitle chapter doi edition editor eid eprint howpublished institution journal key keywords month note number organization pages publisher school series ...

    2024

  62. [70]

    write newline

    " write newline "" before.all 'output.state := FUNCTION add.period duplicate empty 'skip "." * add.blank if FUNCTION if.digit duplicate "0" = swap duplicate "1" = swap duplicate "2" = swap duplicate "3" = swap duplicate "4" = swap duplicate "5" = swap duplicate "6" = swap dupl...

  63. [71]

    write newline

    " write newline "" before.all 'output.state := FUNCTION output.doi doi empty skip "doi:" doi * "" * output if FUNCTION format.archive archivePrefix empty "" archivePrefix ":" * if FUNCTION format.primaryClass primaryClass empty "" " [" primaryClass * "] " * if FUNCTION format....

  64. [72]

    write newline

    " write newline "" before.all 'output.state := FUNCTION string.to.integer 't := t text.length 'k := #1 'char.num := t char.num #1 substring 's := s is.num s "." = or char.num k = not and char.num #1 + 'char.num := while char.num #1 - 'char.num := t #1 char.num substring FUNCTI...

  65. [73]

    write newline

    " write newline "" before.all 'output.state := FUNCTION string.to.integer 't := t text.length 'k := #1 'char.num := t char.num #1 substring 's := s is.num s "." = or char.num k = not and char.num #1 + 'char.num := while char.num #1 - 'char.num := t #1 char.num substring FUNCTI...

  66. [74]

    sn-nature.bst

    FUNCTION identify.nature.version "sn-nature.bst" " [2024/07/19 v1.1 bibliography style]" * top ENTRY address archive author booktitle chapter edition editor eprint howpublished institution journal key keywords month note number organization pages publisher school series title ...

    2024

  67. [75]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  68. [76]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  69. [77]

    sn-vancouver-num.bst

    FUNCTION identify.vancouver.version "sn-vancouver-num.bst" " [2024/07/19 v1.1 Vancouver bibliography style]" * top ENTRY address assignee author booktitle chapter cartographer day edition editor howpublished institution inventor journal key keywords month note number organizat...

    2024

  70. [78]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.