Pith. sign in

REVIEW 7 major objections 7 minor 77 references

Absolute Evaluation Measures for Machine Learning: A Survey

T0 review · 7 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A survey shows how to pick evaluation measures that work across datasets

desk verdict Useful taxonomy and decision trees, but the equations are too unreliable for a reference survey. read the letter →

arxiv 2507.03392 v1 pith:VOXW2PB3 submitted 2025-07-04 cs.LG

Silvia Beddar-Wiesing , Alice Moallemy-Oureh , Marie Kempkes , Josephine M. Thomas This is my paper · ORCID
classification cs.LG
keywords absoluteevaluationmeasuresclassificationmetricsclusteringrankingchancecorrectionimbalanceddatametricselectiondecisiontrees
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that machine-learning models can be compared on a fixed scale by using absolute evaluation measures, metrics whose values lie in a fixed interval and do not depend on a reference model or on the dataset's range. To make that possible in practice, it surveys classification, clustering, and ranking measures, organizes them by data balance, number of classes, chance correction, and which error type matters, and condenses the advice into decision trees. The payoff would be a principled way to ask how well a single model performs on a single dataset, and to compare that number across datasets and model architectures. The paper deliberately leaves regression out, since residual-based losses such as RMSE are not absolute in its sense.

What carries the argument

The central object is the absolute evaluation measure itself, defined as a function carrying a learning problem, a dataset, and a model to a value in a fixed interval $[a,b]$, together with the confusion matrix that most classification measures build on. The survey's machinery is categorical: it partitions measures by class balance, class count, availability of ground truth, chance correction, and error priority, then encodes the partition as decision trees. These categories do the argumentative work, because the choice of metric is reduced to matching the task's data and error profile to the correct cell of the tree.

What would settle it

Take a fixed confusion matrix from a public benchmark, compute each formula in Sections 2 through 4, and compare it with the value obtained directly from the cited source's definition; a material discrepancy in any one formula, such as the multi-class kappa of Eq. (22) or the average accuracy of Eq. (9), would mean the decision trees can route practitioners to a measure that is not the one they think they are using.

Watch

Extended reading notes

Core claim

On its own terms, the survey's central claim is that the field has a manageable set of absolute evaluation measures, and that they can be organized into a small number of selection rules. The organizing distinctions are: binary versus multi-class data, balanced versus imbalanced class distributions, single-class versus multi-class tasks, chance-corrected versus not chance-corrected, and false-positive versus false-negative sensitivity. Each section presents formulas for the measures in that cell and closes with a decision tree that routes a practitioner to a recommended metric; for example, MCC and AUC-PR are recommended for severely imbalanced single-class data, ARI for clustering with ground truth, and nDCG for ranking tasks where rank position matters. If this organization is sound, the survey supplies a common vocabulary for evaluation that makes cross-model and cross-dataset comparisons more meaningful.

Load-bearing premise

The formulas in Sections 2 through 4 must faithfully reproduce the measures they cite, including the conditions under which those measures are valid.

Editorial extensions

If this is right

  • A practitioner can use the decision trees to choose a metric without reading the full metric literature, and the fixed-interval property makes the resulting scores comparable across datasets.
  • For imbalanced single-class classification the survey points to MCC and AUC-PR as the more reliable defaults, with chance-corrected measures when random agreement is a concern.
  • In clustering, ground-truth availability decides the metric: purity, the adjusted Rand index, and the Fowlkes-Mallows index when labels exist, and the silhouette coefficient when they do not.
  • In ranking, the survey separates false-positive-sensitive measures such as precision@k and AP@k, recall@k, and rank-sensitive measures such as MRR and nDCG, so the task objective maps directly to a metric.
  • Regression is excluded from the absolute-measure scheme; the paper treats regression as evaluable only through correlation-style ranking metrics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One extension the paper leaves implicit is that its decision-tree structure could be turned into a small selection tool: input data balance, class count, error priority, and ground-truth availability, and output the recommended measure.
  • The survey's 'absolute' criterion concerns the scale of the score, not dataset difficulty; cross-dataset comparisons still need paired test sets or normalized baselines to be fair.
  • A natural next step would be to audit every equation against its cited source and add a worked numerical example per measure, since the decision trees inherit their correctness from those formulas.
  • The same classification-by-balance and chance-correction distinctions could be carried over to regression, where correlation-based ranking metrics are the closest absolute analogues.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

7 major / 7 minor

Summary. This manuscript is a survey of "absolute" evaluation measures for machine learning, organized by learning problem: classification (balanced/imbalanced, single-/multi-class), clustering, and ranking, with a short section on domain-specific measures. The authors propose decision trees that guide practitioners toward a suitable metric given data characteristics (balance, number of classes, error priorities, availability of ground truth). The paper's central claim, stated in the Introduction, is that the surveyed absolute measures enable model comparison independent of the dataset, architecture, and other factors, and that the decision trees provide reliable guidance for metric selection. The survey is intended as a reference compendium that reproduces and explains the definitions of these measures.

Significance. If the displayed definitions were accurate, this survey would be a useful practitioner-oriented reference: it covers a broad set of measures across three problem types, gives a uniform way to think about chance correction, and packages the guidance into decision trees. The authors also sensibly distinguish evaluation measures from performance measures and include a discussion of type-1/type-2 error orientation. However, the value of a reference survey rests almost entirely on the fidelity of the reproduced formulas. Several central equations do not match the cited literature or are internally inconsistent, and at least one decision-tree placement contradicts the survey's own text. Because the decision trees and the Section 6 recommendations are built on these definitions, the central claim that the survey enables reliable, dataset-independent comparison is not yet supported. The manuscript contains no machine-checked proofs or reproducible code, and none are claimed; the burden falls instead on textual accuracy, which is currently not met.

major comments (7)
  1. [§2.2, Eq. (9)] The formula for "average accuracy" averages per-class one-vs-rest accuracies that include the per-class true-negative count TN_i in both numerator and denominator. The cited source [12] defines average accuracy as the mean per-class recall, TP_i/(TP_i+FN_i). For C>2, the displayed formula systematically differs from the cited measure and inflates multi-class scores. Since average accuracy is recommended by the classification decision tree, this error is load-bearing.
  2. [§2.3, Eq. (12)] The prose immediately before Eq. (12) states that balanced accuracy averages recall and precision, but the displayed formula averages TPR and TNR. These are not equivalent in general. The text and the equation must be reconciled, and the corrected version must match the cited source.
  3. [§2.4, Eq. (22)] The multi-class kappa is labeled Cohen's kappa and is claimed to be the general case of the binary kappa from Eq. (14), but its chance-agreement term is set to pe=1/C. This is the Brennan-Prediger uniform assumption, not Cohen's kappa, whose expected agreement is pe=sum_i p_i q_i in terms of the per-class marginals. The displayed formula therefore does not generalize Eq. (14) as claimed.
  4. [§2.4, Eq. (24)] The RC statistic as written is not the standard multi-class Matthews correlation coefficient. The numerator sums over indices i,j,k without defined ranges, and the denominator does not match the known R_K form. As written, Eq. (24) is not computable and does not establish the claimed [-1,1] bound. This formula needs to be replaced with a correct citation of the multi-class MCC.
  5. [§3, Eq. (25)] Cluster purity is defined as a sum over i=1..k of |C(x_i) ∩ \bar C_j|, where k is said to be the number of clusters and C(x_i) is the predicted cluster of a single sample. Cluster purity should be computed by summing over the predicted clusters C_i and taking, for each C_i, the maximum intersection with a ground-truth cluster. As written, the sum ranges over samples rather than over predicted clusters, so the expression cannot be evaluated; moreover, the denominator n introduces a further mismatch between the index range and the normalization.
  6. [§4, Eq. (33)] The definition of AP@k as 1/K times the sum of P@k_i over all ranks i=1..K is not the standard average precision. Average precision averages the precision values at the ranks at which relevant items occur, not at all cutoffs. This changes both the numeric value and the stated interpretation of AP@k as a more stable alternative to P@k.
  7. [§1.2 and Introduction] The Introduction claims that the surveyed absolute measures enable comparing models "independent of the dataset, architecture, and other factors," but Definition 1.1 formalizes an absolute measure only as a function with output in a fixed interval [a,b]. Boundedness does not imply dataset-independence: accuracy, precision@k, and cluster purity all depend on class priors, the choice of k, or the number of clusters. The paper should either weaken the claim or state the additional conditions under which the recommended measures support cross-dataset comparison.
minor comments (7)
  1. [§2, decision tree] In the classification decision tree, the H-measure is placed under "Chance-Corrected" measures, but the text in §2.3 describes it as a cost-based measure that is not chance-corrected. The tree placement should be corrected to avoid contradicting the survey's own description.
  2. [§3, after Eq. (28)] The phrase "groud truth" should read "ground truth".
  3. [Throughout] Capitalization should be standardized: "Matthews correlation coefficient," "Cohen's kappa," "Scott's pi," and "Fowlkes-Mallows index." The current variants "matthew correlation coefficient," "cohen's kappan," and "Fowlkes-Mallow" are inconsistent.
  4. [§2.3, Eq. (15)] The names p_correct and p_incorrect for the two summands of the chance-agreement probability are misleading: both summands together make up pe, and neither is the probability of correct classification by the model.
  5. [§2.1] The sentence identifying the error rate with the Brier score in the binary case is imprecise: the Brier score is a proper scoring rule for probabilistic predictions and coincides with the error rate only for hard 0/1 predictions, not in general. A clarifying qualifier would be appropriate.
  6. [§3, Eq. (29)] The statement that ARI = -1 is "total disagreement with expected rand index of 0.5" is not generally correct: the expected Rand index depends on the cluster sizes and marginal counts, so it is not always 0.5. The sentence should be rephrased.
  7. [§2.4, Eq. (23)] The notation |C| in the weighted kappa formula is ambiguous: it is used both as the total number of samples and as the number of classes in the surrounding text. A distinct symbol for the total count would prevent confusion.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the survey reproduces and organizes externally cited metric definitions; no prediction, fit, or self-citation chain exists, so no output reduces to its own input.

full rationale

This survey makes no predictive or derivational claim that could reduce to its own inputs. Its stated contribution is taxonomic: 'Our key contribution is to systematically categorize and discuss these evaluation measures based on the learning problem and the context of their applicability' (Sec. 6). Each metric formula is presented as a reproduction of an external source—Eq. (9) cites [12], Eq. (13) cites [56], Eq. (14) cites [58], Eq. (22) cites [62], Eq. (26) cites [65]—and the reference list [1]–[77] contains no prior work by the present authors, so there is no self-citation chain, no uniqueness import, and no ansatz smuggled via citation. The decision trees are summaries of the paper's own categorizations (balanced/imbalanced, type-1/type-2 appropriate, chance-corrected) and are a survey convention rather than a derivation chain. Nothing is fitted, predicted, or derived from claims, so none of the seven circularity patterns applies. The skeptic's examples (Eqs. 9, 12, 22, 24, 25, 33) are transcription or fidelity defects relative to the cited literature—for instance, Eq. (25)'s purity sums over i=1..k with C(x_i), a per-sample set, rather than over predicted clusters, and Eq. (22)'s pe=1/C is the Brennan–Prediger uniform-marginal assumption rather than Cohen's chance agreement. Inconsistency with cited sources is a correctness risk, not circularity: no output of the paper is equivalent, by construction, to the paper's own input. Under the hard rule requiring a quotable reduction, the honest finding is no significant circularity, score 0.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The survey introduces no new parameters fitted to data and no new postulated entities. Its central claim rests on two assumptions: the bounded-interval definition of 'absolute' is sufficient, and the quoted formulas are accurate reproductions of the literature. The second assumption is contradicted by several equations in the text.

assumptions (2)
  • domain assumption An evaluation measure is 'absolute' if its output lies in a fixed interval; boundedness alone is treated as sufficient for cross-dataset comparability.
    Sec. 1.2, Definition 1.1. This premise motivates the entire selection of surveyed measures and is not tested against examples or prior definitions.
  • ad hoc to paper The formulas quoted from the cited literature faithfully reproduce the sources and apply under the stated data conditions.
    The survey's practical guidance depends on this premise; Eqs. 9, 22, 23, 25, and 33 appear inconsistent with their cited sources.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Absolute Evaluation Measures for Machine Learning: A Survey." pith.science (2026). https://pith.science/paper/VOXW2PB3

@misc{pith2026250703392,
  author       = {Pith},
  title        = {Pith review of: Absolute Evaluation Measures for Machine Learning: A Survey},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VOXW2PB3}},
  note         = {Machine review of arXiv:2507.03392}
}
read the original abstract

Machine Learning is a diverse field applied across various domains such as computer science, social sciences, medicine, chemistry, and finance. This diversity results in varied evaluation approaches, making it difficult to compare models effectively. Absolute evaluation measures offer a practical solution by assessing a model's performance on a fixed scale, independent of reference models and data ranges, enabling explicit comparisons. However, many commonly used measures are not universally applicable, leading to a lack of comprehensive guidance on their appropriate use. This survey addresses this gap by providing an overview of absolute evaluation metrics in ML, organized by the type of learning problem. While classification metrics have been extensively studied, this work also covers clustering, regression, and ranking metrics. By grouping these measures according to the specific ML challenges they address, this survey aims to equip practitioners with the tools necessary to select appropriate metrics for their models. The provided overview thus improves individual model evaluation and facilitates meaningful comparisons across different models and applications.

Figures

Figures reproduced from arXiv: 2507.03392 by the authors.

Figure 1
Figure 1. ROC curves for different classifiers. A perfect classifier has an FP rate of 0 and a TP rate of 1. 0 25 50 75 100 False Negative Rate 0 20 40 60 80 100 False Positive Rate [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. DET curve example [51]. The red curve represents the best￾performing model, while the blue and black curves show poorer clas￾sification ability. Graphical measures provide an intuitive way to evaluate and compare model per￾formance by visualizing trade-offs between classification metrics across different thresholds. Among these, one of the most commonly used graphical measures is the area under the receiver operatin… view at source ↗
Figure 3
Figure 3. The area under the precision-recall curve (AUC-PR) for two models illustrates the correlation between the precision and recall for a model w.r.t. different decision thresholds. Unlike AUC-ROC, which treats all classification errors equally, the H-measure considers the specific costs of the different misclassifi￾cation errors [60]. This makes it especially useful in situations where the impact of misclassifications i… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

77 extracted references · 73 canonical work pages

  1. [12]

    A systematic analysis of performance measures for classification tasks

    Marina Sokolova and Guy Lapalme. A systematic analysis of performance measures for classification tasks. Information processing & management, 45(4):427–437, 2009

    work page 2009

  2. [1]

    A survey on the four families of performance measures

    Massimiliano Caporin, Grégory M Jannin, Francesco Lisi, and Bertrand B Maillet. A survey on the four families of performance measures. Journal of Economic Surveys, 28(5):917–942, 2014

    work page 2014

  3. [2]

    Strategy, choice of performance measures, and performance

    Wim A Van der Stede, Chee W Chow, and Thomas W Lin. Strategy, choice of performance measures, and performance. Behavioral research in accounting, 18(1):185–205, 2006

    work page 2006

  4. [3]

    Multiple perspectives of performance measures

    Robert H Chenhall and Kim Langfield-Smith. Multiple perspectives of performance measures. European management journal, 25(4):266–282, 2007

    work page 2007

  5. [4]

    An overview of frequently used performance measures

    Stefan Tangen. An overview of frequently used performance measures. Work study, 2003

    work page 2003

  6. [5]

    Learning how to active learn: A deep reinforcement learning approach

    Meng Fang, Yuan Li, and Trevor Cohn. Learning how to active learn: A deep reinforcement learning approach. arXiv preprint arXiv:1708.02383, 2017. 17 Survey on Absolute Evaluation Measures A PREPRINT

    arXiv 2017

  7. [6]

    Balancing performance measures

    Srikant Datar, Susan Cohen Kulp, and Richard A Lambert. Balancing performance measures. Journal of accounting research, 39(1):75–92, 2001

    work page 2001

  8. [7]

    Algorithms for reinforcement learning

    Csaba Szepesvári. Algorithms for reinforcement learning. Synthesis lectures on artificial intelligence and machine learning, 4(1):1–103, 2010

    work page 2010

Show all 77 references
  1. [8]

    Evaluation Metrics and Statistical Tests for Machine Learning

    Oona Rainio, Jarmo Teuho, and Riku Klén. Evaluation Metrics and Statistical Tests for Machine Learning. Scientific Reports, 14(1):6086, 2024

    2024

  2. [9]

    A Review of Evaluation Metrics in Machine Learning Algorithms

    Gireen Naidu, Tranos Zuva, and Elias Mmbongeni Sibanda. A Review of Evaluation Metrics in Machine Learning Algorithms. In Computer Science On-line Conference, pages 15–25. Springer, 2023

    2023

  3. [10]

    Performance evaluation in machine learning

    Nathalie Japkowicz and Mohak Shah. Performance evaluation in machine learning. In Machine Learning in Radiation Oncology, pages 41–56. Springer, 2015

    2015

  4. [11]

    Constructing new and better evaluation measures for machine learning

    Jin Huang and Charles X Ling. Constructing new and better evaluation measures for machine learning. In IJCAI, pages 859–864, 2007

    2007

  5. [13]

    A review on evaluation metrics for data classification evaluations

    Mohammad Hossin and Md Nasir Sulaiman. A review on evaluation metrics for data classification evaluations. International journal of data mining & knowledge management process, 5(2):1, 2015

    2015

  6. [14]

    Classification model evaluation metrics

    Z Vujovi´c. Classification model evaluation metrics. International Journal of Advanced Computer Science and Applications, 12(6):599–606, 2021

    2021

  7. [15]

    An experimental comparison of performance measures for classification

    César Ferri, José Hernández-Orallo, and R Modroiu. An experimental comparison of performance measures for classification. Pattern recognition letters, 30(1):27–38, 2009

    2009

  8. [16]

    Performance evaluation in machine learning: the good, the bad, the ugly, and the way forward

    Peter Flach. Performance evaluation in machine learning: the good, the bad, the ugly, and the way forward. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 9808–9814, 2019

    2019

  9. [17]

    A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability

    Salvador García, Alberto Fernández, Julián Luengo, and Francisco Herrera. A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability. Soft Computing, 13(10):959–977, 2009

    2009

  10. [18]

    Evaluation metrics for unsupervised learning algorithms

    Julio-Omar Palacio-Niño and Fernando Berzal. Evaluation metrics for unsupervised learning algorithms. arXiv preprint arXiv:1905.05667, 2019

    1905 arXiv

  11. [19]

    Comparison of evaluation metrics in classification applications with imbalanced datasets

    Mehrdad Fatourechi, Rabab K Ward, Steven G Mason, Jane Huggins, Alois Schlögl, and Gary E Birch. Comparison of evaluation metrics in classification applications with imbalanced datasets. In 2008 seventh international conference on machine learning and applications, pages 777–7...

    2008

  12. [20]

    Empirical study of machine learning classifier evaluation metrics behavior in massively imbalanced and noisy data

    Gayan K Kulatilleke and Sugandika Samarakoon. Empirical study of machine learning classifier evaluation metrics behavior in massively imbalanced and noisy data. arXiv preprint arXiv:2208.11904, 2022

    2022 arXiv

  13. [21]

    Survey of imbalanced data methodologies

    Lian Yu and Nengfeng Zhou. Survey of imbalanced data methodologies. arXiv preprint arXiv:2104.02240, 2021

    2021 arXiv

  14. [22]

    Learning from imbalanced data: open challenges and future directions

    Bartosz Krawczyk. Learning from imbalanced data: open challenges and future directions. Progress in Artificial Intelligence, 5(4):221–232, 2016

    2016

  15. [23]

    A Survey of Explainable Graph Neural Networks: Taxonomy and Evaluation Metrics

    Yiqiao Li, Jianlong Zhou, Sunny Verma, and Fang Chen. A Survey of Explainable Graph Neural Networks: Taxonomy and Evaluation Metrics. arXiv preprint arXiv:2207.12599, 2022

    2022 arXiv

  16. [24]

    A unified study of machine learning explanation evaluation metrics

    Yipei Wang and Xiaoqian Wang. A unified study of machine learning explanation evaluation metrics. arXiv preprint arXiv:2203.14265, 2022

    2022 arXiv

  17. [25]

    Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics

    Jianlong Zhou, Amir H Gandomi, Fang Chen, and Andreas Holzinger. Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics. Electronics, 10(5):593, 2021

    2021

  18. [26]

    Towards a guideline for evaluation metrics in medical image segmentation

    Dominik Müller, Iñaki Soto-Rey, and Frank Kramer. Towards a guideline for evaluation metrics in medical image segmentation. BMC Research Notes, 15, 12 2022

    2022

  19. [27]

    A survey on machine reading comprehension—tasks, evaluation metrics and benchmark datasets

    Changchang Zeng, Shaobo Li, Qin Li, Jie Hu, and Jianjun Hu. A survey on machine reading comprehension—tasks, evaluation metrics and benchmark datasets. Applied Sciences, 10(21), 2020

    2020

  20. [28]

    Evaluation metrics and evaluation

    Hercules Dalianis. Evaluation metrics and evaluation. In Clinical text mining, pages 45–53. Springer, 2018

    2018

  21. [29]

    How to find strong summary coherence measures? a toolbox and a comparative study for summary coherence measure evaluation

    Julius Steen and Katja Markert. How to find strong summary coherence measures? a toolbox and a comparative study for summary coherence measure evaluation. arXiv preprint arXiv:2209.06517, 2022

    2022 arXiv

  22. [30]

    Evaluating the evaluation metrics for style transfer: A case study in multilingual formality transfer

    Eleftheria Briakou, Sweta Agrawal, Joel Tetreault, and Marine Carpuat. Evaluating the evaluation metrics for style transfer: A case study in multilingual formality transfer. arXiv preprint arXiv:2110.10668, 2021. 18 Survey on Absolute Evaluation Measures A PREPRINT

    2021 arXiv

  23. [31]

    A survey of evaluation metrics used for nlg systems

    Ananya B Sai, Akash Kumar Mohankumar, and Mitesh M Khapra. A survey of evaluation metrics used for nlg systems. ACM Computing Surveys (CSUR), 55(2):1–39, 2022

    2022

  24. [32]

    Performance measures, consistency, and power for artificial neural network models

    JM Twomey and AE Smith. Performance measures, consistency, and power for artificial neural network models. Mathematical and computer modelling, 21(1-2):243–258, 1995

    1995

  25. [33]

    A survey of accuracy evaluation metrics of recommendation tasks

    Asela Gunawardana and Guy Shani. A survey of accuracy evaluation metrics of recommendation tasks. Journal of Machine Learning Research, 10(12), 2009

    2009

  26. [34]

    Gil, Julio Sahuquillo, and Ana Pont

    Josep Domènech, José A. Gil, Julio Sahuquillo, and Ana Pont. Web prefetching performance metrics: A survey. Performance Evaluation, 63(9):988–1004, 2006

    2006

  27. [35]

    Single object tracking: A survey of methods, datasets, and evaluation metrics

    Zahra Soleimanitaleb and Mohammad Ali Keyvanrad. Single object tracking: A survey of methods, datasets, and evaluation metrics. arXiv preprint arXiv:2201.13066, 2022

    2022 arXiv

  28. [36]

    Performance evaluation metrics and approaches for target tracking: A survey

    Yan Song, Zheng Hu, Tiancheng Li, and Hongqi Fan. Performance evaluation metrics and approaches for target tracking: A survey. Sensors, 22(3):793, 2022

    2022

  29. [37]

    Error Metrics and Performance Fitness Indicators for Artificial Intelligence and Machine Learning in Engineering and Sciences

    MZ Naser and Amir H Alavi. Error Metrics and Performance Fitness Indicators for Artificial Intelligence and Machine Learning in Engineering and Sciences. Architecture, Structures and Construction, 3(4):499–517, 2023

    2023

  30. [38]

    Christopher M. Bishop. Pattern Recognition and Machine Learning. Information science and statistics. Springer, New York, NY , 2006. Softcover published in 2016

    2006

  31. [39]

    Deep Learning

    Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning . MIT Press, 2016. http://www. deeplearningbook.org

    2016

  32. [40]

    Friedman

    Trevor Hastie and Robert Tibshirani and Jerome H. Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition. Springer Series in Statistics. Springer, 2009

    2009

  33. [41]

    Statistics for Engineering and Information Science

    Vladimir Naumovich Vapnik.The Nature of Statistical Learning Theory, Second Edition. Statistics for Engineering and Information Science. Springer, 2000

    2000

  34. [42]

    Cohen, Robert E

    William W. Cohen, Robert E. Schapire, and Yoram Singer. Learning to order things. J. Artif. Intell. Res. , 10:243–270, 1999

    1999

  35. [43]

    Generalization bounds for the area under the ROC curve

    Shivani Agarwal, Thore Graepel, Ralf Herbrich, Sariel Har-Peled, and Dan Roth. Generalization bounds for the area under the ROC curve. J. Mach. Learn. Res., 6:393–425, 2005

    2005

  36. [44]

    Evaluating Learning Algorithms: A Classification Perspective

    Nathalie Japkowicz and Mohak Shah. Evaluating Learning Algorithms: A Classification Perspective. Cambridge University Press, 2011

    2011

  37. [45]

    On the confusion matrix in credit scoring and its analytical properties

    Guoping Zeng. On the confusion matrix in credit scoring and its analytical properties. Communications in Statistics-Theory and Methods, 49(9):2080–2093, 2020

    2020

  38. [46]

    Warrens, and Giuseppe Jurman

    Davide Chicco, Matthijs J. Warrens, and Giuseppe Jurman. The matthews correlation coefficient (mcc) is more informative than cohen’s kappa and brier score in binary classification assessment. IEEE Access, 9:78368–78381, 2021

    2021

  39. [47]

    B. Lantz. Machine Learning with R. Community experience distilled. Packt Publishing, 2013

    2013

  40. [48]

    Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing

    Yoav Benjamini and Yosef Hochberg. Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. Journal of the Royal statistical society: series B (Methodological), 57(1):289–300, 1995

    1995

  41. [49]

    Evaluation: from precision, recall and f-measure to roc, informedness, markedness and correlation

    David MW Powers. Evaluation: from precision, recall and f-measure to roc, informedness, markedness and correlation. arXiv preprint arXiv:2010.16061, 2020

    2010 arXiv

  42. [50]

    Precision and recall for regression

    Luis Torgo and Rita Ribeiro. Precision and recall for regression. In International Conference on Discovery Science, pages 332–346. Springer, 2009

    2009

  43. [51]

    The DET Curve in Assessment of Decision Task Performance

    A Martin. The DET Curve in Assessment of Decision Task Performance. In Proc. ESCA Eur. Conf. Speech Comm. Tech., EuroSpeech, 1997, pages 1895–1898, 1997

    1997

  44. [52]

    Optimized precision - a new measure for classifier performance evaluation

    Romesh Ranawana and Vasile Palade. Optimized precision - a new measure for classifier performance evaluation. In 2006 IEEE International Conference on Evolutionary Computation, pages 2254–2261. IEEE, 2006

    2006

  45. [53]

    Further thoughts on precision

    David Gray, David Bowes, Neil Davey, Yi Sun, and Bruce Christianson. Further thoughts on precision. In 15th Annual Conference on Evaluation & Assessment in Software Engineering (EASE 2011) , pages 129–133. IET, 2011

    2011

  46. [54]

    Evaluation Measures for Models Assessment over Imbalanced Data Sets

    Mohamed Bekkar, Hassiba Kheliouane Djemaa, and Taklit Akrouf Alitouche. Evaluation Measures for Models Assessment over Imbalanced Data Sets. J Inf Eng Appl, 3(10), 2013

    2013

  47. [55]

    The advantages of the matthews correlation coefficient (mcc) over f1 score and accuracy in binary classification evaluation

    Davide Chicco and Giuseppe Jurman. The advantages of the matthews correlation coefficient (mcc) over f1 score and accuracy in binary classification evaluation. BMC Genomics, 21, 1 2020. 19 Survey on Absolute Evaluation Measures A PREPRINT

    2020

  48. [56]

    Matthews

    B.W. Matthews. Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochimica et Biophysica Acta (BBA) - Protein Structure, 405(2):442–451, 1975

    1975

  49. [57]

    The advantages of the matthews correlation coefficient (mcc) over f1 score and accuracy in binary classification evaluation

    Davide Chicco and Giuseppe Jurman. The advantages of the matthews correlation coefficient (mcc) over f1 score and accuracy in binary classification evaluation. BMC genomics, 21:1–13, 2020

    2020

  50. [58]

    Wissenschaftliche Beobachtung: eine Einführung

    Werner Greve and Dirk Wentura. Wissenschaftliche Beobachtung: eine Einführung. Beltz, 1997

    1997

  51. [59]

    The precision-recall plot is more informative than the roc plot when evaluating binary classifiers on imbalanced datasets

    Takaya Saito and Marc Rehmsmeier. The precision-recall plot is more informative than the roc plot when evaluating binary classifiers on imbalanced datasets. PloS one, 10(3):e0118432, 2015

    2015

  52. [60]

    Hand and Christoforos Anagnostopoulos

    David J. Hand and Christoforos Anagnostopoulos. Notes on the H-Measure of Classifier Performance. Advances in Data Analysis and Classification, 17(1):109–124, 2023

    2023

  53. [61]

    Fletcher, Suzanne W

    Robert H. Fletcher, Suzanne W. Fletcher, and Edward H. Wagner. Clinical Epidemiology: The Essentials . Lippincott Williams & Wilkins, 1982

    1982

  54. [62]

    Hsu and Ronald Field

    Louis M. Hsu and Ronald Field. Interrater Agreement Measures: Comments on Kappan, Cohen’s Kappa, Scott’s π, and Aickin’s α. Understanding Statistics, 2(3):205–219, 2003

    2003

  55. [63]

    Comparison of Classification Accuracy using Cohen’s Weighted Kappa

    Arie Ben-David. Comparison of Classification Accuracy using Cohen’s Weighted Kappa. Expert Systems with Applications, 34(2):825–832, 2008

    2008

  56. [64]

    Rama Rao and B

    K.V .S.N. Rama Rao and B. Manjula Josephine. Exploring the impact of optimal clusters on cluster purity. In 2018 3rd International Conference on Communication and Electronics Systems (ICCES), pages 754–757, 2018

    2018

  57. [65]

    Objective criteria for the evaluation of clustering methods

    William M Rand. Objective criteria for the evaluation of clustering methods. Journal of the American Statistical association, 66(336):846–850, 1971

    1971

  58. [66]

    Comparing Clusterings: An Overview

    Silke Wagner and Dorothea Wagner. Comparing Clusterings: An Overview. Universität Karlsruhe, Fakultät für Informatik Karlsruhe, 2007

    2007

  59. [67]

    Comparing Partitions

    Lawrence Hubert and Phipps Arabie. Comparing Partitions. Journal of classification, 2:193–218, 1985

    1985

  60. [68]

    A Method for Comparing Two Hierarchical Clusterings

    Edward B Fowlkes and Colin L Mallows. A Method for Comparing Two Hierarchical Clusterings. Journal of the American statistical association, 78(383):553–569, 1983

    1983

  61. [69]

    Comparing Clusterings — An Information Based Distance

    Marina Meil˘a. Comparing Clusterings — An Information Based Distance. Journal of multivariate analysis, 98(5):873–895, 2007

    2007

  62. [70]

    Recommender Systems: Techniques, Applications, and Challenges

    Francesco Ricci, Lior Rokach, and Bracha Shapira. Recommender Systems: Techniques, Applications, and Challenges. Springer US, New York, NY , 2022

    2022

  63. [71]

    Stamp: Short-term attention/memory priority model for session-based recommendation

    Qiao Liu, Yifu Zeng, Refuoe Mokhosi, and Haibin Zhang. Stamp: Short-term attention/memory priority model for session-based recommendation. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’18, page 1831–1839, New York...

    2018

  64. [72]

    Learning to Rank for Information Retrieval

    Tie-Yan Liu. Learning to Rank for Information Retrieval. Foundations and Trends® in Information Retrieval, 3(3):225–331, 2009

    2009

  65. [73]

    Ranking chemical structures for drug discovery: A new machine learning approach

    Shivani Agarwal, Deepak Dugar, and Shiladitya Sengupta. Ranking chemical structures for drug discovery: A new machine learning approach. Journal of chemical information and modeling, 50(5):716–731, 2010

    2010

  66. [74]

    Precision at n

    Nick Craswell. Precision at n. Springer US, Boston, MA, 2009

    2009

  67. [75]

    Jaccard/tanimoto similarity test and estimation methods for biological presence-absence data

    Neo Christopher Chung, Blazej Miasojedow, Michal Startek, and Anna Gambin. Jaccard/tanimoto similarity test and estimation methods for biological presence-absence data. BMC Bioinform., 20-S(15):644, 2019

    2019

  68. [76]

    Metrics for graph comparison: a practitioner’s guide

    Peter Wills and François G Meyer. Metrics for graph comparison: a practitioner’s guide. Plos one, 15(2), 2020

    2020

  69. [77]

    Rylee Thompson, Boris Knyazev, Elahe Ghalebi, Jungtaek Kim, and Graham W. Taylor. On evaluation metrics for graph generative models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. 20

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.