Pith. sign in

REVIEW 2 major objections 5 minor 2 cited by

The Tile: A 2D Map of Ranking Scores for Two-Class Classification

T0 review · 2 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper introduces the Tile, a two-dimensional map that gives every ranking score for two-class classifiers a definite location, with nearby locations inducing similar classifier rankings.

desk verdict The Tile is a well-executed map of an infinite family of ranking scores with sound algebra; the only real gap is the unproved completeness claim in Sec 3.3, which should be traceable to the companion paper or proved here. read the letter →

arxiv 2412.04309 v2 pith:VOSJPEFX submitted 2024-12-05 cs.CV cs.LGcs.PF

Sébastien Piérard , Anaïs Halin , Anthony Cioppa , Adrien Deliège , Marc Van Droogenbroeck This is my paper · ORCID
classification cs.CVcs.LGcs.PF
keywords two-classclassificationrankingscoresperformanceevaluationROCspaceF-betacanonicalpriorshiftno-skillperformances
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's aim is to establish that the variety of scores used to rank two-class classifiers is not arbitrary: up to ordering, every admissible ranking score is a member of one two-parameter family. The Tile is the map $(a,b) \mapsto R_{a,b}$ of the canonical members of that family, so each score, from accuracy and $F_\beta$ to exotic similarity coefficients, lands at a point whose neighbors are scores that rank classifiers similarly. The authors argue that with this map one can read off which classifier is best under any preference, see how prior class frequencies move rankings, characterize any new score by comparing it to the whole map, and understand why balanced accuracy and Cohen's kappa are not new ranking principles. If the completeness claim holds, the Tile is to classifier evaluation what a periodic table is to chemical elements: a single organizing picture for an apparently endless list.

What carries the argument

The load-bearing object is the canonical ranking score $R_{a,b}$, obtained by normalizing the general ranking score $R_I$ so that the importances on the two correct cells and on the two incorrect cells are each balanced. Explicitly, $R_{a,b}(P)=((1-a)P(\{tn\})+aP(\{tp\}))/((1-a)P(\{tn\})+(1-b)P(\{fp\})+bP(\{fn\})+aP(\{tp\}))$. The two parameters $a$ and $b$ separate two choices: which of the two correct outcomes matters more, and which of the two errors matters more. The whole argument is carried by the fact that ordering depends only on ratios of importances, so a score's ordering is a point in the unit square; the paper's geometry then reads that point through the pencil of iso-performance lines in ROC space, whose vertex lies outside the ROC square for every admissible score.

What would settle it

Take any score satisfying Axioms 2 and 3 and plot its iso-performance lines for fixed priors; if these lines do not form a pencil whose vertex lies in the gray regions outside the ROC square, or if no $(a,b)$ reproduces its ordering, the Tile's completeness fails.

Watch

Extended reading notes

Core claim

Working within the axiomatic ranking theory of the companion paper [28], the authors particularize to two-class crisp classification a family of ranking scores $R_I(P)=\mathbb{E}_P[I S]/\mathbb{E}_P[I]$, where $I$ is a nonnegative importance on the four cells of the confusion matrix and $S$ is the satisfaction indicator. Because rescaling the importances on the two correct cells and on the two incorrect cells does not change the induced ordering, every ordering has a canonical representative $R_{a,b}$ with $a,b\in[0,1]$: $a$ balances true positives against true negatives and $b$ balances false positives against false negatives. The Tile is the map $(a,b)\mapsto R_{a,b}$; the paper claims this map is complete, in the sense that for any ranking score there is a canonical score inducing exactly the same ordering, and that the orderings at different points of the Tile are all distinct. It then charts the coordinates of familiar scores—the four corners are $NPV$, $TPR$, $TNR$, and $PPV$, accuracy sits at the center, and the $F_\beta$ family runs along the right edge—and proves geometric correspondences: iso-performance lines in ROC space form a pencil whose vertex encodes $(a,b)$, and operations such as changing the predicted class, swapping classes, or shifting priors act as symmetries or deformations of the square. It also identifies curves $\gamma_\pi$ and $\gamma_\tau$ of scores that tie all no-skill performances, which explains where chance-corrected scores live.

Load-bearing premise

The completeness of the Tile rests on the companion paper's claim that every score satisfying Axioms 2 and 3 is order-equivalent to some canonical $R_{a,b}$, a claim restated here without proof.

Editorial extensions

If this is right

  • A practitioner can read from one picture which classifier wins under any preference: changing $(a,b)$ changes the winner, and the boundaries between winners are convex polygons when priors are balanced.
  • Scores that are known to be order-equivalent, such as $F_1$ and the positive Jaccard index, or balanced accuracy and Youden's index, occupy the same point on the Tile, so the map makes ranking equivalences visible at a glance.
  • Any new score can be characterized by its Kendall rank correlation against all $R_{a,b}$, producing a correlation portrait of that score.
  • With fixed priors, chance-corrected variants such as Cohen's kappa correspond to a single point on the Tile; applying Cohen's correction to any $R_{a,b}$ collapses an entire horizontal line to a point, showing a large loss of ranking diversity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the inherited completeness theorem is made fully explicit, the Tile becomes a design tool: one could invent a new evaluation score by choosing an $(a,b)$ point rather than deriving a formula, and know in advance how it ranks classifiers.
  • The same correlation portrait could be turned into a robustness measure: the area or diameter of the region where a benchmark's ranking does not change would quantify how stable a leaderboard is to score choice.
  • The two-class construction suggests extending the idea to multiclass or soft classifiers by replacing the four events with a continuous satisfaction variable, though the paper does not do that.
  • The no-skill curves $\gamma_\pi$ and $\gamma_\tau$ imply that chance-correction is not a special trick for accuracy: any point on the Tile can be corrected the way Cohen corrected accuracy, producing a score whose ranking is again on the Tile.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper introduces the Tile, a two-dimensional map of ranking scores for two-class classification. The authors particularize the framework of their companion paper A to two-class classification, defining ranking scores R_I as the ratio of an importance-weighted satisfaction expectation to the total importance expectation, and canonical scores R_{a,b} after normalization. They establish geometric correspondences between R_{a,b} and the ROC space via pencils of iso-performance lines, place many known scores (accuracy, TPR, TNR, NPV, PPV, F_beta, Jaccard, balanced accuracy, Cohen's kappa) on the Tile, and present applications: ranking classifiers, visualizing rank correlations, analyzing no-skill performances, and interpreting prior shifts. The appendix provides algebraic proofs (Lemmas 1-10) for the placements and for the effect of performance operations.

Significance. The Tile is a potentially useful visual and conceptual tool: it unifies a broad family of performance orderings on a single diagram and makes explicit a two-parameter structure behind many binary classification scores. The geometric derivations (pencils in ROC space, curves gamma_pi and gamma_tau) are elegant, and the algebraic lemmas are checkable and largely correct. The paper makes concrete, falsifiable statements (e.g., the locations of specific scores and the effect of Cohen's correction), which is a strength. Its central novelty--capturing 'all rankings' on one map--is, however, only as strong as the representability of all admissible orderings by R_I, and this point is not made self-contained.

major comments (2)
  1. [Sec. 3.3] The sentence 'for any ranking score there exists a canonical ranking score such that the orderings induced by them are equal' is the load-bearing universality claim of the paper. It is neither proved nor accompanied by a theorem reference at the point of use. Within the paper's own definition of ranking scores (Eq. 3), the statement follows from Property 2 and Definition 1 by normalizing I so that I(tn)+I(tp)=1 and I(fp)+I(fn)=1; please include this argument, or state and cite the corresponding representation theorem from paper A [28] with its hypotheses and treatment of boundary cases. As written, a reader cannot tell whether the Tile covers all admissible orderings or only the canonical family R_{a,b}.
  2. [Sec. 4.2] The formula for the location of the ordering induced by R_I is given as (a,b) = (I(tp)/(I(tn)+I(tp)), I(fp)/(I(fn)+I(fp))). By Definition 1, the second coordinate should be I(fn)/(I(fn)+I(fp)); as printed, the formula would place, for example, NPV (I(fn)=1, I(fp)=0) at b=0 instead of b=1, contradicting Table 2 and Lemma 6. This is load-bearing because this formula is the recipe for placing any ranking score on the Tile; please correct it.
minor comments (5)
  1. [Sec. 4] The sentence 'The performance orderings induced by the scores RIa,b are all different' is stated without proof. Since the Tile is advertised as having no redundancy, please provide a short proof or a reference for this injectivity claim.
  2. [Sec. 3.2] The assertions that the orderings induced by ACP, P4, and VUT are incompatible with the axioms of ranking are not demonstrated; please add a citation or a brief justification.
  3. [Sec. 4.4] The claim that Scott's pi and Fleiss's kappa do not satisfy the axioms of ranking, even for fixed priors, is unsupported; please add a reference or a counterexample.
  4. [Appendix A.3.3] In the proof of Lemma 8, the line 'dom(kappa) cap P* = { P in P(Omega,Sigma) } cap P*' appears to have a missing condition; please check the typesetting.
  5. [Abstract and Sec. 5] The abstract and conclusion say the Tile 'captures all the rankings', while the body defines ranking scores only as in Eq. (3). Please align the wording with the precise scope of the claim, for instance by saying 'all orderings induced by ranking scores of the form (3)'.

Circularity Check

1 steps flagged · score 4.0 of 10

The Tile's advertised universality ('for any ranking score there exists a canonical ranking score...') is asserted in Sec. 3.3 with no proof and only a self-citation to companion paper A; the explicit score placements are non-circular.

  1. self citation load bearing [Sec. 3.3, paragraph 'Contour plots' (supported only by footnote 1, Sec. 2.4, and App. A.3.5)]
    "Note that this geometric analysis is not peculiar to the canonical ranking scores. It is valid for all ranking scores, as for any ranking score there exists a canonical ranking score such that the orderings induced by them are equal."

    This sentence is the paper's only support for the universal claim that the Tile represents every admissible performance ordering. No proof or theorem number accompanies it; the preceding Property 2 only removes two scaling redundancies inside the already-restricted family (3), so it cannot establish that every score satisfying Axioms 2-3 is order-equivalent to some R_{a,b}. The only cited source for this completeness is the authors' own companion paper [28], invoked in footnote 1 ('paper A [28] presents an axiomatic framework and an infinite family of scores') and later in App.

full rationale

The Tile construction itself is self-contained: Definition 2 maps (a,b) to R_{a,b}, Property 2 normalizes the two scaling degrees of freedom, and the placements of accuracy, F-beta, NPV, TPR, PPV, Jaccard, Youden's index, Cohen's kappa, and weighted accuracy are supported by explicit lemmas with algebraic proofs (Lemmas 6-10). These are independent external anchors and do not reduce to the paper's inputs. The only load-bearing step that is not derived here is the converse/completeness statement in Sec. 3.3, namely that every ranking score is order-equivalent to some canonical R_{a,b}. The paper's own text does not prove this; it inherits the axiomatic characterization from the authors' companion paper A [28], a self-citation. If that companion theorem is valid, then the Tile is indeed a complete map, but this paper does not exhibit the theorem or its proof, and the surrounding reasoning only shows that the already-restricted family (3) collapses to the canonical family under rescaling. This is a genuine self-citation load-bearing step, but it is not a fitted-input or definitional circularity: the geometric and algebraic content of the Tile is independent of the missing converse proof. I therefore assign a moderate score of 4, reflecting one significant self-cited completeness premise while recognizing that the core placements and ROC/PR correspondences are not circular.

Assumptions & free parameters 0 free parameters · 3 assumptions · 2 invented entities

No parameters are fitted to data. The two Tile coordinates (a,b) are independent variables spanning [0,1]^2, and example priors such as 0.2, 0.5, and 0.8 are illustrative inputs, not fitted constants. The central claim rests on the axiomatic framework of the authors' own paper A, notably Axioms 2 and 3 and the completeness of the R_I representation, none of which is re-proved here.

assumptions (3)
  • domain assumption Axiom 2 (from paper A [28]): If one classifier's satisfaction is for sure less or equal than another's, the former is not better than the latter.
    Adopted as a postulate defining which orderings count as valid; invoked in Sec 2.4 and in the geometric interpretation of the red point in Sec 3.3.
  • domain assumption Axiom 3 (from paper A [28]): Convex perturbation or combination of classifiers cannot produce a classifier better than the best or worse than the worst of the original ones.
    Adopted as a postulate restricting admissible ranking scores; used in Sec 3.3 to justify restrictions on the vertex of iso-performance pencils.
  • domain assumption Completeness: every ranking score in the axiomatic sense is order-equivalent to some canonical R_{a,b}.
    Stated in Sec 3.3 without proof and without a theorem number from paper A. The Tile's coverage of 'all rankings' depends on this characterization.
invented entities (2)
  • Importance random variable I (from paper A) independent evidence
    purpose: Assigns non-negative weights to the four outcomes of a two-class confusion matrix; defines the ranking score family R_I and the canonical scores R_{a,b}.
    Not a physical entity; it is a mathematical construction. Its consequences are checkable because the paper's placements of known scores (F_beta, accuracy, rates, predictive values) can be verified directly.
  • The Tile mapping (a,b) -> R_{a,b} independent evidence
    purpose: A 2D map that locates every canonical ranking score and serves as a visualization and analysis tool for ranking classifiers.
    The Tile is a named mathematical object; it has falsifiable content since the positions of known scores and curves gamma_pi and gamma_tau can be checked against score definitions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Tile: A 2D Map of Ranking Scores for Two-Class Classification." pith.science (2026). https://pith.science/paper/VOSJPEFX

@misc{pith2026241204309,
  author       = {Pith},
  title        = {Pith review of: The Tile: A 2D Map of Ranking Scores for Two-Class Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VOSJPEFX}},
  note         = {Machine review of arXiv:2412.04309}
}
read the original abstract

In the computer vision and machine learning communities, as well as in many other research domains, rigorous evaluation of any new method, including classifiers, is essential. One key component of the evaluation process is the ability to compare and rank methods. However, ranking classifiers and accurately comparing their performances, especially when taking application-specific preferences into account, remains challenging. For instance, commonly used evaluation tools like Receiver Operating Characteristic (ROC) and Precision/Recall (PR) spaces display performances based on two scores. Hence, they are inherently limited in their ability to compare classifiers across a broader range of scores and lack the capability to establish a clear ranking among classifiers. In this paper, we present a novel versatile tool, named the Tile, that organizes an infinity of ranking scores in a single 2D map for two-class classifiers, including common evaluation scores such as the accuracy, the true positive rate, the positive predictive value, Jaccard's coefficient, and all F-beta scores. Furthermore, we study the properties of the underlying ranking scores, such as the influence of the priors or the correspondences with the ROC space, and depict how to characterize any other score by comparing them to the Tile. Overall, we demonstrate that the Tile is a powerful tool that effectively captures all the rankings in a single visualization and allows interpreting them.

Figures

Figures reproduced from arXiv: 2412.04309 by the authors.

Figure 1
Figure 1. Introducing the Tile. We introduce a new visual tool, called the Tile, representing an infinite family of ranking scores to evaluate the performances of two-class classifiers at a glance. In this figure, we highlight the correspondences between specific ranking scores on the Tile and their corresponding set of iso￾performance lines in the ROC space. Notably, the variation of iso-performance lines along the right bor… view at source ↗
Figure 2
Figure 2. The geometry of the ranking scores RIa,b in the ROC plane (F P R, T P R). Example given for the class priors π+ = 1 − π− = 0.2 and the importance given by (a, b) = (0.95, 0.7) [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Placement of the canonical ranking scores (left) and of [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Tiles showing the rank correlations (Kendall [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Toy examples showing on the Tile which of the [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: In the Tile, the ranking scores that put all no-skill [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 8
Figure 8. Figure 8: Visualization of how the Tile deforms with a shift for the distribution [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Tiles showing the values of canonical ranking scores achievable by no-skill classifiers when the prior of the positive class is [PITH_FULL_IMAGE:figures/full_fig_p027_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Event Detection in Videos: A Framework for the Development of New Methods

    cs.CV 2026-07 conditional novelty 5.5 of 10

    A framework of tagged multi-environment datasets (including new FSD and SUC), probabilistic Tile-based ranking, and explicit application scenarios for fair video event detection.

  2. A Hitchhiker's Guide to Understanding Performances of Two-Class Classifiers

    cs.CV 2024-12 conditional novelty 5.0 of 10

    The paper defines six Tile-based visualization flavors (correlation, value, baseline, state-of-the-art, ranking, entity) and a four-profile guide for comparing binary classifiers, illustrated on 74 semantic segmentati...

Reference graph

Works this paper leans on

48 extracted references · 46 canonical work pages · cited by 2 Pith papers

  1. [28]

    Foundations of the theory of performance-based ranking

    Sébastien Piérard, Anaïs Halin, Anthony Cioppa, Adrien Deliège, and Marc Van Droogenbroeck. Foundations of the theory of performance-based ranking. arXiv, abs/2412.04227, 2024. 1, 2, 4, 11, 12, 13, 23

    work page arXiv 2024

  2. [1]

    Altman and Martin Bland

    Douglas G. Altman and Martin Bland. Diagnostic tests 2: Predictive values. Br. Medical J., 309(6947):102–102, 1994. 3

    work page 1994

  3. [2]

    Multivariate comparison of classification performance measures

    Davide Ballabio, Francesca Grisoni, and Roberto Todes- chini. Multivariate comparison of classification performance measures. Chemom. Intell. Lab. Syst., 174:33–44, 2018. 2

    work page 2018

  4. [3]

    Strategies for learning in class imbalance problems

    Ricardo Barandela, Josep Salvador Sánchez, Vicente García, and Erick Rangel. Strategies for learning in class imbalance problems. Pattern Recognit., 36(3):849–851, 2003. 3

    work page 2003

  5. [4]

    Batyrshin, Nailya Kubysheva, Valery Solovyev, and Luis A

    Ildar Z. Batyrshin, Nailya Kubysheva, Valery Solovyev, and Luis A. Villa-Vargas. Visualization of similarity measures for binary data and 2x2 tables. Comput. Y Sist., 20(3):345– 353, 2016. 3, 6

    work page 2016

  6. [5]

    Forrest B. Baulieu. A classification of presence/absence based dissimilarity coefficients. J. Classif., 6(1):233–246,

    work page

  7. [6]

    Brown and Herbert T

    Christopher D. Brown and Herbert T. Davis. Receiver oper- ating characteristics curves and related decision measures: A tutorial. Chemom. Intell. Lab. Syst., 80(1):24–38, 2006. 3

    work page 2006

  8. [7]

    Dennis Cradit, and Douglas Steinley

    Michael Brusco, J. Dennis Cradit, and Douglas Steinley. A comparison of 71 binary similarity coefficients: The effect of base rates. PLOS ONE, 16(4):e0247751, 2021. 3

    work page 2021

Show all 48 references
  1. [8]

    Evaluation of gene struc- ture prediction programs

    Moisès Burset and Roderic Guigó. Evaluation of gene struc- ture prediction programs. Genomics, 34(3):353–367, 1996. 3, 14

    1996

  2. [9]

    Ted Byrt, Janet Bishop, and John B. Carlin. Bias, prevalence and kappa. J. Clin. Epidemiology, 46(5):423–429, 1993. 3, 14

    1993

  3. [10]

    Binary classification performance mea- sures/metrics: A comprehensive visualized roadmap to gain new insights

    Gurol Canbek, Seref Sagiroglu, Tugba Taskaya Temizel, and Nazife Baykal. Binary classification performance mea- sures/metrics: A comprehensive visualized roadmap to gain new insights. In Int. Conf. Comput. Sci. Eng. (UBMK), pages 821–826, Antalya, Turkey, 2017. 2, 3

    2017

  4. [11]

    and Tap- pert

    Seung-Seok Choi, Sung-Hyuk Cha, and Charles C. and Tap- pert. A survey of binary similarity and distance measures. J. Syst. Cybern. Informatics, 8(1):43–48, 2010. 2, 3

    2010

  5. [12]

    A coefficient of agreement for nominal scales

    Jacob Cohen. A coefficient of agreement for nominal scales. Educ. Psychol. Meas., 20(1):37–46, 1960. 3, 8

    1960

  6. [13]

    An experimental comparison of performance measures for classification

    Cèsar Ferri, José Hernández-Orallo, and Ramona Modroiu. An experimental comparison of performance measures for classification. Pattern Recognit. Lett., 30(1):27–38, 2009. 2, 3

    2009

  7. [14]

    The geometry of ROC space: Understanding machine learning metrics through ROC isometrics

    Peter Flach. The geometry of ROC space: Understanding machine learning metrics through ROC isometrics. In Int. Conf. Mach. Learn. (ICML) , pages 194–201, Washington, DC, USA, 2003. 3, 5, 6, 14

    2003

  8. [15]

    Joseph L. Fleiss. Measuring nominal scale agreement among many raters. Psychol. Bull., 76(5):378–382, 1971. 8

    1971

  9. [16]

    Esti- mation of the Youden index and its associated cutoff point

    Ronen Fluss, David Faraggi, and Benjamin Reiser. Esti- mation of the Youden index and its associated cutoff point. Biom. J., 47(4):458–472, 2005. 3

    2005

  10. [17]

    Gardner and Matthias Greiner

    Ian A. Gardner and Matthias Greiner. Receiver-operating characteristic curves and likelihood ratios: improvements over traditional methods for the evaluation and application of veterinary clinical pathology tests. Veterinary Clin. Pathol., 35(1):8–17, 2006. 3

    2006

  11. [18]

    Glas, Jeroen Lijmer, Martin H

    Afina S. Glas, Jeroen Lijmer, Martin H. Prins, Gouke Bonsel, and Patrick M. M. Bossuyt. The diagnostic odds ratio: a single indicator of test performance. J. Clin. Epidemiology, 56(11):1129–1135, 2003. 3

    2003

  12. [19]

    Gower and Pierre Legendre

    John C. Gower and Pierre Legendre. Metric and euclidean properties of dissimilarity coefficients. J. Classif., 3(1):5–48,

  13. [20]

    On the class imbalance problem

    Xinjian Guo, Yilong Yin, Cailing Dong, Gongping Yang, and Guangtong Zhou. On the class imbalance problem. In Int. Conf. Nat. Comput., pages 192–201, Jinan, China, 2008. 3

    2008

  14. [21]

    A hitchhiker’s guide to un- derstanding performances of two-class classifiers

    Anaïs Halin, Sébastien Piérard, Anthony Cioppa, and Marc Van Droogenbroeck. A hitchhiker’s guide to un- derstanding performances of two-class classifiers. arXiv, abs/2412.04377, 2024. 1

    2024 arXiv

  15. [22]

    Thomas F. Heston. Standardizing predictive values in diag- nostic imaging research. J. Magn. Reson. Imaging , 33(2): 505–505, 2011. 3, 14

    2011

  16. [23]

    eq- uitable threat score

    Robin J. Hogan, Christopher A. T. Ferro, Ian T. Jolliffe, and David B. Stephenson. Equitability revisited: Why the “eq- uitable threat score” is not equitable. Weather. Forecast., 25 (2):710–726, 2010. 3, 14

    2010

  17. [24]

    The AUK: A simple alternative to the AUC

    Uzay Kaymak, Arie Ben-David, and Rob Potharst. The AUK: A simple alternative to the AUC. Eng. Appl. Artif. Intell., 25(5):1082–1089, 2012. 3

    2012

  18. [25]

    Bradley, Aaron Carass, Car- olin Feldmann, Alejandro F

    Lena Maier-Hein, Matthias Eisenmann, Annika Reinke, Sinan Onogur, Marko Stankovic, Patrick Scholz, Tal Arbel, Hrvoje Bogunovic, Andrew P. Bradley, Aaron Carass, Car- olin Feldmann, Alejandro F. Frangi, Peter M. Full, Bram van Ginneken, Allan Hanbury, Katrin Honauer, Michal Koz...

    2018

  19. [26]

    Towards a classifi- cation of binary similarity measures

    Ivan Ramirez Mejia and Ildar Batyrshin. Towards a classifi- cation of binary similarity measures. In Adv. Soft Comput., pages 325–335. Springer Int. Publ., 2018. 3

    2018

  20. [27]

    How trustwor- thy are performance evaluations for basic vision tasks?IEEE Trans

    Tran Thien Dat Nguyen, Hamid Rezatofighi, Ba-Ngu V o, Ba-Tuong V o, Silvio Savarese, and Ian Reid. How trustwor- thy are performance evaluations for basic vision tasks?IEEE Trans. Pattern Anal. Mach. Intell. , 45(7):8538–8552, 2023. 7

    2023

  21. [29]

    Evaluation: from precision, re- call and F-measure to ROC, informedness, markedness & correlation

    David Martin Ward Powers. Evaluation: from precision, re- call and F-measure to ROC, informedness, markedness & correlation. J. Mach. Learn. Technol., 2(1):37–63, 2011. 2, 3

    2011

  22. [30]

    David M. W. Powers. Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correla- tion. arXiv, abs/2010.16061, 2020. 3

    2010 arXiv

  23. [31]

    Robust classification for imprecise environments

    Foster Provost and Tom Fawcett. Robust classification for imprecise environments. Mach. Learn., 42(3):203–231,

  24. [32]

    William A. Scott. Reliability of content analysis: The case of nominal scale coding. Public Opin. Q. , 19(3):321–325,

  25. [33]

    The hitchhiker’s guide to prior-shift adaptation

    Tomas Sipka, Milan Sulc, and Jiri Matas. The hitchhiker’s guide to prior-shift adaptation. In IEEE Winter Conf. Appl. Comput. Vis. (WACV) , pages 2031–2039, Waikoloa, HI, USA, 2022. 4, 6, 13

    2022

  26. [34]

    Extending F1 metric, probabilistic approach

    Mikołaj Sitarz. Extending F1 metric, probabilistic approach. Adv. Artif. Intell. Mach. Learn., 3(2):1025–1038, 2023. 3, 14

    2023

  27. [35]

    A systematic analysis of performance measures for classification tasks

    Marina Sokolova and Guy Lapalme. A systematic analysis of performance measures for classification tasks. Inf. Process. & Manag., 45(4):427–437, 2009. 2

    2009

  28. [36]

    Putnam P. Texel. Measure, metric, and indicator: An object- oriented approach for consistent terminology. Proc. IEEE Southeastcon, pages 1–5, 2013. 2, 3

    2013

  29. [37]

    Matthijs J. Warrens. The effect of combining categories on Bennett, Alpert and Goldstein’s S. Stat. Methodol. , 9(3): 341–352, 2012. 3

    2012

  30. [38]

    Matthijs J. Warrens. A comparison of multi-way similarity coefficients for binary sequences. Int. J. Res. Rev. Appl. Sci., 16(1):64–75, 2013. 3

    2013

  31. [39]

    Daniel S. Wilks. Statistical methods in the atmospheric sci- ences. Elsevier, fourth edition, 2020. 3, 14

    2020

  32. [40]

    A per- son and context specific approach for skin color classifica- tion

    Matthias Wimmer, Bernd Radig, and Michael Beetz. A per- son and context specific approach for skin color classifica- tion. In IEEE Int. Conf. Pattern Recognit. (ICPR), pages 39– 42, Hong Kong, China, 2006. 3

    2006

  33. [41]

    fundamental bridge

    William John Youden. Index for rating diagnostic tests. Can- cer, 3(1):32–35, 1950. 3, 14 A. Supplementary Material Contents A.1 . List of symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 A.1.1 Mathematical Symbol...

    1950

  34. [42]

    When π+ = 0, which implies that F N= 0 ∧ T P= 0, we have dom(κ) ∩ P∗ = P ∈ P(Ω,Σ) : π−τ− + π+τ+ ̸= 1 ∩ P∗ = P ∈ P(Ω,Σ) : τ− ̸= 1 ∩ P∗ = P ∈ P(Ω,Σ) : F N+ T N̸= 1 ∩ P∗ = P ∈ P(Ω,Σ) : T N̸= 1 ∩ P∗ = P ∈ P(Ω,Σ) : EP [I] ̸= 0 ∩ P∗ = dom(RI ) ∩ P∗

  35. [43]

    When π+ ∈ ]0, 1[, we have dom(κ) ∩ P∗ = P ∈ P(Ω,Σ) : π−τ− + π+τ+ ̸= 1 ∩ P∗ = P ∈ P(Ω,Σ) ∩ P∗ = dom(RI ) ∩ P∗

  36. [44]

    The restricted domains are thus equal in all cases: dom(κ) ∩ P∗ = dom(RI ) ∩ P∗

    When π+ = 1, which implies that T N= 0 ∧ F P= 0, we have dom(κ) ∩ P∗ = P ∈ P(Ω,Σ) : π−τ− + π+τ+ ̸= 1 ∩ P∗ = P ∈ P(Ω,Σ) : τ+ ̸= 1 ∩ P∗ = P ∈ P(Ω,Σ) : F P+ T P̸= 1 ∩ P∗ = P ∈ P(Ω,Σ) : T P̸= 1 ∩ P∗ = P ∈ P(Ω,Σ) : EP [I] ̸= 0 ∩ P∗ = dom(RI ) ∩ P∗ . The restricted domains are thus ...

  37. [45]

    When P ({tp}) = P ({tn}) and P ({f n}) = P ({f p}): V U T(P ) = RIa,b (P ) ∀(a, b) ∈ [0, 1] (10) = A(P ) = T N R(P ) = T P R(P ) (11) = N P V(P ) = P P V(P ) = Fβ(P ) = . . . (12)

  38. [46]

    When P ({tp}) = P ({tn}) and P ({f n}) ̸= P ({f p}): V U T(P ) = P ({tn}) P ({f n}) − P ({f p}) (ln (P ({tn, f n})) − ln (P ({tn, f p})))) (13)

  39. [47]

    When P ({tp}) ̸= P ({tn}) and P ({f n}) = P ({f p}): V U T(P ) = 1 − P ({f n}) P ({tp}) − P ({tn}) (ln (P ({tp, f n})) − ln (P ({tn, f n})))) (14)

  40. [48]

    When P ({tp}) ̸= P ({tn}) and P ({f n}) ̸= P ({f p}): V U T(P ) = 1 2 − 1 2 P ({tn})2 − P ({f n})2 ln (P ({tn, f n})) + P ({tp})2 − P ({f p})2 ln (P ({tp, f p})) + P ({f p})2 − P ({tn})2 ln (P ({f p, tn})) + P ({f n})2 − P ({tp})2 ln (P ({f n, tp})) (P ({tp}) − P ({tn}))(P ({f...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.