Pith. sign in

REVIEW 2 major objections 5 minor 2 cited by

A Hitchhiker's Guide to Understanding Performances of Two-Class Classifiers

T0 review · 2 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that a single 2D map, the Tile, can show how a two-class classifier performs under every possible weighting of its four outcomes, replacing single-score comparisons with a complete picture of rankings.

desk verdict A genuinely useful guide to the Tile for two-class classifier evaluation, but the abstract overclaims universality for a two-parameter score family that does not contain balanced accuracy or mIoU. read the letter →

arxiv 2412.04377 v3 pith:32IHLLK2 submitted 2024-12-05 cs.CV cs.LGcs.PF

Anaïs Halin , Sébastien Piérard , Anthony Cioppa , Marc Van Droogenbroeck This is my paper · ORCID
classification cs.CVcs.LGcs.PF MSC 62H3068T10
keywords Tilevisualizationtwo-classclassificationrankingscoresclassifierevaluationsemanticsegmentationimportanceweightingbenchmarkingperformance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is a guide to a recently introduced visualization called the Tile, which arranges the infinite family of ranking scores for two-class classifiers on a unit square. Its central claim is that the Tile, read through six interpretative flavors, captures a classifier's behavior under every application-specific importance weighting in a single image. The guide organizes four user profiles—theoretical analyst, method designer, benchmarker, and application developer—and tells each which flavor to use and how to interpret it. It illustrates the approach by ranking 74 semantic segmentation models, showing that different models win in different regions of the map and that a single score is not enough to declare a winner.

What carries the argument

The central object is the Tile, the mapping $(a,b) \mapsto R_I$ defined on $[0,1]^2$, where $I(tn)=1-a$, $I(fp)=1-b$, $I(fn)=b$, $I(tp)=a$, and $R_I$ is the weighted ratio of correct outcomes to all four outcomes in Eq. (3). Each point of the Tile is one ranking score: the corners recover TPR, TNR, PPV, and NPV, the center is accuracy, and the right edge holds the $F_\beta$ scores. The argument is carried by the fact that every score in this family is a function of the same four outcome probabilities, so value, rank, correlation, and entity identity can all be drawn on the same square without discarding information.

What would settle it

Take a fixed set of classifiers and an application whose loss is a nonlinear function of the four outcome rates (for example, cost = TPR squared plus a fixed penalty per false alarm). If the application's ranking of those classifiers cannot be reproduced by $R_I$ at any point $(a,b)$ of the Tile, the claim that the Tile accommodates all application-specific preferences is refuted.

Watch

Extended reading notes

Core claim

For any two-class classifier, the paper places every importance-weighted ranking score $R_I$ at a point $(a,b)$ of the unit square, with $a$ controlling the importance of true positives relative to true negatives and $b$ controlling false negatives relative to false positives. The discovery is that this map can be read in six ways: as score value (Value Tile), as rank among a set of entities (Ranking Tile), as the identity of the entity at a fixed rank (Entity Tile), as correlation with a reference score (Correlation Tile), as the baseline envelope (Baseline Value Tile), and as the state-of-the-art envelope (State-of-the-Art Value Tile). On the 74-model illustration, the first rank belongs to four different models in different regions of the Tile, and the guide shows how an application developer can choose among them with known importance, with community practice encoded in a reference score, or with a minimax-rank rule when importance is unknown. Together these flavors make the Tile a single visualization that condenses the infinite family of scores into one picture.

Load-bearing premise

The central premise is that any practical way of scoring a two-class classifier can be expressed as a weighted ratio of the four possible prediction outcomes (true positive, true negative, false positive, false negative), with the two weights spanning the entire unit square.

Editorial extensions

If this is right

  • Benchmarking with a single score becomes a choice of a point on the Tile; a leaderboard that reports only one metric silently fixes an importance weighting.
  • For the 74 models analyzed, no model ranks first everywhere: the Entity Tile shows four different winners, so a fair challenge should report the winner as a region of the Tile rather than a single name.
  • When importance values are unknown, the minimax-rank rule (minimize the worst rank, then the average rank) gives a well-defined selection; in the illustration this chooses a model that never drops below rank 14 anywhere on the Tile.
  • The Correlation Tile offers a formal way to test whether a proposed new score carries information beyond a field-standard reference: compute its correlation with $R_I$ across the Tile and locate the low-correlation region.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that choosing any scalar score is equivalent to choosing a point $(a,b)$ on the Tile; a natural next step is to define dominance regions where one entity beats another at every point, making 'which model is best' a set-valued answer.
  • The near-horizontal bands the paper observes in its illustration follow from an almost constant ratio $P({tp})/P({tn})$ across the 74 models; a testable prediction is that model families with heterogeneous priors will show fragmented tile regions, revealing models with genuinely different error trade-offs.
  • The discussion of multi-class extension suggests a concrete experiment: construct micro- and macro-averaged two-class Tiles for the same multi-class problem and compare their Entity Tiles with the full multi-class ranking, quantifying the information loss the paper warns about.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This paper presents a didactic guide to comparing binary classifiers using the Tile, a two-parameter family of ranking scores R_I defined in Eq. (3). It defines four user scenarios (theoretical analyst, method designer, benchmarker, application developer) and six flavors of the Tile (Correlation, Value, Baseline Value, State-of-the-Art Value, Ranking, and Entity), and illustrates them on 74 semantic segmentation models evaluated on BDD100K after collapsing labels into two classes. The paper claims that the Tile displays all possible application-specific preferences and effectively captures the behavior of classifiers in a single visualization.

Significance. If the paper is taken as a tutorial rather than a new theoretical contribution, it is a useful and well-executed guide: the definitions of the flavors are explicit, Eq. (3) correctly recovers TPR, TNR, PPV, NPV, accuracy, and F1 at the appropriate corners, and the supplementary material provides full numerical tables and code, making the 74-model illustration reproducible. The main value is pedagogical and organizational for practitioners who work within the R_I family. However, the advertised completeness ('all possible application-specific preferences') is not established, and the empirical illustration is an application of the authors' own framework rather than an external validation; these issues are fixable by restricting the claims.

major comments (2)
  1. [Abstract; Section 3.1, Eq. (3); Section 3.2, Scenario 4] The abstract's claim that the Tile displays 'all possible application-specific preferences' is unsupported by the definition of the family R_I. Eq. (3) restricts scores to ratios of one linear form in P({tp}), P({tn}) over a second linear form in all four events; scores such as balanced accuracy, 0.5(TPR+TNR), are sums of such ratios and are not members of the family, so their induced orderings need not be reproducible by any single point (a,b). The paper itself concedes this in Scenario 4 when it states that mIoU 'is not part of the Tile' and uses the Correlation Tile instead. The authors should either prove that every application-relevant preference (or every ordering) lies in the R_I family, or explicitly restrict the claims to scores of the form Eq. (3).
  2. [Abstract; Section 3.2, Scenario 4; Appendix A.4] The paper claims to 'demonstrate that the Tile effectively captures the behavior of classifiers' and to provide a full ranking of 74 state-of-the-art models, but the illustration cannot validate the framework: every flavor is constructed from the same R_I family it is meant to summarize, and no external benchmark, independent cost structure, or falsifiable prediction is used. Appendix A.4 further states that the results are specific to arbitrary choices and that changing the posterior threshold from pi+ to 0.5 'leads to a significantly different two-class classification problem, for which the ranking of the models is different.' At minimum, the claims about capturing behavior and ranking 74 models should be qualified as an illustration of the framework under explicitly stated conventions, not as a demonstration of the framework's validity.
minor comments (5)
  1. [Section 5 (Acknowledgments)] The heading 'Acknowledments' should be spelled 'Acknowledgments'.
  2. [Figure 4 and Section 3.1] The figure caption uses 'macro-IoU' while the text uses 'mean intersection over union (mIoU)'; please unify the terminology.
  3. [Figures 5-8] The hatched areas are explained in the text but never defined in the captions; adding a legend or a short caption sentence would improve readability.
  4. [Appendix A.2] The grid size parameter used to discretize the Tile is not reported; since all figures depend on this discretization, please state the default value and, ideally, a brief sensitivity check.
  5. [Definition 1] The codomain X(Omega,Sigma) is used in Definition 1 but defined only later in the supplementary symbol list; please define it before first use or rephrase the definition.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the Tile flavors are explicit, data-driven mappings, and the paper's only substantive weakness is an overstated universality claim, not a circular derivation.

full rationale

The paper is a guide that applies a previously introduced framework (Piérard et al. [37] and [36]) rather than deriving a new prediction from fitted parameters. Every Tile flavor is an explicit mapping of data already contained in the confusion matrix: the Value Tile (Definition 3) is R_I(P_epsilon) as given by Eq. (3); the Ranking Tile (Definition 6) is the rank induced by the same R_I; the Entity Tile (Definition 7) is the entity at that rank; and the Correlation Tile (Definition 2) is a correlation coefficient between an external score and R_I. These are definitions, not fitted quantities, and no parameter is tuned to a subset of data and then presented as a prediction on a related quantity. The 74-model illustration uses confusion-matrix values listed in Appendix A.7, obtained externally from MMSegmentation models, and the tiles are deterministic functions of those values. The self-citations to [37] and [36] are present and central, but they are not circular in the sense prohibited here: the paper does not invoke a uniqueness theorem to exclude alternatives, and the cited framework is itself derived in those works rather than being defined by the present paper's conclusions. The abstract's phrase 'displaying all possible application-specific preferences' overstates the scope, because the R_I family of Eq. (3) is a two-parameter subfamily and does not include scores such as mIoU; the paper itself concedes this in Scenario 4 ('Although this score is not part of the Tile, the Correlation Tile provides a direct comparison'). This is an unsupported universality claim and a correctness/scope risk, but it is not a case of a claimed result being equivalent to its inputs by construction. Thus no significant circularity is found.

Assumptions & free parameters 2 free parameters · 3 assumptions · 1 invented entities

The paper's central claim depends on the axiomatic ranking theory of [37] and the Tile construction of [36], both by the same authors. The flavors introduced here are definitions, not derived predictions. The main experimental assumptions, the binary label grouping and the posterior threshold, are arbitrary choices disclosed in the appendix.

free parameters (2)
  • positive-class assignment threshold = positive prior pi+ ≈ 0.1242
    Used in Appendix A.4 to assign each semantic label in the four learning sets to the positive or negative class; this choice determines which pixels count as foreground and therefore affects all reported rankings.
  • posterior threshold for predictions = positive prior pi+ ≈ 0.1242
    The paper thresholds model posteriors at pi+ rather than at 0.5; the authors note that using 0.5 leads to a significantly different classification problem and different model rankings.
assumptions (3)
  • domain assumption Performance-based rankings satisfy the axioms of Piérard et al. [37], notably stability when entities are added or removed.
    The Tile and all flavors inherit the axiomatic ranking theory from the authors' prior paper; no proof is reproduced here.
  • domain assumption Every ranking score of interest for two-class classification is of the form R_I in Eq. (3) for some importance measure I.
    This universality claim makes the Tile a complete representation of 'all possible application-specific preferences'; it is a strong premise cited from [37] and not defended here.
  • ad hoc to paper Grouping multi-class segmentation outputs into two classes preserves the comparison relevant to the application.
    The 74-model illustration collapses 19 BDD100K labels into background and foreground based on a proportion threshold; the resulting two-class problems from different training sets are treated as directly comparable.
invented entities (1)
  • Tile flavor mappings (Correlation, Value, Baseline, State-of-the-Art, Ranking, Entity)
    purpose: To display different information on the Tile: correlations with a reference score, score values per entity, extrema across entities, ranks, and the identity of entities at a given rank.
    These are new definitions in this paper. They are visualization functions rather than physical entities, and they make no falsifiable predictions beyond organizing data already present in the confusion matrix.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Hitchhiker's Guide to Understanding Performances of Two-Class Classifiers." pith.science (2026). https://pith.science/paper/32IHLLK2

@misc{pith2026241204377,
  author       = {Pith},
  title        = {Pith review of: A Hitchhiker's Guide to Understanding Performances of Two-Class Classifiers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/32IHLLK2}},
  note         = {Machine review of arXiv:2412.04377}
}
read the original abstract

Properly understanding the performances of classifiers is essential in various scenarios. However, the literature often relies only on one or two standard scores to compare classifiers, which fails to capture the nuances of application-specific requirements. The Tile is a recently introduced visualization tool organizing an infinity of ranking scores into a 2D map. Thanks to the Tile, it is now possible to compare classifiers efficiently, displaying all possible application-specific preferences instead of having to rely on a pair of scores. This hitchhiker's guide to understanding the performances of two-class classifiers presents four scenarios showcasing different user profiles: a theoretical analyst, a method designer, a benchmarker, and an application developer. We introduce several interpretative flavors adapted to the user's needs by mapping different values on the Tile. We illustrate this guide by ranking and analyzing the performances of 74 state-of-the-art semantic segmentation models through the perspective of the four scenarios. Through these user profiles, we demonstrate that the Tile effectively captures the behavior of classifiers in a single visualization, while accommodating an infinite number of ranking scores. Code for mapping the different Tile flavors is available in supplementary material.

Figures

Figures reproduced from arXiv: 2412.04377 by the authors.

Figure 1
Figure 1. Our hitchhiker’s guide. This hitchhiker’s guide to un￾derstanding performances of two-class classifiers addresses four scenarios, answering specific requests from four user profiles: (1) the theoretical analyst, who is interested in understanding the theoretical relationship between different scores typically used for evaluating or ranking methods, (2) the method designer, who would like to analyze the performances … view at source ↗
Figure 3
Figure 3. Tile with ranking scores for two-class classification. Each point of the Tile corresponds to a ranking score RI which can be computed using Eq. (3). The Tile therefore contains an infinity of ranking scores, including the popular Accuracy (A), Negative Predictive Value (NP V ), True Positive Rate (T P R), Positive Pre￾dictive Value (P P V ), True Negative Rate (T NR), and F1 scores. The location of a ranking score R… view at source ↗
Figure 4
Figure 4. illustrates this correlation for the mean intersec￾tion over union (mIoU), a score commonly used to bench￾mark semantic segmentation models. A strong correlation between the mIoU and the Tile scores is observed in a hor￾izontal band near the top. Hence, selecting a score in this band will lead to similar conclusions than the ones provided by the mIoU. However, selecting a score outside this band will most probably a… view at source ↗
Figures from the paper (7 more)
Figure 6
Figure 6. Figure 6: Baseline Value Tile (left) and State-of-the-Art Value Tile (right). These two tiles describe the current benchmark in semantic segmentation. The Baseline Value Tile and the State-of￾the-Art Value Tile provide the infimum and the supremum, respec￾tively, of the canonica…
Figure 5
Figure 5. Figure 5: Value Tile for the 4 entities ranked first (see [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 7
Figure 7. Figure 7: shows the Ranking Tile for the 4 entities ranked first among the 74 state-of-the-art semantic segmentation models (see [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Entity Tile showing the entities that are ranked at the first, second, third, and last positions. Hatched areas in the “who’s first?” and “who’s last?” tiles highlight regions where no￾skilled performances surpass that of the entities in the current map. IoU. The appli…
Figure 9
Figure 9. Figure 9: No-Skill Tile (left) and Relative-Skill Tile (right). These tiles show, on the left, the score values of the best no-skilled performances and, on the right, the relative performance of the state of the art compared to no-skilled performances for the 74 se￾mantic segmen…
Figure 10
Figure 10. Figure 10: Ranking Tiles showing the rank of PSPNet trained on different learning datasets (from left to right: ♠ Cityscapes, ♥ ADE20K, ♦ Pascal VOC 2012, and ♣ COCO-Stuff 164k). We observe that the learning dataset strongly influences the rank￾ing behavior. Defining the two cla…
Figure 11
Figure 11. Figure 11: Behavior of 6 scores for 3 sets of performances. The Correlation Tile shows the estimated rank correlations (with the Spearman’s ρ) between 6 scores and all canonical ranking scores, for a uniform distribution over all possible performances (left), over all performanc…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The Tile: A 2D Map of Ranking Scores for Two-Class Classification

    cs.CV 2024-12 conditional novelty 6.0 of 10

    The Tile maps every canonical ranking score for binary classifiers to a point in the unit square, placing common metrics such as accuracy, F-beta, sensitivity, and precision at geometrically meaningful locations.

  2. Event Detection in Videos: A Framework for the Development of New Methods

    cs.CV 2026-07 conditional novelty 5.5 of 10

    A framework of tagged multi-environment datasets (including new FSD and SUC), probabilistic Tile-based ranking, and explicit application scenarios for fair video event detection.

Reference graph

Works this paper leans on

68 extracted references · 63 canonical work pages · cited by 2 Pith papers

  1. [1]

    Kempton, Petrus C

    Azim Ahmadzadeh, Dustin J. Kempton, Petrus C. Martens, and Rafal A. Angryk. Contingency space: A semimetric space for classification evaluation.IEEE Trans. Pattern Anal. Mach. Intell., 45(2):1501–1513, 2023. 2

    work page 2023

  2. [2]

    The area above the ordinal dominance graph and the area below the receiver operating character- istic graph

    Donald Bamber. The area above the ordinal dominance graph and the area below the receiver operating character- istic graph. J. Math. Psychol., 12(4):387–415, 1975. 2

    work page 1975

  3. [3]

    Unachievable region in precision-recall space and its effect on empirical evaluation

    Kendrick Boyd, Victor Costa, Jesse Davis, and David Page. Unachievable region in precision-recall space and its effect on empirical evaluation. In Int. Conf. Mach. Learn. (ICML), pages 639–646, Edinburgh, UK, 2012. 2

    work page 2012

  4. [4]

    Briggs and Russell Zaretzki

    William M. Briggs and Russell Zaretzki. The skill plot: A graphical technique for evaluating continuous diagnostic tests. Biometrics, 64(1):250–256, 2008. 2

    work page 2008

  5. [5]

    Ted Byrt, Janet Bishop, and John B. Carlin. Bias, prevalence and kappa. J. Clin. Epidemiology, 46(5):423–429, 1993. 6, 8

    work page 1993

  6. [6]

    COCO- stuff: Thing and stuff classes in context

    Holger Caesar, Jasper Uijlings, and Vittorio Ferrari. COCO- stuff: Thing and stuff classes in context. In IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , pages 1209–1218, Salt Lake City, UT, USA, 2018. 4, 5, 13

    work page 2018

  7. [7]

    GCNet: Non-local networks meet squeeze-excitation networks and beyond

    Yue Cao, Jiarui Xu, Stephen Lin, Fangyun Wei, and Han Hu. GCNet: Non-local networks meet squeeze-excitation networks and beyond. In IEEE/CVF Int. Conf. Comput. Vis. Work. (ICCV Work.), pages 1971–1980, Seoul, South Korea,

    work page 1971

  8. [8]

    Carrington, Douglas G

    Andre M. Carrington, Douglas G. Manuel, Paul W. Fieguth, Tim Ramsay, Venet Osmani, Bernhard Wernly, Carol Ben- nett, Steven Hawken, Olivia Magwood, Yusuf Sheikh, Matthew McInnes, and Andreas Holzinger. Deep ROC anal- ysis and AUC as balanced average accuracy, for improved classifier selection, audit and explanation. IEEE Trans. Pat- tern Anal. Mach. Intel...

    work page 2023

Show all 68 references
  1. [9]

    Rethinking atrous convolution for semantic image segmentation

    Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam. Rethinking atrous convolution for semantic image segmentation. arXiv, abs/1706.05587, 2017. 5

    2017 arXiv

  2. [10]

    Encoder-decoder with atrous separable convolution for semantic image segmentation

    Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Eur. Conf. Comput. Vis. (ECCV) , pages 833–851, Munich, Germany, 2018. 5

    2018

  3. [11]

    Schwing, and Alexander Kir- illov

    Bowen Cheng, Alexander G. Schwing, and Alexander Kir- illov. Per-pixel classification is not all you need for semantic segmentation. In Adv. Neural Inf. Process. Syst. (NeurIPS), pages 17864–17875. Curran Assoc. Inc., 2021. 5

    2021

  4. [12]

    Schwing, Alexan- der Kirillov, and Rohit Girdhar

    Bowen Cheng, Ishan Misra, Alexander G. Schwing, Alexan- der Kirillov, and Rohit Girdhar. Masked-attention mask transformer for universal image segmentation. In IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), pages 1280– 1289, New Orleans, LA, USA, 2022. 5

    2022

  5. [13]

    MMSegmentation: OpenMMLab semantic segmentation toolbox and benchmark

    MMSegmentation Contributors. MMSegmentation: OpenMMLab semantic segmentation toolbox and benchmark. https : / / github . com / open - mmlab/mmsegmentation, 2020. 4, 5

    2020

  6. [14]

    The cityscapes dataset for semantic urban scene understanding

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pages 3213– 3223, La...

    2016

  7. [15]

    The relationship between precision-recall and ROC curves

    Jesse Davis and Mark Goadrich. The relationship between precision-recall and ROC curves. In Int. Conf. Mach. Learn. (ICML), pages 233–240, Pittsburgh, Pennsylvania, 2006. 2

    2006

  8. [16]

    Mark Everingham, Luc Van Gool, Christopher K. I. Williams, John Winn, and Andrew Zisserman. The PASCAL visual object classes (VOC) challenge. Int. J. Comput. Vis., 88(2):303–338, 2010. 4, 5, 13

    2010

  9. [17]

    Rethinking BiSeNet for real-time semantic segmentation

    Mingyuan Fan, Shenqi Lai, Junshi Huang, Xiaoming Wei, Zhenhua Chai, Junfeng Luo, and Xiaolin Wei. Rethinking BiSeNet for real-time semantic segmentation. In IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), pages 9711– 9720, Nashville, TN, USA, 2021. 5

    2021

  10. [18]

    ROC graphs: Notes and practical considera- tions for researchers

    Tom Fawcett. ROC graphs: Notes and practical considera- tions for researchers. Mach. Learn., 31:1–38, 2004. 2

    2004

  11. [19]

    An introduction to ROC analysis

    Tom Fawcett. An introduction to ROC analysis. Pattern Recognit. Lett., 27(8):861–874, 2006. 2

    2006

  12. [20]

    The geometry of ROC space: Understanding machine learning metrics through ROC isometrics

    Peter Flach. The geometry of ROC space: Understanding machine learning metrics through ROC isometrics. In Int. Conf. Mach. Learn. (ICML) , pages 194–201, Washington, DC, USA, 2003. 2

    2003

  13. [21]

    Dual attention network for scene segmentation

    Jun Fu, Jing Liu, Haijie Tian, Yong Li, Yongjun Bao, Zhi- wei Fang, and Hanqing Lu. Dual attention network for scene segmentation. In IEEE/CVF Conf. Comput. Vis. Pat- tern Recognit. (CVPR), pages 3141–3149, Long Beach, CA, USA, 2019. 5

    2019

  14. [22]

    Dynamic multi- scale filters for semantic segmentation

    Junjun He, Zhongying Deng, and Yu Qiao. Dynamic multi- scale filters for semantic segmentation. In IEEE/CVF Int. Conf. Comput. Vis. (ICCV), pages 3561–3571, Seoul, South Korea, 2019. 5

    2019

  15. [23]

    Adaptive pyramid context network for seman- tic segmentation

    Junjun He, Zhongying Deng, Lei Zhou, Yali Wang, and Yu Qiao. Adaptive pyramid context network for seman- tic segmentation. In IEEE/CVF Conf. Comput. Vis. Pat- tern Recognit. (CVPR), pages 7511–7520, Long Beach, CA, USA, 2019. 5

    2019

  16. [24]

    Interlaced sparse self-attention for semantic segmentation

    Lang Huang, Yuhui Yuan, Jianyuan Guo, Chao Zhang, Xilin Chen, and Jingdong Wang. Interlaced sparse self-attention for semantic segmentation. arXiv, abs/1907.12273, 2019. 5

    1907 arXiv

  17. [25]

    CCNet: Criss-cross attention for semantic segmentation

    Zilong Huang, Xinggang Wang, Lichao Huang, Chang Huang, Yunchao Wei, and Wenyu Liu. CCNet: Criss-cross attention for semantic segmentation. In IEEE/CVF Int. Conf. Comput. Vis. (ICCV) , pages 603–612, Seoul, South Korea,

  18. [26]

    Hughes-Oliver

    Jacqueline M. Hughes-Oliver. Population and empirical PR curves for assessment of ranking algorithms. arXiv, abs/1810.08635, 2018. 2

    2018 arXiv

  19. [27]

    Panoptic segmentation

    Alexander Kirillov, Kaiming He, Ross Girshick, Carsten Rother, and Piotr Dollar. Panoptic segmentation. In 9 IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , pages 9396–9405, Long Beach, CA, USA, 2019. 5

    2019

  20. [28]

    PointRend: Image segmentation as rendering

    Alexander Kirillov, Yuxin Wu, Kaiming He, and Ross Gir- shick. PointRend: Image segmentation as rendering. In IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , pages 9796–9805, Seattle, W A, USA, 2020. 5

    2020

  21. [29]

    Expectation-maximization attention net- works for semantic segmentation

    Xia Li, Zhisheng Zhong, Jianlong Wu, Yibo Yang, Zhouchen Lin, and Hong Liu. Expectation-maximization attention net- works for semantic segmentation. In IEEE/CVF Int. Conf. Comput. Vis. (ICCV), pages 9166–9175, Seoul, South Korea,

  22. [30]

    Shah, and Aarti Singh

    Yusha Liu, Yichong Xu, Nihar B. Shah, and Aarti Singh. Integrating rankings into quantized scores in peer review. Trans. Mach. Learn. Res., pages 1–28, 2022. 1

    2022

  23. [31]

    Fully convolutional networks for semantic segmentation

    Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , pages 3431–3440, Boston, MA, USA, 2015. 5

    2015

  24. [32]

    On the statistical consistency of algo- rithms for binary classification under class imbalance

    Aditya Menon, Harikrishna Narasimhan, Shivani Agarwal, and Sanjay Chawla. On the statistical consistency of algo- rithms for binary classification under class imbalance. InInt. Conf. Mach. Learn. (ICML), pages 603–611, Atlanta, Geor- gia, USA, 2013. 2

    2013

  25. [33]

    Receiver operating characteristic curve: overview and practical use for clinicians

    Francis Sahngun Nahm. Receiver operating characteristic curve: overview and practical use for clinicians. Korean Journal of Anesthesiology, 75(1):25–36, 2022. 2

    2022

  26. [34]

    How trustwor- thy are performance evaluations for basic vision tasks?IEEE Trans

    Tran Thien Dat Nguyen, Hamid Rezatofighi, Ba-Ngu V o, Ba-Tuong V o, Silvio Savarese, and Ian Reid. How trustwor- thy are performance evaluations for basic vision tasks?IEEE Trans. Pattern Anal. Mach. Intell. , 45(7):8538–8552, 2023. 2

    2023

  27. [35]

    Summa- rizing the performances of a background subtraction algo- rithm measured on several videos

    Sébastien Piérard and Marc Van Droogenbroeck. Summa- rizing the performances of a background subtraction algo- rithm measured on several videos. In IEEE Int. Conf. Image Process. (ICIP), pages 3234–3238, Abu Dhabi, United Arab Emirates, 2020. 8

    2020

  28. [36]

    The Tile: A 2D map of ranking scores for two-class classification

    Sébastien Piérard, Anaïs Halin, Anthony Cioppa, Adrien Deliège, and Marc Van Droogenbroeck. The Tile: A 2D map of ranking scores for two-class classification. arXiv, abs/2412.04309, 2024. 2, 3, 7, 8, 12, 13

    2024 arXiv

  29. [37]

    Foundations of the theory of performance-based ranking

    Sébastien Piérard, Anaïs Halin, Anthony Cioppa, Adrien Deliège, and Marc Van Droogenbroeck. Foundations of the theory of performance-based ranking. In IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , Nashville, TN, USA, 2025. 2, 3, 7, 8

    2025

  30. [38]

    Rudra P. K. Poudel, Stephan Liwicki, and Roberto Cipolla. Fast-SCNN: Fast semantic segmentation network. arXiv, abs/1902.04502, 2019. 5

    1902 arXiv

  31. [39]

    Evaluation: from precision, re- call and F-measure to ROC, informedness, markedness & correlation

    David Martin Ward Powers. Evaluation: from precision, re- call and F-measure to ROC, informedness, markedness & correlation. J. Mach. Learn. Technol., 2(1):37–63, 2011. 2

    2011

  32. [40]

    Vi- sion transformers for dense prediction

    Rene Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vi- sion transformers for dense prediction. In IEEE/CVF Int. Conf. Comput. Vis. (ICCV), pages 12159–12168, Montréal, Can., 2021. 5

    2021

  33. [41]

    Alvarez, Luis M

    Eduardo Romera, Jose M. Alvarez, Luis M. Bergasa, and Roberto Arroyo. ERFNet: Efficient residual factorized Con- vNet for real-time semantic segmentation. IEEE Trans. In- tell. Transp. Syst., 19(1):263–272, 2018. 5

    2018

  34. [42]

    Fully convolutional networks for semantic segmentation

    Evan Shelhamer, Jonathan Long, and Trevor Darrell. Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell., 39(4):640–651, 2017. 5

    2017

  35. [43]

    A systematic analysis of performance measures for classification tasks

    Marina Sokolova and Guy Lapalme. A systematic analysis of performance measures for classification tasks. Inf. Process. & Manag., 45(4):427–437, 2009. 8

    2009

  36. [44]

    Segmenter: Transformer for semantic segmenta- tion

    Robin Strudel, Ricardo Garcia, Ivan Laptev, and Cordelia Schmid. Segmenter: Transformer for semantic segmenta- tion. In IEEE/CVF Int. Conf. Comput. Vis. (ICCV) , pages 7242–7252, Montréal, Can., 2021. 5

    2021

  37. [45]

    Classification assessment methods

    Alaa Tharwat. Classification assessment methods. Appl. Comput. Informatics, 17(1):168–192, 2021. 2

    2021

  38. [46]

    Non-local neural networks

    Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaim- ing He. Non-local neural networks. In IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , pages 7794–7803, Salt Lake City, UT, USA, 2018. 5

    2018

  39. [47]

    FastFCN: Rethinking dilated convolu- tion in the backbone for semantic segmentation

    Huikai Wu, Junge Zhang, Kaiqi Huang, Kongming Liang, and Yizhou Yu. FastFCN: Rethinking dilated convolu- tion in the backbone for semantic segmentation. arXiv, abs/1903.11816, 2019. 5

    1903 arXiv

  40. [48]

    CGNet: A light-weight context guided network for semantic segmentation

    Tianyi Wu, Sheng Tang, Rui Zhang, Juan Cao, and Yong- dong Zhang. CGNet: A light-weight context guided network for semantic segmentation. IEEE Trans. Image Process., 30: 1169–1179, 2021. 5

    2021

  41. [49]

    Unified perceptual parsing for scene understand- ing

    Tete Xiao, Yingcheng Liu, Bolei Zhou, Yuning Jiang, and Jian Sun. Unified perceptual parsing for scene understand- ing. In Eur. Conf. Comput. Vis. (ECCV) , pages 432–448. Springer Int. Publ., 2018. 5

    2018

  42. [50]

    Alvarez, and Ping Luo

    Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M. Alvarez, and Ping Luo. SegFormer: Simple and efficient design for semantic segmentation with transformers. In Adv. Neural Inf. Process. Syst. (NeurIPS) , pages 12077– 12090, 2021. 5

    2021

  43. [51]

    Side adapter network for open-vocabulary seman- tic segmentation

    Mengde Xu, Zheng Zhang, Fangyun Wei, Han Hu, and Xi- ang Bai. Side adapter network for open-vocabulary seman- tic segmentation. In IEEE/CVF Conf. Comput. Vis. Pat- tern Recognit. (CVPR), pages 2945–2954, New Orleans, LA, USA, 2023. 5

    2023

  44. [52]

    Disentangled non-local neural networks

    Minghao Yin, Zhuliang Yao, Yue Cao, Xiu Li, Zheng Zhang, Stephen Lin, and Han Hu. Disentangled non-local neural networks. In Eur. Conf. Comput. Vis. (ECCV) , pages 191–

  45. [53]

    BiSeNet: Bilateral segmentation network for real-time semantic segmentation

    Changqian Yu, Jingbo Wang, Chao Peng, Changxin Gao, Gang Yu, and Nong Sang. BiSeNet: Bilateral segmentation network for real-time semantic segmentation. In Eur. Conf. Comput. Vis. (ECCV) , pages 325–341. Springer Int. Publ.,

  46. [54]

    BiSeNet V2: Bilateral net- work with guided aggregation for real-time semantic seg- mentation

    Changqian Yu, Changxin Gao, Jingbo Wang, Gang Yu, Chunhua Shen, and Nong Sang. BiSeNet V2: Bilateral net- work with guided aggregation for real-time semantic seg- mentation. Int. J. Comput. Vis. , 129(11):3051–3068, 2021. 5

    2021

  47. [55]

    BDD100K: A diverse driving dataset for hetero- geneous multitask learning

    Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingy- ing Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell. BDD100K: A diverse driving dataset for hetero- geneous multitask learning. In IEEE Int. Conf. Comput. Vis. 10 Pattern Recognit. (CVPR) , pages 2633–2642, Seattl...

    2020

  48. [56]

    Object- contextual representations for semantic segmentation

    Yuhui Yuan, Xilin Chen, and Jingdong Wang. Object- contextual representations for semantic segmentation. In Eur. Conf. Comput. Vis. (ECCV) , pages 173–190. Springer Int. Publ., 2020. 5

    2020

  49. [57]

    Con- text encoding for semantic segmentation

    Hang Zhang, Kristin Dana, Jianping Shi, Zhongyue Zhang, Xiaogang Wang, Ambrish Tyagi, and Amit Agrawal. Con- text encoding for semantic segmentation. In IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), pages 7151– 7160, Salt Lake City, UT, USA, 2018. 5

    2018

  50. [58]

    K-Net: Towards unified image segmentation

    Wenwei Zhang, Jiangmiao Pang, Kai Chen, and Chen Change Loy. K-Net: Towards unified image segmentation. In Adv. Neural Inf. Process. Syst. (NeurIPS), pages 10326–10338. Curran Assoc. Inc., 2021. 5

    2021

  51. [59]

    Pyramid scene parsing network

    Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. Pyramid scene parsing network. In IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , pages 6230–6239, Honolulu, HI, USA, 2017. 5

    2017

  52. [60]

    ICNet for real-time semantic segmenta- tion on high-resolution images

    Hengshuang Zhao, Xiaojuan Qi, Xiaoyong Shen, Jianping Shi, and Jiaya Jia. ICNet for real-time semantic segmenta- tion on high-resolution images. In Eur. Conf. Comput. Vis. (ECCV), pages 418–434. Springer Int. Publ., 2018. 5

    2018

  53. [61]

    PSANet: Point- wise spatial attention network for scene parsing

    Hengshuang Zhao, Yi Zhang, Shu Liu, Jianping Shi, Chen Change Loy, Dahua Lin, and Jiaya Jia. PSANet: Point- wise spatial attention network for scene parsing. In Eur. Conf. Comput. Vis. (ECCV) , pages 270–286. Springer Int. Publ., 2018. 5

    2018

  54. [62]

    Torr, and Li Zhang

    Sixiao Zheng, Jiachen Lu, Hengshuang Zhao, Xiatian Zhu, Zekun Luo, Yabiao Wang, Yanwei Fu, Jianfeng Feng, Tao Xiang, Philip H.S. Torr, and Li Zhang. Rethinking seman- tic segmentation from a sequence-to-sequence perspective with transformers. In IEEE/CVF Conf. Comput. Vis. Pat...

  55. [63]

    Scene parsing through ADE20K dataset

    Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Scene parsing through ADE20K dataset. In IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pages 5122–5130, Honolulu, HI, USA,

  56. [64]

    Asymmetric non-local neural networks for semantic segmentation

    Zhen Zhu, Mengdu Xu, Song Bai, Tengteng Huang, and Xi- ang Bai. Asymmetric non-local neural networks for semantic segmentation. In IEEE/CVF Int. Conf. Comput. Vis. (ICCV), pages 593–602, Seoul, South Korea, 2019. 5 11 A. Supplementary Material We provide in this supplementary ...

    2019

  57. [66]

    Direct computation: If the values ofP ({f p}), P ({f n}), P ({tp}), and P ({tn}) are known, the first method con- sists in computing the values of the ranking scores in each point of the Tile using Eq. (3)

  58. [67]

    the harmonic mean) and horizontally with f : x 7→ (1 − x)−1 [36]

    Interpolation: Knowing the values taken by the scores T P R, T N R, N P V, and P P Vat the four corners of the Tile, the values taken by the remaining scores can be determined by averaging, using f-means, vertically with f : x 7→ x−1 (i.e. the harmonic mean) and horizontally w...

  59. [68]

    Equation system resolution: The third method involves first, determining the values of P ({f p}), P ({f n}), P ({tp}), and P ({tn}) by solving a system of 4 equa- tions with 4 unknowns, then, computing the values of the ranking scores using Eq. (3). Specifically, if the values...

    2012

  60. [207]

    Publ., 2020

    Springer Int. Publ., 2020. 5

    2020

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.