FSEVAL: Feature Selection Evaluation Toolbox and Dashboard

Arthur Zimek; Muhammad Rajabinasab

REVIEW 2 major objections 2 minor 2 cited by

FSEVAL: Feature Selection Evaluation Toolbox and Dashboard

T0 review · 2 major / 2 minor · reviewed 2026-07-05 · glm-5.2

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Toolbox standardizes how feature selection algorithms are judged

desk verdict Abstract-only review of a feature selection evaluation toolbox; cannot assess implementation quality or novelty from available material read the letter →

arxiv 2604.18227 v3 pith:VQ4QC6HQ submitted 2026-04-20 cs.LG

Muhammad Rajabinasab , Arthur Zimek This is my paper · ORCID

classification cs.LG

keywords featureselectionevaluationtoolboxmachinelearningsupervisedunsupervisedvisualizationdashboardreproducibility

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces FSEVAL, a software toolbox and visualization dashboard designed to standardize the evaluation of feature selection algorithms. Feature selection — the process of identifying which input variables carry useful signal while discarding redundant ones — is used in both supervised and unsupervised machine learning, but practitioners lack a unified way to compare methods. FSEVAL bundles multiple evaluation metrics and a visual interface so that researchers can run comprehensive, side-by-side assessments of different feature selection algorithms without each team reimplementing its own evaluation pipeline. The central claim is that a single, well-designed toolbox can make feature selection evaluation more consistent, reproducible, and accessible across the field.

What carries the argument

FSEVAL is a software toolbox paired with a visualization dashboard. It integrates multiple evaluation metrics applicable to both supervised and unservised feature selection, and presents results through a unified interface so that different algorithms can be compared on common ground.

What would settle it

If independent users find that FSEVAL's metric coverage omits widely used evaluation criteria, or that its results disagree with manually computed benchmarks on standard datasets, the claim of standardized comprehensive evaluation would not hold.

Watch

Extended reading notes

Core claim

The paper's central contribution is the FSEVAL toolbox itself: a unified evaluation and visualization platform that covers both supervised and unsupervised feature selection, providing standardized metrics and a dashboard so that comparisons between algorithms are no longer ad hoc. The load-bearing premise is that the set of metrics, algorithms, and data handling routines bundled into FSEVAL is broad and correct enough to justify calling the evaluation comprehensive and standardized.

Load-bearing premise

The paper assumes that one toolbox can meaningfully cover the diverse landscape of feature selection algorithms, metrics, and data regimes well enough to justify calling its evaluation comprehensive and standardized. If the coverage is too narrow or the metrics are implemented incorrectly, the central value proposition weakens.

Editorial extensions

If this is right

If adopted, FSEVAL could reduce the reproducibility gap in feature selection research by giving every group the same evaluation baseline.
A standardized evaluation toolbox makes it easier to detect when a newly proposed feature selection method is merely re-deriving an existing approach under different packaging.
The dashboard lowers the barrier for practitioners in applied domains who need to choose a feature selection method but lack the expertise to assemble a full evaluation suite.
Unified metrics across supervised and unsupervised settings could reveal whether algorithms that perform well in one regime transfer to the other.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The value of FSEVAL depends on community adoption: a standardization tool only standardizes if enough researchers use it, so the paper's impact is partly sociological rather than purely technical.
If the toolbox is open and extensible, the most useful long-term contribution may be the evaluation protocol itself rather than any particular implementation, since the protocol could be re-implemented in other frameworks.
Comparing supervised and unsupervised feature selection within one toolbox raises the question of whether there exist universal quality measures for selected feature subsets that are independent of downstream task type — a question the toolbox could empirically surface even if it does not resolve it.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

read the letter

Here's the honest situation: I only have the abstract for this paper, and the abstract is thin. FSEVAL is a software toolbox for evaluating feature selection algorithms, with a visualization dashboard, covering both supervised and unsupervised settings. That's the entire substantive content I can see. There is no full text, no code link, no list of supported algorithms or metrics, no comparison to existing toolboxes, and no quantitative claim of any kind. So this letter is necessarily limited — I'm telling you what I can and cannot infer, not giving you a verdict on the work itself. What the paper appears to offer: a unified evaluation framework for feature selection that bundles multiple metrics and algorithms under one interface with a dashboard. If the implementation is solid, broad, and well-tested, this is genuinely useful infrastructure. Feature selection is a fragmented subfield where people routinely hand-roll their own evaluation pipelines, and a well-maintained, open-source toolbox with correct metric implementations would save real time and improve comparability across papers. That is a legitimate contribution for a tool paper. The problem is that the abstract gives me no way to assess whether any of that holds. It does not enumerate which algorithms or metrics are supported. It does not mention unit tests, benchmark datasets, or comparison with scikit-feature, scikit-learn's feature_selection module, or other existing frameworks. It does not say whether the code is open-source or where to find it. The word 'comprehensive' appears twice but is not substantiated. For a tool paper, the abstract needs to at minimum signal coverage breadth and code availability — this one does neither. The reader's UNVERDICTED verdict with low confidence is the right call given what we have. I disagree with nothing in the stress-test note; it correctly identifies that the real risks (metric bugs, narrow coverage, thin wrapper over existing libraries) are all uncheckable from the abstract. This is not a flaw in the paper's logic — there is no argument to stress-test — but it is a presentation weakness. A tool paper that does not foreground its code repository and coverage in the abstract is making it hard for reviewers to do their job. My recommendation: this deserves a serious referee only if the full text ships open code with documented algorithm/metric coverage and evidence of correct implementation (unit tests, benchmark comparisons, or at least a clear API specification). Without that, there is nothing to review. If the full text does include those things, I would want a referee who can actually run the toolbox and check metric correctness against known baselines. Set would_accept_peer_review=true — the paper's value hinges entirely on details I cannot see, and a referee with full access is the right person to make that call.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes FSEVAL, a feature selection evaluation toolbox with an accompanying visualization dashboard. The stated goal is to provide a standardized, unified framework for comprehensively evaluating feature selection algorithms across both supervised and unsupervised settings. The paper is positioned as a software tool paper for researchers in the field. Only the abstract was available for review; the full text and code repository were not provided. Consequently, this report is based solely on the abstract and cannot assess the correctness of metric implementations, the breadth of algorithm and metric coverage, the quality of the visualization dashboard, or the software engineering practices (testing, documentation, reproducibility).

Significance. If the toolbox delivers on its stated goals, it could be a useful contribution to the feature selection community by lowering the barrier to rigorous, standardized evaluation. However, the abstract makes no falsifiable quantitative claim and provides no evidence of the toolbox's scope, correctness, or adoption. The significance of a software tool paper rests entirely on execution details that are not available for assessment. No machine-checked proofs, reproducible code artifacts, parameter-free derivations, or falsifiable predictions are evident from the abstract. The central value proposition — that FSEVAL enables 'comprehensive' and 'standardized' evaluation — is a claim that requires substantiation through the full paper and code repository.

major comments (2)

The full text of the manuscript was not available for review. Only the abstract was provided. For a software toolbox paper, the load-bearing claims concern metric implementation correctness, breadth of algorithm and metric coverage, usability of the dashboard, comparison with existing FS evaluation frameworks, and software engineering quality (unit tests, documentation, reproducibility). None of these can be assessed from the abstract alone. The manuscript cannot be meaningfully evaluated without the full text and access to the code repository. This is the primary obstacle to providing a verdict.
Even at the abstract level, the claims of 'comprehensive' and 'standardized' evaluation are unsubstantiated. The abstract does not enumerate which feature selection algorithms are supported, which evaluation metrics are included, which data regimes are covered, or how FSEVAL compares to existing toolboxes (e.g., scikit-feature, ASU's feature selection repository). For a tool paper, these details are essential to justify the central value proposition. The abstract should at minimum specify the scope of coverage and position the tool against prior work.

minor comments (2)

The abstract would benefit from concrete specifics: the number of supported algorithms, the number of evaluation metrics, the supported data types, and a brief comparison statement against existing feature selection repositories.
The phrase 'involved with discriminating redundant features from informative ones' is slightly awkward; consider rephrasing for clarity (e.g., 'concerned with distinguishing redundant features from informative ones').

Simulated Author's Rebuttal

2 responses · 1 unresolved

We thank the referee for the careful reading and for clearly identifying the core issue: the review was conducted on the abstract alone, with the full text and code repository unavailable. We agree that a software tool paper cannot be meaningfully evaluated without access to the full manuscript and code. We address each major comment below.

read point-by-point responses

Referee: The full text of the manuscript was not available for review. Only the abstract was provided. For a software toolbox paper, the load-bearing claims concern metric implementation correctness, breadth of algorithm and metric coverage, usability of the dashboard, comparison with existing FS evaluation frameworks, and software engineering quality (unit tests, documentation, reproducibility). None of these can be assessed from the abstract alone. The manuscript cannot be meaningfully evaluated without the full text and access to the code repository. This is the primary obstacle to providing a verdict.

Authors: The referee is entirely correct that the full text and code repository are essential for evaluating a software tool paper, and we agree that the abstract alone is insufficient. We believe this was a submission or review system issue rather than a deficiency in the manuscript itself: the full paper, supplementary materials, and the public code repository (with documentation, unit tests, and usage examples) were submitted as part of the package. We respectfully request that the full manuscript and repository be made available for review so that the referee can assess the specific execution details listed — metric implementation correctness, algorithm and metric coverage, dashboard usability, comparison with existing frameworks, and software engineering practices. We are confident that these materials substantiate the claims made in the abstract. That said, we acknowledge the referee's point that the abstract could be more informative even in isolation, and we will revise it accordingly (see our response to the second major comment). revision: partial
Referee: Even at the abstract level, the claims of 'comprehensive' and 'standardized' evaluation are unsubstantiated. The abstract does not enumerate which feature selection algorithms are supported, which evaluation metrics are included, which data regimes are covered, or how FSEVAL compares to existing toolboxes (e.g., scikit-feature, ASU's feature selection repository). For a tool paper, these details are essential to justify the central value proposition. The abstract should at minimum specify the scope of coverage and position the tool against prior work.

Authors: This is a fair criticism. Even granting that the full text was unavailable, the abstract should stand on its own and provide enough detail for a reader to understand the tool's scope and positioning. We will revise the abstract to (1) enumerate the categories of feature selection algorithms supported (e.g., filter, wrapper, embedded, and hybrid methods across supervised and unsupervised settings), (2) list the key evaluation metrics included (e.g., stability measures, clustering quality metrics, downstream classifier accuracy, redundancy analysis), (3) specify the data regimes covered, and (4) explicitly position FSEVAL against existing toolboxes such as scikit-feature and the ASU repository, noting what FSEVAL adds — namely, a unified dashboard for visualization and a standardized evaluation pipeline across both supervised and unsupervised settings. We agree that without these details the terms 'comprehensive' and 'standardized' are unsupported at the abstract level. revision: yes

standing simulated objections not resolved

The referee's primary concern — that the full text and code were unavailable — cannot be resolved through revision alone. We believe the full manuscript and repository were submitted but were not accessible to the referee, likely due to a system or handling issue. We respectfully request that the complete submission package be made available for re-review. We cannot manufacture a response to substantive critiques of implementation correctness, coverage breadth, or software engineering quality when those critiques have not yet been raised against the actual content.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: this is a descriptive software toolbox paper with no derivation chain to evaluate.

full rationale

The abstract of FSEVAL presents a software toolbox and visualization dashboard for evaluating feature selection algorithms. It makes no quantitative claims, derives no equations, and invokes no theorems or prior results as load-bearing premises. The central claim is descriptive — that the toolbox provides standardized, comprehensive evaluation — and cannot be assessed for circularity because there is no logical derivation chain to inspect. No self-citation, fitted-parameter-as-prediction, or definitional reduction is present in the available text. Any real risks (metric correctness, coverage breadth) are correctness/implementation concerns, not circularity concerns, and would require the full text and code repository to evaluate. The honest finding is no circularity at the abstract level.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

No free parameters or invented entities are evident from the abstract. The axioms are domain assumptions about evaluation metric sufficiency and regime coverage.

assumptions (2)

domain assumption Existing feature selection evaluation metrics are correct and sufficient for comprehensive comparison.
The toolbox bundles evaluation metrics; the abstract assumes these metrics are the right ones to use. Stated implicitly throughout the abstract.
domain assumption A unified toolbox can meaningfully cover both supervised and unsupervised feature selection regimes.
The abstract claims coverage of both settings; this is a design assumption that the full paper would need to justify.

how reviews work

0 comments

Cite this review

Pith. "Pith review of FSEVAL: Feature Selection Evaluation Toolbox and Dashboard." pith.science (2026). https://pith.science/paper/VQ4QC6HQ

@misc{pith2026260418227,
  author       = {Pith},
  title        = {Pith review of: FSEVAL: Feature Selection Evaluation Toolbox and Dashboard},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VQ4QC6HQ}},
  note         = {Machine review of arXiv:2604.18227}
}

read the original abstract

Feature selection is a fundamental machine learning and data mining task, involved with discriminating redundant features from informative ones. It is an attempt to address the curse of dimensionality by removing the redundant features, while unlike dimensionality reduction methods, preserving explainability. Feature selection is conducted in both supervised and unsupervised settings, with different evaluation metrics employed to determine which feature selection algorithm is the best. In this paper, we propose FSEVAL, a feature selection evaluation toolbox accompanied with a visualization dashboard, with the goal to make it easy to comprehensively evaluate feature selection algorithms. FSEVAL aims to provide a standardized, unified, evaluation and visualization toolbox to help the researchers working in the field, conduct extensive and comprehensive evaluation of feature selection algorithms with ease.

Figures

Figures reproduced from arXiv: 2604.18227 by the authors.

**Figure 1.** The FSEval Dashboard: An interactive environment for analytical insights and comparisons on feature selection algorithms. A critical feature of the dashboard is its ability to generate high-fidelity, publicationready figures [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗

**Figure 2.** Publication-ready line plot generated by the dashboard, demonstrating the performance trajectory and variance of multiple algorithms. 2500 5000 7500 10000 12500 15000 17500 20000 Number of Features 10 4 10 3 10 2 10 1 10 0 10 1 10 2 10 3 Runtime (s) Variance Correlation Laplacian Random VCSDFS LIDFS SOGFS LLSRFS SCFS MCFS [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

**Figure 3.** Scalability analysis showing runtime (seconds) against the number of features. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Scalability analysis showing runtime (seconds) against the number of instances. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Towards Truly Unsupervised Evaluation of Feature Selection
cs.LG 2026-08 conditional novelty 5.0 of 10

A new label-free evaluation framework ranks feature selection methods by the optimal transport distance between the selected feature subset and the first principal components of the full data.
An Empirical Study of Feature Selection Granularity
cs.LG 2026-07 conditional novelty 4.0 of 10

Greedy recursive elimination almost consistently beats one-shot global feature ranking on supervised metrics across five selectors, at higher computational cost.

Pith tools

Reviewed July 5, 2026 · model on record in the stance chip above.