{"id":"4005d633-1c80-455e-b59e-ce24f0968a7b","arxiv_id":"2506.00053","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A routine feature-selection and random-subspace pipeline for brain cancer classification reports a 96% score, but the paper's own tables show 93.8% and the method is a known technique.","lead":"This paper applies Kendall gene filtering and a random-feature-subset ensemble to a public brain cancer microarray dataset, claiming 96% classification accuracy. The claim is not supported by the paper's own tables, which show 93.8% for the best configuration, and the feature selection step leaks test information into training.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central 96% claim is internally unsupported: the best reported result is 93.8% (NB+LDA, Table 3), and the Kendall pre-filter before 3-fold CV leaks test information, so the headline accuracy cannot stand.","rationale":"The reader's weakest assumption correctly identifies data leakage as a serious threat. I agree that applying Kendall selection before CV invalidates the accuracy estimate. I additionally find a more direct internal inconsistency: the abstract's 96% and the 9.09% improvement do not match any result in the paper; the best tabled accuracy is 93.8%. This means the central claim is unsupported even before considering leakage. The 9.09% figure is exactly 0.96/0.88, which reveals the intended computation, but the 0.96 never appears in the results. The confusion-matrix percentages are impossible, reinforcing that the empirical reporting is unreliable. I recommend keeping the REJECT verdict; the requested changes would be to provide a leakage-free nested evaluation and to reconcile or remove the 96% claim.","tokens_in":9675,"tokens_out":2907,"duration_ms":29727,"concrete_test":"Re-run the GSE50161 pipeline with leakage-free feature selection: within each of the 3 CV folds, apply the Kendall test and normalization only to the training portion, select the top 20,890 genes, then train NB on LDA projections and report test accuracy for that fold. Compare the average to the paper's 96% and to Table 3's 93.8%. Also search the author code/reproducibility materials for any configuration that yields 96%; if no such configuration exists, the central claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is the abstract's 'test score of 96%... outperforming existing methods by 9.09%.' No reported experiment in Tables 2-6 reaches 96%; the highest value is 0.938 for NB+LDA (Table 3, repeated in Table 6). The 9.09% improvement only matches 0.96/0.88 from Table 1, not 0.938/0.88 = 6.6%. Thus the headline number appears disconnected from the results. Independently, Section 3.1 states the Kendall statistical test is applied to the full dataset before any train/test split, and Section 4.1 confirms selection of the 20,890 genes from all 130 samples before model training. With 3-fold CV, two of three folds are used as test data in turn, so each test sample has already contributed to gene ranking and normalization. This is a standard leakage path that inflates cross-validated accuracy. The confusion-matrix descriptions also contain impossible percentages (e.g., 590% for class 1, 100% misclassifications), further undermining the quantitative reporting. Together these issues leave the central claim without valid empirical support.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a supervised learning pipeline for brain cancer microarray classification (GSE50161). The method first applies a Kendall statistical test to the full dataset to reduce the feature space from 54,675 to 20,890 genes, then constructs ensemble classifiers through feature selection without replacement (FSWOR) and four projection techniques (PCA, LDA, GRP, SRP), evaluated with 3-fold cross-validation. The central claims are that the LDA-projection NB ensemble achieves a test accuracy of 96%, outperforming prior work by 9.09%.","tokens_in":9897,"tokens_out":2312,"duration_ms":23940,"significance":"If the claims were validated, the paper would offer a useful empirical benchmark for high-dimensional gene-expression classification. The work uses a publicly available dataset and presents a clearly structured pipeline. However, the reported headline accuracy is contradicted by the paper's own tables, the validation protocol leaks test information into feature selection, and the confusion-matrix descriptions contain impossible percentages. As a result, the central empirical contribution is currently unsupported.","major_comments":[{"comment":"The abstract and Table 1 claim a test score of 96% and a 9.09% improvement over prior methods, but Tables 3 and 6 report the best accuracy as 0.938 for Naïve Bayes with LDA. The stated 9.09% improvement appears to be computed as 0.96/0.88, not as 0.938/0.88 ≈ 6.6%. The headline number is therefore not derivable from the reported experimental results.","section":"Abstract, Table 1, Table 3, Table 6"},{"comment":"The Kendall statistical test is applied to the complete dataset before any train/test split, and the 20,890-gene subset is selected using all 130 samples. Because the evaluation uses 3-fold cross-validation, each held-out fold has already contributed to gene ranking and normalization. This is a standard feature-selection leakage path that inflates the cross-validated accuracy, so the comparison with the baseline is not a fair estimate of generalization performance.","section":"Sections 3.1 and 4.1"},{"comment":"The confusion-matrix descriptions in Figure 12 contain percentages that exceed 100% (e.g., 590% correct for class 1, 220% misclassified, 110%, 900%, 870%) and are internally inconsistent across classes. If these values are literal, they indicate severe numerical errors; if they are unnormalized counts mislabeled as percentages, the reporting is misleading. Either way, the detailed per-class performance claims cannot be accepted.","section":"Figure 12 and Section 4.2"},{"comment":"The paper evaluates five classifiers under four projections (20 combinations) and then selects, post hoc, the best model per projection for the summary in Table 6. With no separate validation set and no adjustment for multiple comparisons, the reported best accuracy (0.938) is optimistically biased and does not represent a pre-specified model's expected test performance.","section":"Table 6 and Section 4.2"}],"minor_comments":[{"comment":"The text contains numerous typographical errors and formatting inconsistencies, including “project ion”, “Naïve Bays”, “violets Gaussian Markov assumption” (should be “violates”), and inconsistent spacing in numbers such as “20, 890” and “54, 675”.","section":"Throughout"},{"comment":"The FSWOR technique is described only qualitatively; no algorithm, pseudocode, or precise definition of the feature-subset sizes or sampling strategy is provided, which limits reproducibility.","section":"Section 3.2"},{"comment":"The captions refer to accuracy versus “number of projections,” but the manuscript does not state the ranges of projection dimensions tested, the stopping criterion, or how the optimal dimension was chosen.","section":"Figures 8–11"},{"comment":"The comparison baseline is attributed to “Bruno et al.” in the introduction, but the reference list contains Feltes et al. [1]; the source of the 0.88 CV score is not clearly identified.","section":"Table 1 and References"}],"recommendation":"reject","confidential_remarks":"The central empirical claims are internally contradicted by the paper's own tables, and the experimental protocol includes a known leakage path that would bias the reported accuracy. These problems are load-bearing and cannot be fixed by local editing; a re-run of the experiments under a nested or properly split feature-selection protocol would be required, and the headline claim would likely change materially."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper's central claim doesn't survive contact with its own tables. The abstract promises 96% test accuracy and 9.09% over prior work, but the highest number in Tables 2-6 is 0.938 (NB with LDA). The 9.09% figure only works for 0.96 vs 0.88, not for 0.938 vs 0.88. So the headline result appears nowhere in the results.\n\nWhat's worth credit: the paper is a clean empirical exercise—five classifiers, four projections, one public brain-cancer microarray dataset (GSE50161/CUMIDA). The tables are readable and mostly consistent with each other. Applying a univariate filter plus a random-subspace ensemble to this dataset is a reasonable thing to try. That's also the problem: FSWOR is a standard random subspace/feature bagging idea and Kendall filtering is a textbook univariate filter, so the claimed novelty is thin. The literature review cites random projection papers but doesn't engage with the feature-selection literature that would put this in context.\n\nSoft spots, in order of severity. First, feature selection leaks. Sections 3.1 and 4.1 say the Kendall test is applied to the full 130-sample dataset before any split, and with 3-fold CV each test fold has already contributed to gene ranking. The 93.8% is therefore optimistically biased, and the comparison with Feltes et al. is unfair. Second, the confusion-matrix text contains impossible percentages—590%, 220%, 110%—which points to numerical errors in how the matrices were averaged or read. Third, no code, no hyperparameters, no random seed, and only one dataset, so nothing is independently reproducible or externally validated. Fourth, the comparison rests on a single baseline, and the abstract's 'Bruno et al.' attribution is muddled (the baseline is Feltes et al.). None of these are fatal to the idea of FSWOR-plus-filter, but they are fatal to the submitted results.\n\nIf I were editing: desk reject. The central empirical claim is internally contradicted, and the main methodological choice (selection before cross-validation) invalidates the accuracy estimate. The authors could salvage this by moving the Kendall filter inside the CV loop, reporting the actual ensemble results, and fixing the confusion-matrix arithmetic. As submitted, it doesn't deserve referee time.","headline":"The headline 96% accuracy is contradicted by the paper's own tables, and the feature-selection-before-CV leakage invalidates even the lower reported numbers.","tokens_in":10483,"tokens_out":2557,"would_cite":false,"duration_ms":27312,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A pipeline that combines Kendall gene filtering, random feature selection without replacement, and LDA projection reaches 96 percent test accuracy on a brain cancer microarray benchmark.","keywords":["brain cancer","gene expression","microarray","feature selection without replacement","Kendall test","random projection","LDA","ensemble classification"],"falsifier":"Recompute the pipeline with the Kendall test and FSWOR sampling performed only inside each training fold, and check whether the held-out test accuracy stays at 96 percent; a material drop would show the reported gain over the 88 percent baseline comes from selection leakage rather than the method itself.","tokens_in":9427,"feed_emoji":"🧬","tokens_out":5771,"duration_ms":46986,"temperature":0.7,"pith_summary":"This paper claims that a three-stage pipeline—Kendall statistical testing to prune genes, random feature selection without replacement to build diverse views, and projection before classification—raises brain cancer subtype classification accuracy on the GSE50161 dataset to 96 percent. That would beat the previous benchmark on the same data by 9.09 percent. The authors test five classifiers under four projections and find Naïve Bayes with LDA projection is the winning configuration. Their case is that statistical preselection plus diversity from sampling without replacement solves the small-sample, high-dimensional overfitting problem.","feed_headline":"Kendall-test plus LDA pipeline hits 96% on brain cancer data","feed_subtitle":"Filtering 54,675 genes with the Kendall test and LDA projection beats the prior benchmark by 9 points on GSE50161.","key_machinery":"Feature Selection Without Replacement (FSWOR) is the central mechanism: features are drawn randomly without replacement to form distinct subsets, so each member of the ensemble trains on a different view of the data, which mitigates overfitting and adds diversity. Kendall's statistical test first reduces the gene space from 54,675 to 20,890 genes, and each feature subset is then projected using PCA, LDA, Gaussian random projection, or sparse random projection. LDA, which maximizes class separability, is the projection that carries the winning result. The experiment compares five classifiers (DT, SVM, LR, KNN, NB) under each projection using 3-fold cross-validation, with the ensemble combining the individual learners.","core_discovery":"The central discovery is that feature selection without replacement, combined with Linear Discriminant Analysis projection, pushes a Naïve Bayes classifier to 0.938 accuracy (96 percent in the paper's reporting) on the brain cancer microarray dataset GSE50161, above the 0.88 previously reported for this benchmark. Under the same preprocessing, the same classifier reaches only 0.892 with PCA, 0.815 with Gaussian random projection, and 0.835 with sparse random projection, while SVM becomes the best model under PCA, GRP, and SRP. The paper attributes the gain to the ensemble diversity created by sampling features without replacement and to LDA's ability to make the five tissue classes linearly separable in the reduced space. Decision trees perform worst under random projections, with accuracy as low as 0.619 for SRP.","pith_inferences":["An untested but direct follow-up is to wrap the Kendall test and FSWOR inside each cross-validation fold; that would tell whether the 96 percent figure survives data leakage from the feature-selection step.","The dominance of LDA over random projections on this dataset suggests the five tissue classes are close to linearly separable after gene pruning, so the FSWOR ensemble's diversity may matter less than the projection choice.","On datasets where the signal is spread across many weak features rather than concentrated in a few strong genes, random projections might overtake LDA; comparing the two on such data would test the generality of the claim."],"forward_implications":["On the GSE50161 dataset, the proposed FSWOR+LDA+Naïve Bayes setup reaches 96 percent cross-validation accuracy, a 9.09 percent relative gain over the prior 88 percent benchmark.","LDA is the decisive projection: Naïve Bayes with LDA scores 0.938, versus 0.892 with PCA, 0.815 with GRP, and 0.835 with SRP.","SVM is the most consistent classifier across projections, with accuracy 0.873-0.915, while decision trees degrade sharply under random projections.","The Kendall test cuts the feature space from 54,675 to 20,890 genes, reducing dimensionality before any model is trained."],"supporting_citations":[{"why":"Supplies the Cumida brain cancer dataset and the 0.88 benchmark accuracy that the proposed pipeline claims to exceed by 9.09%.","marker":"[1]"},{"why":"Cited as the source of the random feature selection without replacement idea that creates ensemble diversity.","marker":"[20]"},{"why":"Random projection enhancement method that motivates using projections to improve model performance in high dimensions.","marker":"[21]"},{"why":"Heterogeneous graph random projection method cited to justify the efficiency of random projection-based dimensionality reduction.","marker":"[22]"},{"why":"Probability framework for sequential random projection, cited as theoretical motivation for random projection's distance-preserving behavior.","marker":"[24]"}],"fun_headline_variants":["FSWOR plus LDA pushes brain-cancer classifier to 96%","Kendall-filtered genes and LDA projection reach 96% on GSE50161","No-replacement feature sampling with LDA beats 88% baseline at 96%","96% accuracy: LDA beats PCA and random projections on brain cancer","Exclusive feature selection and LDA outperform prior brain-cancer benchmark"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The Kendall test and the feature-sampling step are applied to the full dataset before the cross-validation folds are split, so the reported accuracy assumes that using all samples for feature selection does not leak information into the held-out folds.","fun_headline_variants_meta":{"raw":{"variants":["FSWOR plus LDA pushes brain-cancer classifier to 96%","Kendall-filtered genes and LDA projection reach 96% on GSE50161","No-replacement feature sampling with LDA beats 88% baseline at 96%","96% accuracy: LDA beats PCA and random projections on brain cancer","Exclusive feature selection and LDA outperform prior brain-cancer benchmark"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001012,"raw_usage":{"total_tokens":4264,"prompt_tokens":925,"completion_tokens":3339,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":541,"completion_tokens_details":{"reasoning_tokens":3238}},"tokens_in":541,"tokens_out":3339,"duration_ms":23832,"temperature":1.0,"reasoning_tokens":3238,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:56:58.919185+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the pipeline with the Kendall test and FSWOR sampling performed only inside each training fold, and check whether the held-out test accuracy stays at 96 percent; a material drop would show the reported gain over the 88 percent baseline comes from selection leakage rather than the method itself.","supporting_citations":[{"cited_title":"These alterations disrupt normal gene expression patterns [3], affecting genes that regulate cell growth, survival, and invasion while suppressing growth-inhibiting genes","cited_arxiv_id":null,"evidence_quote":"Supplies the Cumida brain cancer dataset and the 0.88 benchmark accuracy that the proposed pipeline claims to exceed by 9.09%."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Cited as the source of the random feature selection without replacement idea that creates ensemble diversity."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Random projection enhancement method that motivates using projections to improve model performance in high dimensions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Heterogeneous graph random projection method cited to justify the efficiency of random projection-based dimensionality reduction."},{"cited_title":"A., Gaitsch, H., Wicks, E","cited_arxiv_id":null,"evidence_quote":"Probability framework for sequential random projection, cited as theoretical motivation for random projection's distance-preserving behavior."}],"review_version":1}