{"id":"68831f40-9849-4ef4-b796-5d34191c817c","arxiv_id":"2509.09264","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"iRPF adaptively combines multiple covariance-based detectors and automatic knee-based thresholds, claiming superior artifact rejection over RPF, RP, Isolation Forest, and Autoreject on two public EEG databases.","lead":"This paper introduces iRPF, an automatic EEG artifact rejection method that adaptively thresholds Riemannian-distance and other covariance-based detectors to discard contaminated epochs. It reports better recall, specificity, precision, and F1 than four existing automated cleaning methods on two public EEG databases, at millisecond speeds per epoch.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Manual relabeling of EYEAR ground truth threatens the validity of iRPF's headline performance gains","rationale":"The reader identified the FRMS symmetry assumption as the weakest link, but that assumption affects generalization to new datasets; the empirical results on the two evaluated databases are already consistent with it. The more load-bearing concern is the integrity of the ground truth in EYEAR: the authors changed labels after inspecting the data, and the effect of this change on the comparison is unknown. This directly threatens the validity of the reported superiority on the primary database, which is the basis of the abstract's claims. Since the reader's verdict is already CONDITIONAL, my concern does not alter the verdict, but it identifies a different and more pressing issue that should be resolved before accepting the headline results.","tokens_in":22608,"tokens_out":6127,"duration_ms":72769,"concrete_test":"Obtain the original EYEAR labels from the public repository. Have at least two independent EEG experts annotate all 4-second epochs for artifact type (blink/HEM/VEM/clean) without knowledge of iRPF or the authors' relabeling. Compute inter-rater agreement (Cohen's kappa) between the authors' final labels and each expert's labels. Re-run the full comparison (iRPF, RPF, RP, IF, AR) using only the original labels, and separately using only epochs where all annotators agree. If iRPF is not consistently top-ranked under independent labels, the headline claim is a labeling artifact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that iRPF outperforms all four competitors relies heavily on the EYEAR database, where iRPF reaches F1=0.96. Section 2.6.1 states that the authors 'manually reviewed the data and relabeled the affected epochs' after discovering that some 'rest' epochs contained blinks. This is an unvalidated modification of the test labels by the method's developers, with no reported inter-rater reliability or comparison to the original labels. Because iRPF is specifically designed to detect eye blinks, even subtle label bias could inflate its recall, precision, and F1. The ablation study (Table 7) shows that the outlier rejection component alone contributes 0.18 F1 on EYEAR with EOG, so the reported margin over competitors is sensitive to label details. If the relabeling is biased, the superiority claim cannot be trusted. The FRMS symmetry assumption is a generalizability concern for other datasets; the manual relabeling is a validity concern for the results as reported.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes an improved Riemannian Potato Field (iRPF) for automatic EEG artifact rejection. The method extends the existing RPF framework with four main contributions: (1) an adaptive FRMS-based outlier rejection step preceding barycenter estimation, (2) a Kneedle-based adaptive robust barycenter estimation that removes the fixed threshold, (3) additional Euclidean and diagonal-Euclidean distance features in the potato field, and (4) additional p-value combination functions and a meta-combination strategy for computing the signal quality index, together with automatic knee-based thresholding of the SQI. The method is evaluated on two publicly available databases, EYEAR (with and without EOG channels) and TUAR, against RPF, RP, Isolation Forest, and Autoreject. The authors report consistent improvements in recall, specificity, precision, and F1-score, supported by permutation-based multiple-comparison tests and Cohen's d effect sizes, and report an execution time below 8 ms per epoch.","tokens_in":22881,"tokens_out":6026,"duration_ms":62804,"significance":"If the reported results hold, iRPF would be a practically valuable contribution: it is a fully automatic, fast, and publicly implemented method with a plausible mechanism for improving robustness of Riemannian-geometry-based artifact rejection. The statistical analysis is a clear strength: the use of a 200K-permutation t-max step-down procedure to control family-wise error rate, the reporting of effect sizes, and the inclusion of runtime comparisons are all commendable. The central claim of superiority over four baselines is, however, conditional on the integrity of the EYEAR labels, the fairness of the baseline configurations, and the validity of the FRMS symmetry assumption. These issues are addressable and do not, on the present evidence, require rejection, but they require substantive revision before the claim can be accepted.","major_comments":[{"comment":"The authors state that after discovering that some epochs labeled 'rest' were contaminated by blinks, they 'manually reviewed the data and relabeled the affected epochs.' This is a developer-driven modification of the test labels, and the manuscript reports no details about the relabeling procedure, no inter-rater reliability, and no comparison with the original labels. Because iRPF is specifically designed to detect eye blinks, even subtle label bias could inflate its recall, precision, and F1. This is load-bearing: the strongest reported result (F1 = 0.96 on EYEAR with EOG) and the large ablation contribution of the outlier-rejection component (Table 7) come from this database. Please report the number and nature of relabeled epochs, provide results using the original database labels, and/or perform a sensitivity analysis excluding or conservatively re-labeling the affected epochs with","section":"Section 2.6.1 (EYEAR database)"},{"comment":"The RPF baseline threshold is set to 0.5 because it 'yielded the highest averaged F1-score.' If this selection was made on the same test data used for the comparison, the RPF baseline is effectively oracle-tuned, while iRPF must discover its threshold automatically. The ablation study uses the same practice: when automatic thresholding is disabled, threshold 0.01 is adopted because it 'yielded the best results.' This makes the comparison between automatic and manual thresholding difficult to interpret. Please use nested cross-validation or report performance across a grid of thresholds with uncertainty, and clarify explicitly whether the 0.5 and 0.01 choices were made on the evaluation folds.","section":"Section 3.1 and Table 7 (baseline threshold selection)"},{"comment":"The method's adaptive outlier-rejection threshold relies on the assumption that the FRMS distribution of normal EEG is symmetric around the median-based mean, so that the threshold t_hrej = mu_frms + u_lim(mu_frms - l_lim) removes extreme outliers without discarding legitimate high-amplitude activity. The ablation study (Table 7) shows that this component contributes 0.18 F1 on EYEAR with EOG, so the assumption is load-bearing. Two issues arise. First, u_lim is a free parameter described as 'adjustable based on the characteristics of the database,' which conflicts with the 'fully automatic' claim in the abstract. Second, no empirical validation of the symmetry assumption is provided, and clinical EEG with pathological high-amplitude activity or different amplifier settings may violate it. Please report the empirical FRMS distributions on both databases (e.g., skewness, quantiles, the loc","section":"Section 2.4.1 (automatic outlier rejection and FRMS symmetry)"},{"comment":"The authors state that to ensure a fair comparison they applied the proposed automatic outlier rejection method before using RPF and RP. As a result, the evaluated RPF and RP are not the original published methods, but hybrids that include a key iRPF component. This does not necessarily bias the comparison against iRPF, but it means the reported comparison does not isolate the contribution of iRPF over the original methods, and the abstract's phrase 'compared to ... Riemannian Potato, and Riemannian Potato Field' is potentially misleading. Please also evaluate the original, unmodified RPF and RP, or clearly label the modified baselines as 'RPF + iRPF outlier rejection' and 'RP + iRPF outlier rejection'.","section":"Section 3.1 (modified baselines)"}],"minor_comments":[{"comment":"The sample covariance estimator is written as (1/(N-1)) X X^T, but with X in R^{N x T} the normalization should be by T-1, not N-1. If the implementation uses a standard covariance routine this is likely only a typo, but it should be corrected for reproducibility.","section":"Eq. (1)"},{"comment":"Typo: 'connivance matrices' should be 'covariance matrices.'","section":"Section 2.4.3"},{"comment":"The paper reports only Cohen's d values and figures for the main performance metrics; no table gives the per-method means and standard deviations of recall, specificity, precision, and F1 for each database. Please add such a table so that the claimed gains can be inspected numerically.","section":"General / Results"},{"comment":"In the description of EOG electrodes, the right-eye electrode labels appear inconsistent (EOGL1, EOGL2, EOGR3). Please verify the nomenclature against Table 2.","section":"Section 2.5"},{"comment":"The definition of the RPF SQI using p-values and the statement that epochs with p below a threshold are rejected should be made more explicit, since a threshold of 0.5 is much higher than the original RPF literature convention of 0.01 and can be confusing without a clear statement of the rejection direction.","section":"Section 2.3 / threshold direction"}],"recommendation":"major_revision","confidential_remarks":"The statistical machinery is a genuine strength, and the method is potentially useful. The main risk is not circularity but label integrity in EYEAR and the fairness of the baseline tuning. I would ask the editor to require the original-label sensitivity analysis and a clarification of the threshold-selection procedure before acceptance. The overlap with the authors' own RPF work [65] is a self-citation concern but is not inappropriate given the direct methodological lineage."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The iRPF is a sensible, well-built extension of Riemannian Potato Field: adaptive thresholding via Kneedle, extra Euclidean and diagonal-Euclidean distances, meta-combined p-values, and a careful ablation study. The code is public, and the evaluation uses permutation-based FWER control and effect sizes. I believe the method is real and likely useful, especially for large-scale EEG studies where manual cleaning does not scale.\n\nThat said, the reader's conditional verdict is the right one. The strongest concern is the manual relabeling of EYEAR (Section 2.6.1): the authors discovered that some 'rest' epochs actually contained blinks and relabeled them. With no inter-rater reliability check or comparison to the original labels, the headline F1=0.96 on EYEAR is vulnerable to subtle bias, especially since iRPF is specifically designed to catch blinks. The stress-test note lands. I also agree that the RPF baseline threshold was tuned on the same test data to maximize F1 (Section 3.1), which inflates the comparison. The TUAR subset selection (196 of 310 recordings) is another soft spot, though the authors do acknowledge the need for sufficient clean data. The \"fully automated\" claim also overreaches: the potato field still has to be designed by an expert for each headset.\n\nThe FRMS symmetry assumption is a real generalizability caveat for pathological EEG, but it is not a fatal flaw; the ablation shows the outlier rejection component matters a lot, so the method's edge on clinical data remains to be demonstrated. The paper does not report raw metric tables anywhere, only Cohen's d and figures for some comparisons, which makes independent verification harder than it should be.\n\nOverall: this is a solid methods paper with a plausible, probably directionally correct improvement. For peer review, yes, send it to a serious referee. The authors should be asked to disclose the relabeling, make the relabeled labels available or validate them against independent raters, and either retune RPF on a separate set or report the tuning procedure explicitly. If those fixes land, the method deserves recognition.","headline":"A well-engineered extension of RPF with serious evaluation, but the manual relabeling of EYEAR ground truth and test-data-tuned baseline keep the headline claims from being fully trustworthy.","tokens_in":23358,"tokens_out":1673,"would_cite":false,"duration_ms":21421,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"iRPF, a fully automatic EEG artifact rejection method, outperforms four rivals with no manual threshold tuning.","keywords":["EEG artifact rejection","Riemannian geometry","covariance matrices","adaptive thresholding","signal quality index","p-value combination","outlier rejection","brain-computer interfaces"],"falsifier":"Run iRPF on an EEG dataset with expert-labeled clean epochs containing high-amplitude but non-artifactual activity (e.g., generalized epileptiform discharges, high-amplitude occipital alpha, or drowsy slow waves) and count how many labeled-clean epochs are rejected. If a substantial fraction (e.g., >10%) of clean epochs are rejected in the high-amplitude subjects, the symmetry assumption fails and the automatic outlier rejection is not universally safe.","tokens_in":22564,"feed_emoji":"🧠","tokens_out":3949,"duration_ms":41999,"temperature":0.7,"pith_summary":"The paper claims that artifact rejection in EEG can be made fully automatic and adaptive, eliminating the manual threshold choices that handicap existing methods. It introduces iRPF, which builds on the Riemannian Potato Field idea by combining many small covariance-based detectors targeting specific artifacts, then adds four improvements: automatic outlier rejection, adaptive barycenter estimation, extra distance metrics, and a meta-combination of p-values. On two public EEG databases comprising 226 recordings, iRPF reports higher recall, specificity, precision, and F1 than Riemannian Potato, Riemannian Potato Field, Isolation Forest, and Autoreject, with statistical significance and large effect sizes in most comparisons, while running in under 8 ms per epoch. If correct, this gives researchers a fast, data-driven preprocessing step that does not distort the signal and may scale to large datasets and real-time use.","feed_headline":"iRPF beats four rival EEG artifact filters, no manual tuning","feed_subtitle":"Adaptive thresholds and covariance-based detectors reject artifacts in under 8 ms per epoch.","key_machinery":"The central object is the \"potato field\": a collection of low-dimensional Riemannian potatoes, each a detector built on the covariance matrix of a specific subset of EEG channels in a specific frequency band, whose distances to a robust barycenter become z-scores, then p-values, merged into a per-epoch Signal Quality Index. Carrying the argument are two adaptive mechanisms: an outlier-rejection threshold defined from the field root mean square (FRMS) of the data, assuming clean EEG's FRMS is symmetric about its median-based mean, and a knee-detection (Kneedle) algorithm that sets the acceptance threshold from the sorted SQI values, making the whole pipeline free of manually chosen thresholds","core_discovery":"The central claim is that iRPF is a fully automatic and adaptive EEG artifact rejection method that outperforms established alternatives. It replaces fixed thresholds with a knee-detection algorithm that reads the data's own distribution, adds a field-root-mean-square-based outlier rejection stage so the geometric mean of covariance matrices is not pulled toward extreme artifacts, and enriches the Riemannian Potato Field with Euclidean and diagonal-Euclidean distances to catch vertical eye movements and myogenic artifacts that the Riemannian distance alone tends to miss. Combining these distances' p-values through multiple combination functions in a meta-combination yields a Signal Quality I","pith_inferences":["If the FRMS symmetry assumption holds broadly, the same adaptive thresholding recipe could transfer to other biosignals whose epochs are naturally represented as covariance matrices, such as MEG or EMG.","A testable extension: on clinical recordings with high-amplitude pathological oscillations (e.g., epileptiform discharges), iRPF's outlier rejection may reject legitimate brain activity because the clean FRMS distribution is right-skewed; measuring false-rejection rates on such data would expose the boundary of the symmetry assumption.","The meta-combination strategy is a general recipe — combining p-value functions sensitive to different parts of the distribution could improve signal-quality indices beyond EEG artifact rejection.","Because potato-field design still requires knowledge of electrode layout and artifact physiology, a natural next step left implicit by the paper is to auto-generate potatoes from channel coordinates alone; the paper's own robustness results suggest performance varies little with reasonable field definitions."],"forward_implications":["EEG preprocessing for brain-computer interfaces or clinical studies could run iRPF without per-dataset threshold tuning and still match or exceed tuned competitors.","By adding Euclidean and diagonal-Euclidean distances, iRPF is claimed to detect vertical eye movements and muscle artifacts that the affine-invariant Riemannian distance alone tends to miss.","The reported speed (under 8 ms per epoch on a laptop) supports large-scale offline processing and suggests real-time feasibility after further development.","The ablation study shows automatic outlier rejection carries much of the gain: dropping it reduces F1 from 0.96 to 0.78 on the eye-artifact database with EOG channels, indicating the advantage rests on the full combination of components.","Because iRPF rejects rather than corrects epochs, it preserves the original signal, which is a stated advantage for downstream cognitive and clinical analyses."],"fun_headline_variants":["EEG artifact rejection: iRPF outruns 4 rivals, no tuning","Fully automatic EEG cleanup: iRPF beats 4 filters in speed and accuracy","Fast automatic EEG artifact rejection: iRPF wins on all metrics","iRPF: adaptive EEG artifact filter outperforms four methods"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The method assumes that the field root mean square (FRMS) of normal EEG is symmetrically distributed around its median-based mean, so that the adaptive threshold (mean plus the gap to the lower limit) separates extreme artifacts from legitimate high-amplitude brain activity.","fun_headline_variants_meta":{"raw":{"variants":["EEG artifact rejection: iRPF outruns 4 rivals, no tuning","Fully automatic EEG cleanup: iRPF beats 4 filters in speed and accuracy","Fast automatic EEG artifact rejection: iRPF wins on all metrics","iRPF: adaptive EEG artifact filter outperforms four methods"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000562,"raw_usage":{"total_tokens":2543,"prompt_tokens":820,"completion_tokens":1723,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":1641}},"tokens_in":564,"tokens_out":1723,"duration_ms":12262,"temperature":1.0,"reasoning_tokens":1641,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T19:23:49.263277+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run iRPF on an EEG dataset with expert-labeled clean epochs containing high-amplitude but non-artifactual activity (e.g., generalized epileptiform discharges, high-amplitude occipital alpha, or drowsy slow waves) and count how many labeled-clean epochs are rejected. If a substantial fraction (e.g., >10%) of clean epochs are rejected in the high-amplitude subjects, the symmetry assumption fails and the automatic outlier rejection is not universally safe.","supporting_citations":[],"review_version":1}