CatNet: Controlling the False Discovery Rate in LSTM with SHAP Feature Importance and Gaussian Mirrors

Jiaan Han; Junxiao Chen; Yanzhe Fu

arxiv: 2411.16666 · v4 · submitted 2024-11-25 · stat.ML · cs.AI· cs.LG· q-fin.ST

CatNet: Controlling the False Discovery Rate in LSTM with SHAP Feature Importance and Gaussian Mirrors

Jiaan Han , Junxiao Chen , Yanzhe Fu This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-05-23 08:27 UTCgrok-4.3open to challenge →

classification stat.ML cs.AIcs.LGq-fin.ST

keywords false discovery rateLSTMSHAP valuesfeature selectionGaussian mirrortime serieskernel independence

0 comments

The pith

CatNet controls the false discovery rate in LSTM feature selection using SHAP derivatives and Gaussian mirrors.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents CatNet as a method to select significant features in LSTM models while controlling the false discovery rate. It quantifies feature importance via the derivative of SHAP values and applies the Gaussian Mirror algorithm with a vector mirror statistic. A kernel-based independence measure is introduced to handle correlations in time-series data. The approach shows robustness on simulated and real data, leading to reduced overfitting and better model interpretability. The method is positioned as extendable to other sequential deep learning models.

Core claim

CatNet employs the derivative of SHAP values to quantify the feature importance, and constructs a vector-formed mirror statistic for FDR control with the Gaussian Mirror algorithm. To avoid instability due to nonlinear or temporal correlations among features, a new kernel-based independence measure is proposed. CatNet performs robustly on different model settings with both simulated and real-world data.

What carries the argument

The vector-formed mirror statistic constructed from SHAP value derivatives for use in the Gaussian Mirror algorithm to control FDR.

If this is right

The method reduces overfitting in LSTM applications.
It improves the interpretability of LSTM models.
It works robustly across various model settings and datasets.
The framework can be extended to other time-series or sequential deep learning models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If the vector mirror statistic succeeds here, the same SHAP derivative approach may integrate into FDR control for other recurrent networks.
The kernel independence measure could be checked on non-sequential data to see if it generalizes beyond time series.
Extension to other models would require verifying that the mirror statistic remains stable when the base learner changes.

Load-bearing premise

The derivative of SHAP values combined with the kernel-based independence measure reliably quantifies feature importance without instability from nonlinear or temporal correlations.

What would settle it

A dataset with strong temporal correlations where the reported FDR exceeds the target level after applying CatNet would show the method does not control false discoveries as claimed.

read the original abstract

We introduce CatNet, an algorithm that effectively controls False Discovery Rate (FDR) and selects significant features in LSTM. CatNet employs the derivative of SHAP values to quantify the feature importance, and constructs a vector-formed mirror statistic for FDR control with the Gaussian Mirror algorithm. To avoid instability due to nonlinear or temporal correlations among features, we also propose a new kernel-based independence measure. CatNet performs robustly on different model settings with both simulated and real-world data, which reduces overfitting and improves interpretability of the model. Our framework that introduces SHAP for feature importance in FDR control algorithms and improves Gaussian Mirror can be naturally extended to other time-series or sequential deep learning models.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

CatNet claims FDR control for LSTM features via SHAP derivatives plus a new kernel measure inside Gaussian Mirrors, but the abstract gives neither the mirror statistic construction nor any validation that the FDR guarantee holds.

read the letter

The main takeaway is that this paper introduces CatNet as a way to do FDR-controlled feature selection inside LSTMs by taking derivatives of SHAP values for importance, adding a kernel-based independence measure to handle correlations, and feeding the result into a vector mirror statistic under the Gaussian Mirror procedure. That combination is presented as new and as reducing overfitting on both simulated and real data. The abstract frames it as extensible to other sequential models. That is the actual content on offer. It is a targeted extension of two existing lines of work rather than a broad new framework. If the pieces fit together as described, it could be a useful incremental tool for people already using SHAP on time-series networks. The paper does at least name the practical problem of temporal and nonlinear dependence that standard mirror statistics can run into. Beyond that, there is little to credit because the abstract supplies no equations, no explicit definition of the kernel measure, and no simulation numbers or theorem. The central claim therefore rests on unshown steps: that the SHAP derivative yields a stable score, that the kernel adjustment produces a mirror statistic with the required sign-flip symmetry under the null, and that the whole procedure controls FDR at the nominal level. Without those details the robustness statement cannot be checked. The same gap applies to the claim of reduced overfitting. This work is aimed at the narrow group already working on FDR methods for deep sequential models. A reader outside that group will not find enough to adopt or build on. For a referee, the paper would need the missing constructions and at least one clear simulation showing FDR control before it is worth the time. As presented, it does not reach that threshold.

Referee Report

2 major / 0 minor

Summary. The manuscript introduces CatNet, an algorithm for FDR-controlled feature selection in LSTM networks. It quantifies feature importance via the derivative of SHAP values, introduces a kernel-based independence measure to mitigate effects of nonlinear or temporal correlations, constructs a vector-formed mirror statistic, and applies the Gaussian Mirror procedure to control FDR. The authors claim the method is robust across model settings on both simulated and real-world data, reduces overfitting, and improves interpretability, with natural extension to other sequential deep-learning models.

Significance. If the central claims hold, the work would supply a concrete way to import SHAP-based importance into mirror-statistic FDR control for recurrent networks, addressing a practical need in time-series feature selection. The combination of SHAP derivatives with a kernel independence measure and vector mirrors is novel and could be reusable; however, the significance cannot be assessed until the explicit construction, the properties of the kernel measure, and the FDR guarantee are demonstrated.

major comments (2)

Abstract: The central claim that CatNet 'effectively controls' FDR rests on the derivative of SHAP values plus the kernel-based independence measure producing a valid mirror statistic whose symmetry or sign-flip properties allow the Gaussian Mirror algorithm to deliver FDR control at the target level. No explicit definition of the mirror statistic, no statement of the kernel measure, and no theorem or simulation establishing null FDR control are supplied, so the guarantee does not follow from the stated ingredients.
Abstract: The assumption that the proposed kernel-based independence measure 'avoids instability due to nonlinear or temporal correlations' is load-bearing for the validity of the mirror statistic, yet the measure is introduced without definition, without proof of its independence properties, and without any reported check that it preserves the sign-flip symmetry required by Gaussian Mirrors.

Simulated Author's Rebuttal

2 responses · 2 unresolved

We thank the referee for the careful review and for identifying points where the abstract could better reflect the manuscript content. We address each major comment below, clarifying what is provided in the full text and indicating revisions.

read point-by-point responses

Referee: Abstract: The central claim that CatNet 'effectively controls' FDR rests on the derivative of SHAP values plus the kernel-based independence measure producing a valid mirror statistic whose symmetry or sign-flip properties allow the Gaussian Mirror algorithm to deliver FDR control at the target level. No explicit definition of the mirror statistic, no statement of the kernel measure, and no theorem or simulation establishing null FDR control are supplied, so the guarantee does not follow from the stated ingredients.

Authors: The manuscript supplies the explicit construction of the vector-formed mirror statistic in Section 3.2 and the kernel-based independence measure in Section 3.3. Section 4 reports simulation results across multiple settings that demonstrate FDR control at the target level. We agree the abstract is overly concise and does not reference these elements or the empirical nature of the validation. We will revise the abstract to note the constructions and the simulation-based evidence. A formal theorem establishing the FDR guarantee is not present; the control is justified by combining the Gaussian Mirror procedure with the kernel measure's empirical performance. revision: yes
Referee: Abstract: The assumption that the proposed kernel-based independence measure 'avoids instability due to nonlinear or temporal correlations' is load-bearing for the validity of the mirror statistic, yet the measure is introduced without definition, without proof of its independence properties, and without any reported check that it preserves the sign-flip symmetry required by Gaussian Mirrors.

Authors: The kernel-based independence measure is defined in Section 3.3 of the manuscript. Simulations in Section 4 provide empirical evidence that FDR control is maintained, which serves as an indirect check on the measure's effectiveness under nonlinear and temporal dependence. We concur that the abstract does not define the measure or discuss symmetry preservation. We will revise the abstract to indicate that the measure is introduced to address such correlations and that its utility is supported by the reported simulations. A formal proof of the independence properties or explicit verification of sign-flip symmetry is not included. revision: yes

standing simulated objections not resolved

Absence of a formal theorem establishing FDR control under the proposed kernel measure
Absence of a proof of the kernel measure's independence properties and preservation of sign-flip symmetry

Circularity Check

0 steps flagged

No significant circularity; derivation builds on external methods without reduction to self-defined inputs

full rationale

The paper's central construction combines the derivative of SHAP values for importance scoring, a proposed kernel-based independence measure to handle correlations, and the Gaussian Mirror algorithm for FDR control via a vector mirror statistic. These components reference established external techniques (SHAP and Gaussian Mirror) rather than deriving from self-citations or fitted parameters that are then renamed as predictions. No self-definitional steps, fitted-input predictions, or load-bearing self-citation chains are evident from the abstract or described framework. The approach is presented as an extension applicable to other sequential models, with performance validated on simulated and real data, keeping the derivation self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 1 invented entities

Central claim rests on domain assumptions about SHAP derivatives providing valid importance scores and the adapted Gaussian Mirror controlling FDR; the kernel-based independence measure is an invented component without external validation shown in abstract. No explicit free parameters listed.

axioms (2)

domain assumption Derivative of SHAP values quantifies feature importance in LSTM models
Invoked to measure importance for the mirror statistic.
domain assumption Gaussian Mirror algorithm extends to vector-formed statistics for FDR control in this setting
Core mechanism for FDR control.

invented entities (1)

kernel-based independence measure no independent evidence
purpose: Avoid instability due to nonlinear or temporal correlations among features
Proposed to support the main method when standard assumptions fail.

pith-pipeline@v0.9.0 · 5654 in / 1325 out tokens · 35143 ms · 2026-05-23T08:27:39.976134+00:00 · methodology

CatNet: Controlling the False Discovery Rate in LSTM with SHAP Feature Importance and Gaussian Mirrors

Core claim

What carries the argument

If this is right

Where Pith is reading between the lines

Load-bearing premise

What would settle it

discussion (0)