Pith. sign in

REVIEW 4 major objections 6 minor 24 references

Consistency of Feature Attribution in Deep Learning Architectures for Multi-Omics

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper shows that SHAP-based feature rankings in multi-view deep learning models for multi-omics data are unstable across architecture choices and random weight initializations, so a single attribution run should not be trusted as a…

desk verdict A careful empirical caution about SHAP instability in small-sample multi-view nets, but the broad warning outruns the evidence. read the letter →

arxiv 2507.22877 v1 pith:37BH234H submitted 2025-07-30 stat.ML cs.LG

Daniel Claborne , Javier Flores , Samantha Erwin , Luke Durell , Rachel Richardson , Ruby Fore , Lisa Bramer This is my paper · ORCID
classification stat.MLcs.LG
keywords SHAPfeatureattributionmulti-omicsmulti-viewdeeplearningrankstabilityweightedKendall'staubiomarkerdiscoverymodelinterpretability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to test whether SHAP, a popular method for explaining which input features drive a deep model's predictions, gives stable answers when the model is a multi-view network trained on multi-omics data. It finds that the ranked list of important biomolecules changes with the network's layer sizes, with the way views are combined, and even across training runs that differ only in random weight initialization and dropout. The conclusion is a caution: a single SHAP-ranked feature list from such a model is not a reliable basis for naming driver biomolecules. The paper also offers a simple remedy: train several times and inspect the distribution of ranks rather than trusting one ranking, and it shows that the instability can be hidden because downstream classifiers still do well on the selected features.

What carries the argument

The argument is carried by two instruments: the SHAP attribution score, averaged in absolute value over samples, which converts a trained network into a ranked list of biomolecules; and the weighted Kendall's tau, a rank-correlation measure that gives more weight to matches at the top, used to quantify how much rankings degrade when noise features are added or architecture changes. A third instrument is the per-feature distribution of ranks over repeated training runs, which reveals that supposedly top-ranked biomolecules can fall to middle ranks under a different random initialization. These diagnostics jointly show that attribution is not an intrinsic property of the biomolecule but an artifact of the particular fitted network.

What would settle it

Train a multi-view deep network on synthetic multi-omics data with a planted, known sparse set of driver biomolecules, rerun SHAP over many random initializations and across several architecture families, and check whether the top-ranked features consistently recover the planted drivers. If the planted drivers stay at the top across all runs and architectures, the claimed sensitivity would be specific to the paper's two datasets rather than a general property of SHAP on multi-view networks.

Watch

Extended reading notes

Core claim

SHAP scores computed on feed-forward multi-view networks do not yield consistent feature rankings. Adding irrelevant noise features to one omics view degrades the weighted Kendall's tau between original and augmented rankings, and the pattern depends on the hidden layer sizes of the other view. Even for the highest-ranked biomolecules, ranks fluctuate widely across training runs started with different random weights; a feature that occasionally spikes to high importance can mostly sit near the bottom. The same instability appears in a more complex multi-view baseline architecture used as a validation check. Despite this, random-forest classifiers fit on the top 10% of SHAP-ranked features retain much of the full-feature AUC, so predictive performance alone does not reveal attribution instability.

Load-bearing premise

The paper's warning about SHAP transfer depends on its simplified feed-forward multi-view network being representative of the multi-view deep learning models that researchers actually apply to multi-omics data; if those models behave differently, the measured instability may not generalize.

Editorial extensions

If this is right

  • A single SHAP run on one trained multi-view network is not enough to nominate biomarkers; the same data and task can produce a different top list if the weights are re-initialized.
  • Reports of SHAP-identified genes or metabolites should include the spread of ranks across multiple training runs, since a feature can be top-ranked in one run and below median in another.
  • Increasing the number of uninformative features in one omics view can push true drivers of that view down in the global ranking, so cross-view importance comparisons are sensitive to relative view size.
  • Downstream performance checks, such as fitting a random forest or clustering on the top-ranked features, can look fine even when the underlying rankings are unstable, so good performance is not evidence that the attribution is consistent.
  • Matching or dynamically adjusting layer sizes does not reliably stabilize SHAP rankings; the effect reverses depending on the base size of the other view's network, so there is no simple capacity-based fix.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the observed instability scales with model complexity, then generative multi-view architectures such as variational autoencoders likely show even larger SHAP variance than this feed-forward template, since their training adds further random components such as variational sampling.
  • A natural testable extension is an ensemble consensus rule: average SHAP ranks across many seeds and architectures and test whether the consensus top-k is more reproducible and biologically plausible than any single run; the paper stops short of proposing this as a remedy.
  • The noise-inflation experiment implies a practical diagnostic for applied studies: permute or knock out one view's features and measure the rank shift of known positive controls before trusting cross-view importance comparisons.
  • Because both datasets are small and the simplified architecture avoids variational losses, the magnitude of the observed instability may be a lower bound for real-world applications with more complex models and larger, noisier omics panels.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper investigates the consistency of SHAP-based feature attributions across architectures and random initializations in a simplified multi-view feed-forward network applied to two small multi-omics datasets (ICL102, n=120 and ICL104, n=60). Experiments add noise features to one view, vary hidden layer sizes, repeat training with different random seeds, and evaluate downstream random-forest classification and clustering performance using top-ranked features. The authors report that SHAP rankings degrade with added noise, vary across architectures, and vary substantially across training runs. They conclude that SHAP rankings are not reliable in this setting and recommend caution, proposing visual inspection of rank distributions across runs as an alternative diagnostic. The raw data are available from public repositories, but processed data are only available on request and no code is provided.

Significance. If the conclusion is limited to the tested regime, the paper provides a useful empirical caution and a concrete demonstration of attribution instability in small-sample, overparameterized multi-view networks. The design includes sensible independent checks: downstream random-forest and clustering tasks are used to assess the quality of selected features rather than relying solely on internal attribution rankings, and weighted Kendall's tau focuses on top-rank agreement. However, the contribution is currently weakened by the narrow architectural and dataset scope, the unspecified alternative method promised in the abstract, and several internal inconsistencies. The study does not yet support a general claim about SHAP stability across multi-view deep learning for multi-omics.

major comments (4)
  1. [Abstract / Methods: Model Architecture / Data] The abstract and conclusion state that the results suggest caution when using attribution methods on multi-view deep learning models applied to multi-omics data, but all core experiments use a single simplified feed-forward architecture (Methods: Model Architecture) on two small datasets (ICL104 with n=60 and ICL102 with n=120) with thousands of input features, full-batch Adam, and no hyperparameter tuning. The authors explicitly describe the architecture as a simplification of Lee et al. and deliberately avoid stochastic encoders and variational losses. The one supplementary repetition with the Lee et al. model does not control for sample size, regularization, or stochasticity. The observed instability may be specific to heavily overparameterized deterministic networks on small samples rather than a general property of multi-view deep learning on multi-omics. To support the stated broad conclusion, the authors should either restrict the claim to the tested architecture regime or add experiments with regularized and/or stochastic architectures and, if possible, a larger cohort.
  2. [Abstract / Discussion / Conclusion] The abstract promises 'an alternative, simple method to assess the robustness of identification of important biomolecules,' but the only proposal in the manuscript is visual inspection of rank distributions across training runs (Discussion, Figure 3). No algorithm, threshold, or diagnostic statistic is defined, the method is not compared with existing stability measures (e.g., rank stability, Jaccard index, rank-biased overlap), and it is not validated on data with known ground-truth importance. Since the claimed contribution includes this method, it needs to be specified precisely and evaluated quantitatively.
  3. [Results: Variation in Features, Figure 3] Figure 3 and its caption state that ranks are computed 'out of 351 possible ranks,' but the dataset dimensions reported in the Data section (ICL102: 80 metabolites, 496 lipids, 3722 proteins; ICL104: 138, 491, 4896) and the noise-augmentation experiments (up to 4000 added features) do not produce 351 features in any described configuration. If ranks are computed on a subset (e.g., only annotated metabolites) or after filtering, that must be stated. As written, the rank denominator is unexplained and undermines the quantitative interpretation of the figure.
  4. [Results: Binary and Multi-class Viral Dataset / Discussion] There is a direct contradiction between the Results section, which states 'We do however see a decrease in V-measure score as we reduce the proportion of top features selected,' and the Discussion, which states 'we saw no significant drop as measured by V-measure.' Because the paper uses downstream clustering performance as evidence that selected features retain discriminative quality, this inconsistency affects the interpretation of a central result and must be resolved.
minor comments (6)
  1. [Throughout] The manuscript contains numerous typographical errors, including 'ePect' for 'effect', 'diPerent' for 'different', 'DiPicult' for 'difficult', and others. A careful proofreading pass is needed.
  2. [Data Availability] Processed data are available only on request and no code is provided. For an empirical study whose claims rest on specific preprocessing, feature sets, and training procedures, releasing code and processed data would substantially improve reproducibility.
  3. [Methods: Model Architecture] The text refers to 'Lee et. al.' without specifying whether this is reference [7] (Lee and van der Schaar, variational information bottleneck) or reference [15] (Lee et al., gene-centric convolutional encoders). The description of the architecture suggests [7], but the ambiguity should be removed.
  4. [Results: Performance of Features Subsets] The paper reports AUC for a multiclass dataset but does not specify whether this is one-vs-rest, macro-average, or another aggregation. The definition should be given in the Methods section.
  5. [Methods: Effect of Feature Compression] The weighted Kendall's tau values in Figure 2 are reported without confidence intervals or statistical tests. Given that only 10 training runs are used per condition, some measure of uncertainty would help assess whether the observed drop-offs are significant.
  6. [Discussion] The paper does not discuss the approximation error of DeepExplainer (the SHAP variant used). Since DeepExplainer is an approximation to Shapley values, its own variability could contribute to the observed instability; a brief comment or sensitivity analysis would strengthen the interpretation.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper makes an empirical observation about SHAP ranking variability, with no fitted parameter disguised as a prediction and no load-bearing self-citation chain.

full rationale

This is a purely empirical study. The central claim—that SHAP feature rankings vary across architectural choices and random initializations—is a measured result, not a derivation. The paper fits no parameters that are then relabeled as predictions; SHAP scores are computed from trained models and compared, and the downstream random-forest and clustering checks are independent benchmarks of the selected feature subsets rather than restatements of the SHAP rankings. The reference to Lee et al. is used to justify the simplified architecture via a performance comparison, and the authors explicitly repeat one main experiment with the Lee et al. model in the supplement to check generalizability; this is external validation, not circular reasoning. The reader's and skeptic's concerns about small sample sizes, the simplified feed-forward architecture, and possible overparameterization are substantive generalizability limitations, but they are not circularity. No equation is defined in terms of the result, no self-citation is load-bearing, and no known result is merely renamed. The paper honestly reports instability, poor clustering performance, and inconsistent effects of layer sizing, which further supports that the findings are empirical rather than constructed. Therefore the circularity score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim depends on the experimental design rather than on mathematical postulates. The listed axioms are the background assumptions about the SHAP approximation, the noise simulation, the rank metric, and the downstream validation metrics that the paper invokes without testing them independently.

free parameters (2)
  • Hidden layer sizes for marginal networks = 64, 128, 512 per layer
    Hand-chosen to represent common practice; the main instability results differ across these settings, so the conclusions depend on these design choices.
  • Combination layer sizes and training iterations = Varies per experiment; training stopped at validation plateau plus 20%
    Experimental design choices that affect SHAP values; not fitted to an external target.
assumptions (4)
  • domain assumption DeepExplainer's gradient-based approximations adequately approximate Shapley values for these networks.
    Invoked in the Introduction when equating SHAP with Shrikumar et al.'s method; if the approximation is poor, the observed instability might reflect approximation error rather than true attribution variability.
  • domain assumption Adding Gaussian noise features with matched mean and variance simulates uninformative input features.
    Used in the noise-augmentation experiments; real biologically irrelevant features may behave differently.
  • standard math Weighted Kendall's Tau is an appropriate metric for comparing feature rankings with ties.
    Cited from Vigna; used throughout the rank-stability analyses.
  • domain assumption Random forest and V-measure provide meaningful downstream validation of feature strength.
    Used to claim that feature subsets retain discriminative power; V-measure scores are very low, so clustering-based validation may be uninformative.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Consistency of Feature Attribution in Deep Learning Architectures for Multi-Omics." pith.science (2026). https://pith.science/paper/37BH234H

@misc{pith2026250722877,
  author       = {Pith},
  title        = {Pith review of: Consistency of Feature Attribution in Deep Learning Architectures for Multi-Omics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/37BH234H}},
  note         = {Machine review of arXiv:2507.22877}
}
read the original abstract

Machine and deep learning have grown in popularity and use in biological research over the last decade but still present challenges in interpretability of the fitted model. The development and use of metrics to determine features driving predictions and increase model interpretability continues to be an open area of research. We investigate the use of Shapley Additive Explanations (SHAP) on a multi-view deep learning model applied to multi-omics data for the purposes of identifying biomolecules of interest. Rankings of features via these attribution methods are compared across various architectures to evaluate consistency of the method. We perform multiple computational experiments to assess the robustness of SHAP and investigate modeling approaches and diagnostics to increase and measure the reliability of the identification of important features. Accuracy of a random-forest model fit on subsets of features selected as being most influential as well as clustering quality using only these features are used as a measure of effectiveness of the attribution method. Our findings indicate that the rankings of features resulting from SHAP are sensitive to the choice of architecture as well as different random initializations of weights, suggesting caution when using attribution methods on multi-view deep learning models applied to multi-omics data. We present an alternative, simple method to assess the robustness of identification of important biomolecules.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 19 canonical work pages

  1. [1]

    (a) Separate 'views' or `omics types to be input to separate marginal networks

    Figure 1: High level overview of our model architecture. (a) Separate 'views' or `omics types to be input to separate marginal networks. (b) Two hidden linear layers with ReLU activations per marginal network, each processing a single view. (c) Two possible combination schemes: The equal-dimension outputs of each marginal network are averaged feature wise...

    work page 2013

  2. [2]

    CustOmics: A versatile deep-learning based strategy for multi-omics integration,

    H. Benkirane, Y . Pradat, S. Michiels, and P .-H. Cournède, “CustOmics: A versatile deep-learning based strategy for multi-omics integration,” PLOS Comput. Biol., vol. 19, no. 3, p. e1010921, Mar. 2023, doi: 10.1371/journal.pcbi.1010921

    work page doi:10.1371/journal.pcbi.1010921 2023

  3. [3]

    A hierarchical integration deep flexible neural forest framework for cancer subtype classification by integrating multi-omics data,

    J. Xu, P . Wu, Y . Chen, Q. Meng, H. Dawood, and H. Dawood, “A hierarchical integration deep flexible neural forest framework for cancer subtype classification by integrating multi-omics data,” BMC Bioinformatics, vol. 20, no. 1, p. 527, Oct. 2019, doi: 10.1186/s12859-019-3116-7

    work page doi:10.1186/s12859-019-3116-7 2019

  4. [4]

    MOLI: multi-omics late integration with deep neural networks for drug response prediction,

    H. Sharifi-Noghabi, O. Zolotareva, C. C. Collins, and M. Ester, “MOLI: multi-omics late integration with deep neural networks for drug response prediction,” Bioinformatics, vol. 35, no. 14, pp. i501–i509, Jul. 2019, doi: 10.1093/bioinformatics/btz318

    work page doi:10.1093/bioinformatics/btz318 2019

  5. [5]

    Capturing the latent space of an Autoencoder for multi-omics integration and cancer subtyping,

    Madhumita and S. Paul, “Capturing the latent space of an Autoencoder for multi-omics integration and cancer subtyping,” Comput. Biol. Med., vol. 148, p. 105832, Sep. 2022, doi: 10.1016/j.compbiomed.2022.105832

    work page arXiv 2022

  6. [6]

    Deep Learning–Based Multi-Omics Integration Robustly Predicts Survival in Liver Cancer,

    K. Chaudhary, O. B. Poirion, L. Lu, and L. X. Garmire, “Deep Learning–Based Multi-Omics Integration Robustly Predicts Survival in Liver Cancer,” Clin. Cancer Res., vol. 24, no. 6, pp. 1248–1259, Mar. 2018, doi: 10.1158/1078-0432.CCR-17-0853

    work page doi:10.1158/1078-0432.ccr-17-0853 2018

  7. [7]

    A Variational Information Bottleneck Approach to Multi-Omics Data Integration,

    C. Lee and M. van der Schaar, “A Variational Information Bottleneck Approach to Multi-Omics Data Integration,” in Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, PMLR, Mar. 2021, pp. 1513–1521. Accessed: Apr. 30,

    work page 2021

  8. [8]

    Verifying explainability of a deep learning tissue classifier trained on RNA-seq data,

    M. Yap et al., “Verifying explainability of a deep learning tissue classifier trained on RNA-seq data,” Sci. Rep., vol. 11, no. 1, p. 2641, Jan. 2021, doi: 10.1038/s41598-021-81773-9

    work page doi:10.1038/s41598-021-81773-9 2021

Show all 24 references
  1. [9]

    Deep learning decodes the principles of diPerential gene expression,

    S. Tasaki, C. Gaiteri, S. Mostafavi, and Y . Wang, “Deep learning decodes the principles of diPerential gene expression,” Nat. Mach. Intell., vol. 2, no. 7, pp. 376–386, Jul. 2020, doi: 10.1038/s42256-020-0201-6

    2020 doi

  2. [10]

    Learning Important Features Through Propagating Activation DiPerences,

    A. Shrikumar, P . Greenside, and A. Kundaje, “Learning Important Features Through Propagating Activation DiPerences,” in Proceedings of the 34th International Conference on Machine Learning, PMLR, Jul. 2017, pp. 3145–3153. Accessed: Jan. 29,

    2017

  3. [12]

    A review of cancer data fusion methods based on deep learning,

    Y. Z h a o et al., “A review of cancer data fusion methods based on deep learning,” Inf. Fusion, vol. 108, p. 102361, Aug. 2024, doi: 10.1016/j.inPus.2024.102361

    2024

  4. [13]

    Deep latent space fusion for adaptive representation of heterogeneous multi-omics data,

    C. Zhang, Y. Chen, T. Zeng, C. Zhang, and L. Chen, “Deep latent space fusion for adaptive representation of heterogeneous multi-omics data,” Brief. Bioinform., vol. 23, no. 2, p. bbab600, Mar. 2022, doi: 10.1093/bib/bbab600

    2022 doi

  5. [14]

    Multi-Omics Factor Analysis—a framework for unsupervised integration of multi-omics data sets,

    R. Argelaguet et al., “Multi-Omics Factor Analysis—a framework for unsupervised integration of multi-omics data sets,” Mol. Syst. Biol., vol. 14, no. 6, p. e8124, Jun. 2018, doi: 10.15252/msb.20178124

    2018 doi

  6. [15]

    Gene-centric multi-omics integration with convolutional encoders for cancer drug response prediction,

    M. Lee, P .-J. Kim, H. Joe, and H.-G. Kim, “Gene-centric multi-omics integration with convolutional encoders for cancer drug response prediction,” Comput. Biol. Med., vol. 151, p. 106192, Dec. 2022, doi: 10.1016/j.compbiomed.2022.106192

    2022

  7. [16]

    The Cancer Genome Atlas Pan-Cancer analysis project,

    J. N. Weinstein et al., “The Cancer Genome Atlas Pan-Cancer analysis project,” Nat. Genet., vol. 45, no. 10, Art. no. 10, Oct. 2013, doi: 10.1038/ng.2764

    2013 doi

  8. [17]

    Focal Loss for Dense Object Detection,

    T.-Y . Lin, P . Goyal, R. Girshick, K. He, and P . Dollár, “Focal Loss for Dense Object Detection,” Feb. 07, 2018, arXiv: arXiv:1708.02002. doi: 10.48550/arXiv.1708.02002

    2018 doi arXiv

  9. [18]

    Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers,

    R. M. Schmidt, F . Schneider, and P . Hennig, “Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers,” Aug. 10, 2021, arXiv: arXiv:2007.01547. doi: 10.48550/arXiv.2007.01547

    2021 doi arXiv

  10. [19]

    A compendium of multi-omics data illuminating host responses to lethal human virus infections,

    A. J. Eisfeld et al., “A compendium of multi-omics data illuminating host responses to lethal human virus infections,” Sci. Data, vol. 11, no. 1, p. 328, Apr. 2024, doi: 10.1038/s41597-024-03124-3

    2024 doi

  11. [20]

    A Weighted Correlation Index for Rankings with Ties,

    S. Vigna, “A Weighted Correlation Index for Rankings with Ties,” Oct. 31, 2014, arXiv: arXiv:1404.3325. doi: 10.48550/arXiv.1404.3325

    2014 doi arXiv

  12. [21]

    Comparison of random forest, artificial neural networks and support vector machine for intelligent diagnosis of rotating machinery,

    T. Han, D. Jiang, Q. Zhao, L. Wang, and K. Yin, “Comparison of random forest, artificial neural networks and support vector machine for intelligent diagnosis of rotating machinery,” Trans. Inst. Meas. Control, vol. 40, no. 8, pp. 2681–2693, May 2018, doi: 10.1177/0142331217708242

    2018 doi

  13. [23]

    V-Measure: A Conditional Entropy-Based External Cluster Evaluation Measure,

    A. Rosenberg and J. Hirschberg, “V-Measure: A Conditional Entropy-Based External Cluster Evaluation Measure,” in Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), J. Eisner, E...

    2007

  14. [2018]

    The ICL102 data were originally collected as part of a study to evaluate human host cellular response to Influenza A virus (subtype H7N9): wild-type strain Influenza A/Anhui/1/2013 (AH1-WT), mutant viruses NS1-L103F/I106M (AH1-F/M), and partially ferret-adapted (AH1-691) infecti...

    2013

  15. [2022]

    Available: https://papers.nips.cc/paper/2019/hash/11b9842e0a271P252c1903e7132cd68-Abstract.html

    [Online]. Available: https://papers.nips.cc/paper/2019/hash/11b9842e0a271P252c1903e7132cd68-Abstract.html

    2019

  16. [2023]

    Available: https://papers.nips.cc/paper_files/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html

    [Online]. Available: https://papers.nips.cc/paper_files/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html

    2017

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.