REVIEW 3 major objections 5 minor 2 cited by
TxPert: Leveraging Biochemical Relationships for Out-of-Distribution Transcriptomic Perturbation Prediction
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read TxPert claims to predict unseen gene-perturbation effects at near-experimental fidelity by reading gene identity through biological knowledge graphs.
desk verdict Solid empirical paper with a load-bearing provenance gap: the headline SOTA depends on proprietary graphs that may encode the test labels. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is latent transfer over gene-gene knowledge graphs. A basal-state encoder (an MLP on control expression profiles, optionally with batch-matched control averaging) produces an embedding of the cell before perturbation; a GNN perturbation encoder maps each gene to an embedding by message passing over one or more interaction graphs, and the perturbation embedding is added to the basal embedding as a latent shift, with a decoder predicting the log-transformed expression profile. The multi-graph variant Exphormer-MG builds a union of edges across STRINGdb, GO, PxMap, and TxMap, using multi-hot edge-provenance features and an expander graph to support long-range attention. The paper identifies attention-based GNNs as suited for noisy graphs and multi-hop message passing as compensating for edge dropout; four-layer models stay stable when up to 60% of edges are removed.
What would settle it
Check whether the genes, cell lines, and perturbation identities used to construct PxMap and TxMap appear in the held-out splits of the Replogle, Norman, and Nadig datasets; if they do, retrain TxPert without those graphs and compare. A second direct test is to rerun the full TxPert pipeline using only STRINGdb and GO on public data and see whether it still beats the general baseline and approaches experimental reproducibility.
Extended reading notes
Core claim
The paper's central claim is that a single framework, TxPert, can generalize to perturbations it has never seen by reading the perturbed gene's identity through multiple biological knowledge graphs. For unseen single perturbations within a cell line, TxPert reports state-of-the-art Pearson $\Delta$ correlation, beating GEARS, scLAMBDA, and a non-learned general baseline, and matching or approaching experimental reproducibility in K562, Jurkat, and HEPG2 cells. For double perturbations, it outperforms the additive baseline and both published methods. For transfer to an unseen cell line, TxPert beats both the general baseline and the adapted scLAMBDA baseline in all four held-out lines. The authors treat the graph priors as essential: progressively rewiring the STRINGdb graph degrades performance, and combining STRINGdb, GO, PxMap, and TxMap into a multi-graph Exphormer transformer improves performance beyond any single graph.
Load-bearing premise
The load-bearing premise is that the proprietary PxMap and TxMap graphs were built from perturbation screens that do not overlap the held-out test perturbations or cell lines; if those graphs already encode the response to a test perturbation, the reported 'unseen' generalization is inflated.
Editorial extensions
If this is right
- If the results hold, unseen single perturbations can be prioritized in silico at a fidelity approaching the replicate-to-replicate consistency of the assay itself in well-characterized cell lines.
- Double-perturbation response prediction from seen singles can replace costly combinatorial screens for first-pass prioritization of candidate gene pairs.
- Zero-shot prediction to an unseen cell line is feasible for seen perturbations, making cross-context virtual screening a realistic target as cell-line diversity in training data grows.
- Evaluation standards using batch-matched controls and retrieval metrics should be adopted by later perturbation models, since global mean baselines capture generic stress responses rather than perturbation-specific signal.
Reading between the lines
- Editorial inference: because the paper does not show that the proprietary PxMap and TxMap graphs were built without overlap with the held-out perturbations, a public-data-only variant of TxPert is the natural test of whether the out-of-distribution claim reflects genuine generalization or partly information leakage.
- Editorial inference: the strength of the mean baseline implies that correlation-style metrics can look good even when a model is merely reproducing generic stress biology, so retrieval metrics and per-perturbation specificity should carry more weight in future evaluations.
- Editorial inference: a testable extension is to feed larger, multi-cell-line perturbation atlases into the basal-state encoder; the paper's own unseen-cell-line results suggest this would directly reduce the gap to experimental reproducibility.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. TxPert is a deep learning framework for predicting transcriptomic responses to genetic perturbations in three out-of-distribution settings: unseen single-gene perturbations within a known cell line, unseen double perturbations, and perturbations transferred to unseen cell lines. The model combines a basal-state encoder with a graph neural network perturbation encoder operating on multiple gene-gene interaction graphs, including curated sources (STRINGdb, GO) and two proprietary graphs derived from perturbation screens (PxMap, TxMap). The authors report that TxPert outperforms GEARS, scLAMBDA, and a non-learned general baseline in these settings, and that it approaches or matches experimental reproducibility for unseen single perturbations in some cell lines. The paper also proposes evaluation practices: batch-matched controls, stratified knowledge-level analysis, retrieval metrics, and a split-half experimental reproducibility estimate.
Significance. If the central claims hold, this is a useful contribution to perturbation prediction: the evaluation framework is more careful than much of the prior literature, with batch-matched controls, retrieval metrics, an explicit non-learned baseline, ablation studies, and an estimate of experimental reproducibility. The authors ship code for the public-data parts of the pipeline and are honest about limitations (e.g., the failure to predict the perturbation target's own downregulation). The proprietary-graph provenance question, the absence of error bars, and the missing linear baseline are the main factors preventing acceptance of the state-of-the-art claims as stated.
major comments (3)
- [Sections 2.2, 4.3.2, 5] The central OOD claim for unseen single perturbations (Fig. 3A) rests on TxMap, a graph built from proprietary single-cell transcriptomics perturbation screens (Section 2.2), and Fig. 4D shows that adding TxMap improves performance. Section 4.3.2 states that screen-derived graphs are computed as pairwise cosine similarities of gene embeddings derived from perturbation screens, but the manuscript does not report whether the internal screens used to build TxMap (or PxMap) include the genes, perturbations, or cell lines that appear in the Replogle, Nadig, or Norman evaluation splits. If any internal screen covered an ostensibly held-out perturbation, the graph edge weights encode the observed transcriptional response of that perturbation, and a GNN can retrieve it via neighboring nodes, inflating the reported OOD performance. Because TxMap is proprietary and not released (Section 5), this cannot be verified externally. The authors should provide a provenance audit: a list of the genes, perturbation types, and cell lines covered by the internal screens, and an overlap analysis against the evaluation splits; alternatively, they should report the main results with TxMap and PxMap excluded and show that the SOTA conclusion survives.
- [Sections 2.2, 4.1.3, 4.2.1] The paper tunes each architecture individually and reports the best model variant per task (Section 2.2: 'we tuned each architecture individually and report the best model variant per task'; Section 4.1.3), and the main results in Figures 3A, 3B, and 3C are point estimates with no error bars or confidence intervals. This makes it impossible to assess whether the reported differences between TxPert and baselines, or between TxPert and the experimental reproducibility estimate, are statistically meaningful; the claim in Section 2.3 that TxPert 'is competitive with experimental reproducibility' is a comparison of point estimates. The authors should report means and confidence intervals across multiple seeds, and ideally a table of all model variants per task, to address the risk of selection overfitting.
- [Sections 4.2.3 and 3] No linear baseline is included, despite the paper's own introduction noting that independent benchmarks have found linear methods to be strong (Ahlmann-Eltze et al., 2024; Kernfeld et al., 2023). The General Baseline is a non-learned mean/additive model, not a fitted linear model, and a linear regression baseline (e.g., per-gene ridge regression with perturbation one-hot features) could be substantially stronger. To support the 'state-of-the-art' claim, the evaluation framework should include a simple learned linear baseline trained under the same data splits.
minor comments (5)
- [Section 4.2.2] In the metric definitions, 'Person ∆' is a typo and should read 'Pearson ∆'.
- [Section 4.3.2] In the cosine similarity formula, the denominator repeats ∥xgi ∥; the second norm should be ∥xgj ∥.
- [Section 5] The sentence 'All data used here is already publicly available; with the exception of the PxMap and TxMap graphs' is self-contradictory; rephrase to clarify that all evaluation data are publicly available and only the two graphs are proprietary.
- [Figure 1D and Section 4.3.3] The legend states that expected random performance is 0.9 for the normalized retrieval metric because the 0.9 quantile is plotted; this dependence should be stated explicitly in the main text to avoid confusion.
- [Appendix A, 'Graph Attention Network'] The attention formula introduces the notation ATTθ,W(hu,hv) but then defines auv and α(q,k); the notation should be made consistent.
Circularity Check
No demonstrated circularity; TxPert is trained end-to-end on held-out perturbations, though the proprietary TxMap/PxMap graphs pose an unquantified data-independence risk.
full rationale
Walking the derivation chain in Sections 4.1–4.2, the prediction is produced by a learned decoder g_phi applied to a latent shift of a basal-state embedding and GNN-derived perturbation embeddings; all parameters are optimized with MSE on training perturbations, and test perturbations are held out by perturbation ID (Section 4.2.1), so no output is a renamed fitted constant. The General Baseline and experimental-reproducibility estimates are explicit comparators, not model outputs. The SOTA claim is supported by held-out evaluations (Fig. 3) and ablations (Fig. 4), not by citation alone. The main caveat is that TxMap and PxMap are proprietary graphs built from perturbation screens (Sections 2.2 and 4.3.2) and are not released (Section 5). If the internal screens used to construct TxMap overlap the Replogle, Nadig, or Norman test perturbations or cell lines, the graph edges could encode the very transcriptional responses the model is asked to predict, which would inflate the OOD results; however, the paper never asserts such an overlap, and the manuscript's equations do not exhibit a by-construction identity between graph weights and test labels. The absence of a provenance audit is a data-independence risk for external verification rather than a demonstrated circular step, so the circularity score is low.
Assumptions & free parameters
free parameters (4)
- Edge filtering threshold for screen-based graphs (top 1%) =
1% (top edges by absolute weight)
- Maximum in-degree for curated graph filtering (20 incoming nodes) =
20 incoming nodes per target
- TxPert model variant per task (Exphormer-MG, GAT-MultiLayer, GAT-Hybrid, etc.) =
Varies by task (best on validation)
- Basal state matching and averaging settings =
On/off chosen per task; averaging used in standard config
assumptions (4)
- domain assumption Gene interaction knowledge graphs (STRINGdb, GO, PxMap, TxMap) encode relationships that provide useful inductive bias for predicting transcriptional perturbation responses.
- ad hoc to paper TxMap and PxMap are constructed from perturbation screens without overlapping the evaluation perturbations, so the OOD test remains clean.
- domain assumption Batch-matched controls (matching and averaging) provide a faithful basal state that removes batch effects without removing perturbation signal.
- standard math Standard definitions of GNNs, attention, and MSE loss are correct and applicable.
invented entities (2)
-
PxMap
-
TxMap
Cite this review
Pith. "Pith review of TxPert: Leveraging Biochemical Relationships for Out-of-Distribution Transcriptomic Perturbation Prediction." pith.science (2026). https://pith.science/paper/SPIQXI2P
@misc{pith2026250514919,
author = {Pith},
title = {Pith review of: TxPert: Leveraging Biochemical Relationships for Out-of-Distribution Transcriptomic Perturbation Prediction},
year = {2026},
howpublished = {\url{https://pith.science/paper/SPIQXI2P}},
note = {Machine review of arXiv:2505.14919}
}
read the original abstract
Accurately predicting cellular responses to genetic perturbations is essential for understanding disease mechanisms and designing effective therapies. Yet exhaustively exploring the space of possible perturbations (e.g., multi-gene perturbations or across tissues and cell types) is prohibitively expensive, motivating methods that can generalize to unseen conditions. In this work, we explore how knowledge graphs of gene-gene relationships can improve out-of-distribution (OOD) prediction across three challenging settings: unseen single perturbations; unseen double perturbations; and unseen cell lines. In particular, we present: (i) TxPert, a new state-of-the-art method that leverages multiple biological knowledge networks to predict transcriptional responses under OOD scenarios; (ii) an in-depth analysis demonstrating the impact of graphs, model architecture, and data on performance; and (iii) an expanded benchmarking framework that strengthens evaluation standards for perturbation modeling.
Figures
Figures from the paper (11 more)
Forward citations
Cited by 2 Pith papers
-
PertReason: A Knowledge-Grounded Benchmark and Framework for Cell-State-Conditioned Mechanistic Reasoning of Perturbation Effects
PertReasonQA scores AI models on cell-state-conditioned mechanistic reasoning about perturbation effects, and PertReasonLM, trained with reasoning supervision, reaches 0.736 balanced accuracy and 0.976 edge recall ver...
-
Learning Adaptive Perturbation-Conditioned Contexts for Robust Transcriptional Response Prediction
AdaPert improves transcriptional perturbation prediction by conditioning sparse knowledge-graph subgraphs on each perturbed gene and training with signal/noise separation losses, achieving the best DEG recovery on K56...
Reference graph
Works this paper leans on
-
[1]
Ahlmann-Eltze, C., Huber, W., and Anders, S. Deep learning-based predictions of gene perturbation effects do not yet outperform simple linear methods.BioRxiv, pp. 2024–09,
work page 2024
-
[6]
M., Zhou, Y., Crepaldi, L., Usluer, S., Dunham, A., Braunger, J
Feng, C., Peets, E. M., Zhou, Y., Crepaldi, L., Usluer, S., Dunham, A., Braunger, J. M., Su, J., Strauss, M. E., Muraro, D., et al. A genome-scale single cell crispri map of trans gene regulation across human pluripotent stem cell lines.bioRxiv, pp. 2024–11,
work page 2024
-
[9]
Sheils, T. K., Mathias, S. L., Kelleher, K. J., Siramshetty, V. B., Nguyen, D.-T., Bologa, C. G., Jensen, L. J., Vidovi´ c, D., Koleti, A., Sch¨ urer, S. C., et al. Tcrd and pharos 2021: mining the human proteome for disease biology.Nucleic Acids Research, 49(D1):D1334–D1346,
work page 2021
-
[10]
Shirzad, H., Lin, H., Velingker, A., Venkatachalam, B., Woodruff, D., and Sutherland, D. A theory for compressibility of graph transformers for transductive learning.arXiv preprint arXiv:2411.13028, 2024a. Shirzad, H., Lin, H., Venkatachalam, B., Velingker, A., Woodruff, D. P., and Sutherland, D. J. Even sparser graph transformers.Advances in Neural Infor...
-
[12]
Cellplm: pre-training of cell language model beyond single cells.BioRxiv, pp
Wen, H., Tang, W., Dai, X., Ding, J., Jin, W., Xie, Y., and Tang, J. Cellplm: pre-training of cell language model beyond single cells.BioRxiv, pp. 2023–10,
work page 2023
-
[13]
Overcoming Oversmoothness in Graph Convolutional Networks via Hybrid Scattering Networks
Wenkel, F., Min, Y., Hirn, M., Perlmutter, M., and Wolf, G. Overcoming oversmoothness in graph convolutional networks via hybrid scattering networks.arXiv preprint arXiv:2201.08932,
-
[14]
Cayley graph propagation.arXiv preprint arXiv:2410.03424,
Wilson, J., Bechler-Speicher, M., and Veliˇ ckovi´ c, P. Cayley graph propagation.arXiv preprint arXiv:2410.03424,
-
[15]
Wong, D. R., Hill, A., and Moccia, R. Simple controls exceed best deep learning algorithms and reveal foundation model effectiveness for predicting genetic perturbations.bioRxiv, pp. 2025–01,
work page 2025
Show all 20 references
-
[18]
A., Borja, R
Zhang, J., Ubas, A. A., Borja, R. de, Svensson, V., Thomas, N., Thakar, N., Lai, I., Winters, A., Khan, U., Jones, M. G., et al. Tahoe-100m: A giga-scale single-cell perturbation atlas for context-dependent gene function and cellular modeling.bioRxiv, pp. 2025–02,
2025
-
[19]
, vN ⋆ } and edges E⊂V×V representing gene-gene interactions
21 TxPert Appendix : Leveraging Biochemical Relationships for Out-of-Distribution Transcriptomic Perturbation Prediction A GNN perturbation encoder model details General graph network structure.We consider a simple weighted graph G = (V, E, w) with a set of N ⋆ ∈N nodes (or ge...
2021
-
[20]
Exphormer
=f L ◦f L−1 ◦ · · · ◦f1(H0). This model uses an attention mechanism to aggregate information during message-passing (MP). For every edge, we learn an attention score that characterizes its importance that is then normalized at every node across incoming edges. On the node leve...
2024
-
[2004]
Z., and Szalai, B
Csendes, G., Szalay, K. Z., and Szalai, B. Benchmarking a foundational cell model for post-perturbation rnaseq prediction.bioRxiv, pp. 2024–09,
2024
-
[2016]
Dwivedi, V. P. and Bresson, X. A generalization of transformer networks to graphs.arXiv preprint arXiv:2012.09699,
2012 arXiv
-
[2018]
Modeling and predicting single-cell multi-gene perturbation responses with sclambda.bioRxiv, pp
Wang, G., Liu, T., Zhao, J., Cheng, Y., and Zhao, H. Modeling and predicting single-cell multi-gene perturbation responses with sclambda.bioRxiv, pp. 2024–12,
2024
-
[2019]
M., Nassar, M., Osi´ nski, B., Eksi, R., Zhang, K., and Graepel, T
Wu, Y., Wershof, E., Schmon, S. M., Nassar, M., Osi´ nski, B., Eksi, R., Zhang, K., and Graepel, T. Perturbench: Benchmarking machine learning models for cellular perturbation analysis.arXiv preprint arXiv:2408.10609,
-
[2021]
Link prediction on multilayer networks through learning of within-layer and across-layer node-pair structural features and node embedding similarity
Zangari, L., Mandaglio, D., and Tagarelli, A. Link prediction on multilayer networks through learning of within-layer and across-layer node-pair structural features and node embedding similarity. In Proceedings of the ACM Web Conference 2024, pp. 924–935,
2024
-
[2022]
K., Bronstein, M
Di Giovanni, F., Rusch, T. K., Bronstein, M. M., Deac, A., Lackenby, M., Mishra, S., and Veliˇ ckovi´ c, P. How does over-squashing affect the power of gnns?arXiv preprint arXiv:2306.03589,
-
[2023]
The reactome pathway knowledgebase 2024.Nucleic acids research, 52(D1): D672–D678,
Milacic, M., Beavers, D., Conley, P., Gong, C., Gillespie, M., Griss, J., Haw, R., Jassal, B., Matthews, L., May, B., et al. The reactome pathway knowledgebase 2024.Nucleic acids research, 52(D1): D672–D678,
2024
-
[2024]
B., Mesbahi, Y
Bendidi, I., Whitfield, S., Kenyon-Dean, K., Yedder, H. B., Mesbahi, Y. E., Noutahi, E., and Denton, A. K. Benchmarking transcriptomics foundation models for perturbation analysis: one pca still rules them all.arXiv preprint arXiv:2410.13956,
-
[2025]
S., Battle, A., and Cahan, P
Kernfeld, E., Yang, Y., Weinstock, J. S., Battle, A., and Cahan, P. A systematic comparison of computational methods for expression forecasting.BioRxiv, pp. 2023–07,
2023
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.