Pith. sign in

REVIEW 3 major objections 5 minor 51 references

Frozen but Not Always Accessible: A Representation Analysis of Genomic Language Models

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Frozen genomic embeddings recover 95–100% of fine-tuned promoter performance but only 60–88% on splice sites.

desk verdict A plausible and practically useful qualitative claim about frozen genomic LMs, but the exact recovery percentages need more matched baselines and cleaner tables before they should be trusted. read the letter →

arxiv 2608.05329 v1 pith:FY32WNTC submitted 2026-08-05 q-bio.GN

Nirjhor Datta , Swakkhar Shatabda , M. Sohel Rahman This is my paper · ORCID
classification q-bio.GN
keywords genomiclanguagemodelsfrozenembeddingsrepresentationprobingpromoterpredictionsplice-sitevariant-effectaccessibilityrecoverymetric
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks when a frozen genomic language model can stand in for fine-tuning on sequence-prediction tasks. Its central claim is that the answer depends on the biological task, not the model family: across five model architectures, frozen probes recover 95–100% of fine-tuned performance on promoter tasks but only 60–88% on splice-site tasks. The authors conclude that global, composition-driven signals are accessible from frozen embeddings, while local, position-sensitive signals such as splice sites and some pathogenic variants are only partially encoded and often hidden from simple readouts. This matters because frozen probing is much cheaper than fine-tuning, and the paper maps which tasks can safely skip adaptation.

What carries the argument

The load-bearing machinery is the frozen-probing protocol with a recovery metric. Token-level hidden states from a frozen pretrained encoder are pooled by one of six readouts—mean, max, CLS, decoder last-token, norm-attention, and top-k—then fed to a lightweight probe (logistic regression, RBF-SVM, or an MLP with hidden layers 512-256). The recovery metric is ${\rm Frozen\ Probe\ Score} / {\rm Fine\text{-}Tuned\ Score} \times 100$, which converts every task into a comparison against a fine-tuned ceiling. The paper separates diagnostic grid-best readout scans, which measure readout sensitivity, from validation-selected checks that choose one configuration on a held-out split and report test performance once.

What would settle it

Run a matched frozen-probe versus fine-tuned comparison on the Nucleotide Transformer Splice All task using identical preprocessing, splits, and readout; if the matched recovery exceeds 95%, the task-dependent gap claim is refuted. A second direct test is a token-level attention probe on intermediate layers for Splice All—if it reaches fine-tuned accuracy, then local signal is accessible and the paper's 'not always accessible' conclusion would need to be narrowed.

Watch

Extended reading notes

Core claim

The paper establishes a task-dependent accessibility pattern across DNABERT-2, Nucleotide Transformer, HyenaDNA, GENERATOR-v2, and Omni-DNA: frozen representations recover 95–100% of fine-tuned scores on promoter tasks (GENERATOR-v2 reaches 100% on Promoter TATA), but average splice-site recovery drops to 60–88%, with Splice All recovery as low as 46–55% on some models. Embedding geometry mirrors these results: promoter classes show high nearest-neighbor consistency and linear separability in frozen pooled space, whereas splice classes have negative silhouette scores and weak neighborhood consistency. Layer-wise probing, in-silico mutagenesis, and variant-effect analysis show that local splice signal is partially present in token-level or intermediate representations, but the paper concludes it is not always accessible through final pooled embeddings and lightweight probes.

Load-bearing premise

The load-bearing premise is that benchmark- or model-reported fine-tuned scores are comparable ceilings for the authors' frozen-probe protocol; if those references used different preprocessing, splits, sequence lengths, or model versions, the exact 95–100% and 60–88% recovery ranges could shift.

Editorial extensions

If this is right

  • For promoter, coding-region, and species-discrimination tasks, practitioners can freeze the encoder, train a lightweight probe, and expect accuracy close to fine-tuning at a fraction of the compute.
  • For splice-site and pathogenic-variant prediction, freezing is not a safe default; the paper's controlled LoRA comparison shows the probe can fall far short (Splice All at 49.57 versus 95.27 fine-tuned) even when readout and protocol are matched.
  • Readout choice changes scores but does not close the local-task gap, so the gap reflects how splice classes are organized in embedding space, not merely which pooling function is used.
  • The same promoter-versus-splice pattern appears across transformer, Hyena, generative, and unified model families, which supports the conclusion that task property, not architecture, decides frozen accessibility.
  • Frozen probing is 4–10× faster than LoRA fine-tuning in the matched comparison, so the compute savings are real but conditional on task type.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: if an attention-based token-level probe over intermediate layers pushed Splice All recovery toward the fine-tuned ceiling, the conclusion would shift from 'local signal is not accessible' to 'the final pooled embedding discards it'.
  • Editorial extension: the geometry diagnostics suggest a cheap screening rule for other tasks—measure silhouette and kNN consistency of frozen embeddings on a validation set; tasks with negative silhouette are likely to need adaptation, and tasks with positive silhouette can likely be frozen.
  • Editorial extension: the headline recovery percentages inherit the paper's own caveat about reference fine-tuned scores; computing recovery against matched fine-tuned baselines for every task, not just DNABERT-2, would give a more trustworthy speed-accuracy trade-off table.
  • Editorial extension: because mutagenesis sensitivity improves with top-k pooling (1.24× over mean pooling), per-position featurization rather than full fine-tuning may partially close the splice gap; this is consistent with the paper's data but not tested there.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies when frozen embeddings from five genomic language models (DNABERT-2, Nucleotide Transformer, HyenaDNA, GENERATOR-v2, and Omni-DNA) can substitute for fine-tuning on sequence-classification and variant-effect tasks. It defines a unified frozen-probing protocol with multiple pooling and readout options, lightweight probe classifiers, and a recovery metric equal to the frozen-probe score divided by the fine-tuned score. The central reported result is task-dependent: frozen probes recover 95–100% of fine-tuned performance on promoter tasks but only 60–88% average recovery on splice-site tasks, with supplementary analyses on Genomic Benchmark, layer-wise probing, in-silico mutagenesis, variant-effect representation shifts, and embedding geometry. A controlled DNABERT-2 LoRA comparison is included as a matched frozen-versus-adapted baseline.

Significance. If the quantitative claims held as stated, the paper would give practitioners a useful rule for when frozen genomic representations are sufficient, and the observation that task type (global/composition-driven versus local/position-sensitive) rather than model family governs accessibility would be a valuable addition. The study's methodological strengths include the unified probing protocol, the explicit separation of diagnostic grid-best analysis from validation-selected checks, provenance-marked reference fine-tuned scores, and the matched DNABERT-2 LoRA comparison. These choices make the qualitative local/global pattern credible even though the exact headline percentages need revision before publication.

major comments (3)
  1. [Section 4 (RQ1), Table 1; Section 2.5] Table 1, which supplies the abstract's headline ranges, appears to report diagnostic grid-best frozen scores rather than validation-selected ones, and this is not stated in the table or the text. For HyenaDNA, the Splice All frozen value 0.5243±0.0051 in Table 1 matches the diagnostic grid-best entry in Table 4, whereas the validation-selected value is 0.5288±0.0044. Section 2.5 explicitly states that diagnostic grid-best results are not unbiased model-selection estimates, and validation-selected readout selection is reported only for HyenaDNA. The 95–100% and 60–88% ranges may therefore include test-set selection over readout and probe configurations and are not necessarily achievable by any single protocol. Please report Table 1 under the validation-selected protocol for all five models, or mark every cell as diagnostic and remove the unqualified ranges from the abstract.
  2. [Section 3.3 and Eq. (7); Table 9] The fine-tuned denominators in Table 1 are benchmark- or model-reported reference values, not scores obtained under the same preprocessing, splits, sequence lengths, and readout as the frozen probes. The only fully matched comparison, DNABERT-2 LoRA in Table 9, gives a Splice All recovery of 49.57/95.27 ≈ 52%, which is below the abstract's lower bound of 60%; the same matched comparison gives promoter recovery of about 97.6%, consistent with the promoter range. Thus the qualitative promoter-versus-splice gap is robust in the matched setting, but the exact ranges in the abstract (95–100% and 60–88%) are not established for models outside the four-model NT-benchmark subset. Please recompute the headline ranges using matched fine-tuned denominators (for example, by extending the LoRA protocol to all backbones) or explicitly restate them as relative to unmatched reference values.
  3. [Section 4 (RQ1), Table 2] Table 2 has internal inconsistencies that need correction before the Genomic Benchmark claims can be assessed. The NT Emb(best) column lists 90.61 for both Human Ensembl Regulatory and Human OCR, which appears to be a duplicated value; the reported DNABERT-2 average of 81.49 does not match the average of the eight preceding cells (83.94); and the HyenaDNA average (79.39) also differs from the computed value (80.14). Please correct the cells and recompute the averages, then confirm that the qualitative statements about coding-region, species-discrimination, and OCR tasks survive.
minor comments (5)
  1. [Sections 2.4 and 2.5] RBF-SVM is introduced as a probe in Section 2.4 and appears in Appendix B, but the diagnostic grid in Section 2.5 is defined as {mean, max, CLS, last, attn, top-k} × {LR, MLP}; clarify whether RBF-SVM is part of the readout-selection grid or only a separate analysis.
  2. [Table 1] Add a footnote explaining which cells are diagnostic grid-best and which are validation-selected; the current caption says only that frozen values use the unified probing protocol, which is not sufficient given the distinction drawn in Section 2.5.
  3. [Table 5] The final row reports approximate values with ranges (∼0.60, ∼0.52, 0.36–0.47) instead of mean ± standard deviation; use exact values for consistency with the other rows.
  4. [Section 4 (RQ4), Table 7] The column header 'Hyena DNA2' is ambiguous; it should be DNABERT-2 or HyenaDNA as appropriate.
  5. [Abstract and Section 6] Section 6 acknowledges that pooling may dilute local signals and that only a limited set of probes was tested; the abstract's strong '95–100%' and '60–88%' wording should be brought into line with these caveats.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: recovery ratios are externally referenced measurements, not fitted predictions; headline-range caveats are comparability issues, not self-referential derivation.

full rationale

The paper's central quantities are recovery ratios defined in Eq. (7) as Frozen Probe Score / Fine Tuned Score, which is a measurement against external references rather than a quantity constructed from its own outputs. No fitted parameter is later renamed as a prediction: the probe classifiers are trained on training splits only, standardization uses training-set statistics (Section 2.4), and validation-selected readouts use a held-out split with the test set used only for final reporting (Section 2.5). The fine-tuned denominators are explicitly marked in Section 3.3 as benchmark- or model-reported reference values, not as products of the paper's own fitting procedure, so the recovery numbers are not forced by definition. The controlled DNABERT-2 LoRA comparison in Section 2.7 and Table 9 provides an independent matched baseline; its lower Splice All recovery (about 52%) is a substantive challenge to the abstract's headline range, but that is a correctness or comparability concern, not circular reasoning. There are no load-bearing self-citations, no imported uniqueness theorem, and no ansatz smuggled in via citation. The diagnostic grid-best analysis is explicitly labeled as a readout-sensitivity diagnostic rather than an unbiased model-selection estimate (Section 2.5, Tables 3 and 4), and while using such values in Table 1 raises methodological questions, it does not create a derivation loop. The Limitations section candidly notes pooling, probe, and variant-type restrictions, but those limitations do not close the claimed derivation chain. Overall, the paper is an empirical benchmark study whose conclusions are self-contained measurements with external-validity caveats, not circular arguments.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The study does not derive a theory; it evaluates empirical access. The main unstated assumption is that benchmark-reported fine-tuned scores are comparable denominators for recovery. No invented entities. No fitted parameters in the derivation sense, though probe hyperparameters and top-k values are chosen by grid search.

assumptions (2)
  • domain assumption Benchmark-reported fine-tuned scores are comparable enough to serve as recovery denominators.
    Recovery ratios in Section 4 RQ1 and the abstract divide frozen-probe scores by externally reported fine-tuned values from heterogeneous protocols; Section 3.3 acknowledges they are reference values, not controlled comparisons.
  • domain assumption Frozen inference-mode embeddings represent the reusable feature extractors in practice.
    The entire study treats zero-gradient frozen encoders with lightweight probes as the representative reuse setting; this excludes adapter-like or prompt-based methods that might improve accessibility.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Frozen but Not Always Accessible: A Representation Analysis of Genomic Language Models." pith.science (2026). https://pith.science/paper/FY32WNTC

@misc{pith2026260805329,
  author       = {Pith},
  title        = {Pith review of: Frozen but Not Always Accessible: A Representation Analysis of Genomic Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FY32WNTC}},
  note         = {Machine review of arXiv:2608.05329}
}
read the original abstract

Genomic foundation models are increasingly reused as frozen feature extractors for downstream sequence prediction, offering a compute-efficient alternative to full fine-tuning. However, it remains unclear when biological information encoded by these models is accessible without task-specific adaptation. We present a representation-accessibility analysis of frozen genomic language models across regulatory, epigenetic, promoter, splice-site, and variant-effect prediction tasks. We evaluate DNABERT-2, Nucleotide Transformer, HyenaDNA, GENERATOR-v2, and Omni-DNA under unified frozen-probing protocols, while separating diagnostic readout analyses from validation-selected checks. Our results reveal a consistent task-dependent pattern: frozen probes recover 95-100 % of fine-tuned performance on promoter tasks, but average splice-site recovery drops to 60-88 %. Frozen embeddings are also competitive on broad Genomic Benchmark tasks such as coding-region and species-discrimination classification, but show larger gaps on some regulatory and OCR tasks. Layer-wise probing, in-silico mutagenesis, variant-effect prediction, and embedding geometry show that local biological signal is partially present in frozen representations, but is not always accessible through final pooled embeddings.

Figures

Figures reproduced from arXiv: 2608.05329 by the authors.

Figure 1
Figure 1. Top-level overview of the proposed framework. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 35 canonical work pages

  1. [1]

    Bioinformatics , volume=

    DNABERT: pre-trained Bidirectional Encoder Representations from Transformers model for DNA-language in genome , author=. Bioinformatics , volume=. 2021 , publisher=

    2021

  2. [3]

    Concurrency: the Works of Leslie Lamport , pages=

    Time, clocks, and the ordering of events in a distributed system , author=. Concurrency: the Works of Leslie Lamport , pages=

    work page

  3. [4]

    ACM Sigplan Notices , volume=

    FastTrack: efficient and precise dynamic race detection , author=. ACM Sigplan Notices , volume=. 2009 , publisher=

    work page 2009

  4. [5]

    Advances in neural information processing systems , volume=

    Hyenadna: Long-range genomic sequence modeling at single nucleotide resolution , author=. Advances in neural information processing systems , volume=

  5. [6]

    BMC genomics , volume=

    Unsupervised evaluation of pre-trained DNA language model embeddings , author=. BMC genomics , volume=. 2025 , publisher=

    work page 2025

  6. [7]

    BMC Genomic Data , volume=

    Genomic benchmarks: a collection of datasets for genomic sequence classification , author=. BMC Genomic Data , volume=. 2023 , publisher=

    2023

  7. [9]

    Nature Methods , volume=

    Nucleotide transformer: building and evaluating robust foundation models for human genomics , author=. Nature Methods , volume=. 2025 , publisher=

    work page 2025

  8. [10]

    Nature methods , volume=

    Predicting effects of noncoding variants with deep learning--based sequence model , author=. Nature methods , volume=. 2015 , publisher=

    work page 2015

Show all 51 references
  1. [11]

    Nature biotechnology , volume=

    Predicting the sequence specificities of DNA-and RNA-binding proteins by deep learning , author=. Nature biotechnology , volume=. 2015 , publisher=

    2015

  2. [12]

    arXiv preprint arXiv:1905.06316 , year=

    What do you learn from context? probing for sentence structure in contextualized word representations , author=. arXiv preprint arXiv:1905.06316 , year=

    1905 arXiv

  3. [13]

    A structural probe for finding syntax in word representations , author=. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pages=

    2019

  4. [14]

    Nucleic acids research , volume=

    DanQ: a hybrid convolutional and recurrent deep neural network for quantifying the function of DNA sequences , author=. Nucleic acids research , volume=. 2016 , publisher=

    2016

  5. [15]

    Computational and Mathematical Methods in Medicine , volume=

    Analysis of DNA sequence classification using CNN and hybrid models , author=. Computational and Mathematical Methods in Medicine , volume=. 2021 , publisher=

    2021

  6. [16]

    Applied Sciences , volume=

    Optimizing DNA Sequence Classification via a Deep Learning Hybrid of LSTM and CNN Architecture , author=. Applied Sciences , volume=. 2025 , publisher=

    2025

  7. [17]

    Nature methods , volume=

    Effective gene expression prediction from sequence by integrating long-range interactions , author=. Nature methods , volume=. 2021 , publisher=

    2021

  8. [18]

    Nature Communications , volume=

    Learning the histone codes with large genomic windows and three-dimensional chromatin interactions using transformer , author=. Nature Communications , volume=. 2022 , publisher=

    2022

  9. [19]

    Advances in neural information processing systems , volume=

    Language models are few-shot learners , author=. Advances in neural information processing systems , volume=

  10. [20]

    arXiv preprint arXiv:2108.07258 , year=

    On the opportunities and risks of foundation models , author=. arXiv preprint arXiv:2108.07258 , year=

    arXiv

  11. [21]

    Nature genetics , volume=

    A general framework for estimating the relative pathogenicity of human genetic variants , author=. Nature genetics , volume=. 2014 , publisher=

    2014

  12. [22]

    Nucleic acids research , volume=

    CADD: predicting the deleteriousness of variants throughout the human genome , author=. Nucleic acids research , volume=. 2019 , publisher=

    2019

  13. [23]

    arXiv preprint arXiv:1610.01644 , year=

    Understanding intermediate layers using linear classifier probes , author=. arXiv preprint arXiv:1610.01644 , year=

    arXiv

  14. [24]

    Advances in neural information processing systems , volume=

    Svcca: Singular vector canonical correlation analysis for deep learning dynamics and interpretability , author=. Advances in neural information processing systems , volume=

  15. [25]

    Proceedings of the 57th annual meeting of the association for computational linguistics , pages=

    Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned , author=. Proceedings of the 57th annual meeting of the association for computational linguistics , pages=

  16. [26]

    Biomedical reports , volume=

    Mechanism of alternative splicing and its regulation , author=. Biomedical reports , volume=. 2015 , publisher=

    2015

  17. [27]

    The American Journal of Human Genetics , volume=

    The regulation of splice-site selection, and its role in human disease , author=. The American Journal of Human Genetics , volume=. 1997 , publisher=

    1997

  18. [28]

    PloS one , volume=

    Recognition of prokaryotic and eukaryotic promoters using convolutional deep learning neural networks , author=. PloS one , volume=. 2017 , publisher=

    2017

  19. [29]

    Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    MP-RNA: unleashing multi-species RNA foundation model via calibrated secondary structure prediction , author=. Findings of the Association for Computational Linguistics: EMNLP 2024 , pages=

    2024

  20. [30]

    The genomics long-range benchmark: advancing DNA language models , author=

  21. [31]

    Nature , volume=

    Genome modelling and design across all domains of life with Evo 2 , author=. Nature , volume=. 2026 , publisher=

    2026

  22. [32]

    bioRxiv , pages=

    A foundational model for joint sequence-function multi-species modeling at scale for long-range genomic prediction , author=. bioRxiv , pages=. 2025 , publisher=

    2025

  23. [33]

    Findings of the association for computational linguistics: EMNLP 2024 , pages=

    Geneverse: A collection of open-source multimodal large language models for genomic and proteomic research , author=. Findings of the association for computational linguistics: EMNLP 2024 , pages=

    2024

  24. [35]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    RBPtool: A Deep Language Model Framework for Multi-Resolution RBP-RNA Binding Prediction and RNA Molecule Design , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  25. [36]

    International Conference on Learning Representations , volume=

    DNABERT-2: Efficient foundation model and benchmark for multi-species genomes , author=. International Conference on Learning Representations , volume=

  26. [37]

    Proceedings of machine learning research , volume=

    Caduceus: Bi-directional equivariant long-range dna sequence modeling , author=. Proceedings of machine learning research , volume=

  27. [40]

    Advances in Neural Information Processing Systems , volume=

    Dart-eval: A comprehensive dna language model evaluation benchmark on regulatory dna , author=. Advances in Neural Information Processing Systems , volume=

  28. [41]

    arXiv preprint arXiv:2511.02888 , year=

    NABench: large-scale benchmarks of nucleotide foundation models for fitness prediction , author=. arXiv preprint arXiv:2511.02888 , year=

  29. [42]

    Hugo Dalla-Torre, Liam Gonzalez, Javier Mendoza-Revilla, Nicolas Lopez Carranza, Adam Henryk Grzywaczewski, Francesco Oteri, Christian Dallago, Evan Trop, Bernardo P De Almeida, Hassan Sirelkhatim, and 1 others. 2025. Nucleotide transformer: building and evaluating robust foun...

    2025

  30. [43]

    Katar \' na Gre s ov \'a , Vlastimil Martinek, David C ech \'a k, Petr S ime c ek, and Panagiotis Alexiou. 2023. Genomic benchmarks: a collection of datasets for genomic sequence classification. BMC Genomic Data, 24(1):25

    2023

  31. [44]

    Haonan He, Yuchen Ren, Yining Tang, Ziyang Xu, Junxian Li, Minghao Yang, Di Zhang, Dong Yuan, Tao Chen, Shufei Zhang, and 1 others. 2024. Biology-instructions: A dataset and benchmark for multi-omics sequence understanding capability of large language models. arXiv preprint ar...

    2024

  32. [45]

    Yanrong Ji, Zhihan Zhou, Han Liu, and Ramana V Davuluri. 2021. Dnabert: pre-trained bidirectional encoder representations from transformers model for dna-language in genome. Bioinformatics, 37(15):2112--2120

    2021

  33. [46]

    Jiyue Jiang, Yitao Xu, Zikang Wang, Yihan Ye, Yanruisheng Shao, Yuheng Shan, Jiuming Wang, Xiaodan Fan, Jiao Yuan, and Yu Li. 2025. Rbptool: A deep language model framework for multi-resolution rbp-rna binding prediction and rna molecule design. In Proceedings of the 2025 Conf...

    2025

  34. [47]

    Zehui Li, Vallijah Subasri, Yifei Shen, Dongsheng Li, Yiren Zhao, Guy-Bart Stan, and Caihua Shan. 2025. Omni-dna: A unified genomic foundation model for cross-modal and multi-task learning. arXiv preprint arXiv:2502.03499

    2025 arXiv

  35. [48]

    Tianyu Liu, Yijia Xiao, Xiao Luo, Hua Xu, Wenjin Zheng, and Hongyu Zhao. 2024 a . Geneverse: A collection of open-source multimodal large language models for genomic and proteomic research. In Findings of the association for computational linguistics: EMNLP 2024, pages 4819--4836

    2024

  36. [49]

    Zicheng Liu, Jiahui Li, Siyuan Li, Zelin Zang, Cheng Tan, Yufei Huang, Yajing Bai, and Stan Z Li. 2024 b . Genbench: A benchmarking suite for systematic evaluation of genomic foundation models. arXiv preprint arXiv:2406.01627

    2024 arXiv

  37. [50]

    Eric Nguyen, Michael Poli, Marjan Faizi, Armin Thomas, Michael Wornow, Callum Birch-Sykes, Stefano Massaroli, Aman Patel, Clayton Rabideau, Yoshua Bengio, and 1 others. 2023. Hyenadna: Long-range genomic sequence modeling at single nucleotide resolution. Advances in neural inf...

    2023

  38. [51]

    Yair Schiff, Chia-Hsiang Kao, Aaron Gokaslan, Tri Dao, Albert Gu, and Volodymyr Kuleshov. 2024. Caduceus: Bi-directional equivariant long-range dna sequence modeling. Proceedings of machine learning research, 235:43632

    2024

  39. [52]

    Evan Trop, Yair Schiff, Edgar Mariano Marroquin, Chia Hsiang Kao, Aaron Gokaslan, McKinley Polen, Mingyi Shao, Bernardo P de Almeida, Thomas Pierrot, Yang I Li, and 1 others. 2024. The genomics long-range benchmark: advancing dna language models

    2024

  40. [53]

    Wei Wu, Qiuyi Li, Yuanyuan Zhang, Zhihao Zhan, Ruipu Chen, Mingyang Li, Kun Fu, Junyan Qi, Yongzhou Bao, Chao Wang, and 1 others. 2025. Generator: a long-context generative genomic foundation model. arXiv preprint arXiv:2502.07272

    2025

  41. [54]

    Heng Yang and Ke Li. 2024. Mp-rna: unleashing multi-species rna foundation model via calibrated secondary structure prediction. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 5278--5296

    2024

  42. [55]

    Zhihan Zhou, Yanrong Ji, Weijian Li, Pratik Dutta, Ramana Davuluri, and Han Liu. 2023. Dnabert-2: Efficient foundation model and benchmark for multi-species genome. arXiv preprint arXiv:2306.15006

    2023 arXiv

  43. [56]

    Zhihan Zhou, Yanrong Ji, Weijian Li, Pratik Dutta, Ramana Davuluri, and Han Liu. 2024. Dnabert-2: Efficient foundation model and benchmark for multi-species genomes. In International Conference on Learning Representations, volume 2024, pages 41642--41665

    2024

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.