REVIEW 4 major objections 4 minor 38 references
Find Central Dogma Again: Leveraging Multilingual Transfer in Large Language Models
T0 review · 4 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read Sentence-pair training lets GPT-2 rediscover the genetic code
desk verdict A creative cross-modal transfer experiment whose headline 81% is an artifact of best-of-100 seed selection and a test set that rewards shallow sequence statistics. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism is cross-modal language transfer mediated by a shared Byte Pair Encoding tokenizer and a structurally analogous fine-tuning task. PAWS-X, a cross-lingual sentence-paraphrase dataset, is chosen because deciding whether two sentences mean the same thing is formally similar to deciding whether a DNA sequence and a protein sequence encode the same information. The model learns the sentence1-to-sentence2 mapping during fine-tuning, and the paper argues this mapping transfers to the DNA-to-protein alignment task.
What would settle it
Build a test set where negative protein sequences are generated by shuffling or mutating codons of true positives so that overall amino-acid composition and sequence length are nearly unchanged; if the fine-tuned model still scores near 81%, it has not learned the codon mapping.
Extended reading notes
Core claim
The paper claims that a GPT-2 model pretrained on sequences from seven natural languages plus DNA and protein, then fine-tuned on Chinese-Korean paraphrase pairs from PAWS-X, can classify whether a DNA sequence and a protein sequence correspond, reaching 81% accuracy on a test set where negatives are random proteins of low similarity to the true translation. The authors interpret this as the model rediscovering the genetic code through language transfer, without prior knowledge of the central dogma. They further report that pretraining on more languages improves transfer, that fine-tuning on structurally similar natural-language tasks helps, and that the effect is highly sensitive to random seeds during both pretraining and fine-tuning.
Load-bearing premise
The claim rests on the assumption that the DNA-protein test set actually measures knowledge of the genetic code; since the test uses only cleanly matching positives and random, low-similarity proteins as negatives, a classifier could score highly by detecting simpler statistical differences.
Editorial extensions
If this is right
- Multilingual pretraining improves zero-shot transfer to genomic tasks beyond English-only or biology-only pretraining.
- Fine-tuning on a structurally similar natural-language dataset can substitute for supervised training on the genomic task itself.
- Pre-trained models without fine-tuning already show above-random DNA-protein alignment accuracy, rising with the number of languages seen.
- The transfer is fragile: only a small fraction of random seeds find a good solution, so multiple runs or ensembling are needed to obtain the reported accuracy.
Reading between the lines
- A probe that tests whether the model's representations encode individual codons, rather than pair-level statistics, would directly test whether 'rediscovery of the central dogma' means codon-level knowledge.
- Using negative examples that preserve amino-acid composition and length but shuffle codons would separate genuine codon learning from simpler distributional cues.
- Averaging fine-tuned models across seeds, rather than selecting the best of 100, could turn the fragile transfer into a reliable method and would test whether the rare-seed behavior is real learning rather than luck.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper trains GPT-2 from scratch on a mixture of DNA sequences, protein sequences, and multilingual Wikipedia text, then fine-tunes it on the PAWS-X paraphrase-similarity dataset and evaluates it on a binary DNA-protein alignment task. The authors report a best accuracy of 81% on their "dna_protein_pair_rand" test set and interpret this as zero-shot rediscovery of the genetic code via natural-language transfer. Additional experiments examine seed sensitivity, the choice of fine-tuning language, and direct prediction with the pretrained model without fine-tuning.
Significance. If the central claim were valid, this would be a striking demonstration of cross-modal transfer from natural language to the genetic code and would merit broad attention. The authors deserve credit for releasing code, models, and data, and for including a candid stability analysis in Section 3.3 that quantifies large seed-to-seed variability. However, the headline 81% is the maximum of 100 fine-tuning runs, the test set is constructed in a way that allows shallow statistical shortcuts, and no trivial feature-based baselines are reported. The current evidence therefore does not support the claim that the model has rediscovered the central dogma.
major comments (4)
- [§2.3 / Supplementary §1.3] The test set is simplified in a way that makes the binary classes separable without learning the codon-to-amino-acid mapping. The construction removes the 100-bp flanks, keeps only the high-confidence ORF positives (approximately 3,000 of 8,533), and uses random proteins with similarity below 0.45 as negatives. Under these conditions, global sequence statistics such as GC content, codon-usage bias, and amino-acid composition differ systematically between matched and random pairs. The paper reports no baseline using such features (e.g., DNA k-mer frequencies, amino-acid composition, or alignment scores), so the 81% accuracy does not demonstrate knowledge of the genetic code.
- [§3.3 (Best finetune Language), Table 5, Fig. 4] The headline 81% is the maximum accuracy over 100 fine-tuning runs with different random seeds, and Fig. 4B shows that only about 9% of seeds exceed 70% on dna_protein_pair_rand, with most runs near chance. Thus the reported number is an extreme order statistic, not a typical or reproducible transfer outcome. The paper should report the full distribution (mean, median, quartiles) and either evaluate a fixed model or select hyperparameters on a held-out validation set rather than on the test set.
- [§2.4 / §3.4, Table 7] The label-swap rule in §2.4 uses test labels to choose the orientation for each run whenever accuracy is below 0.5, and §3.4 states that direct-prediction results "take the best result" over random seeds. Combined with best-of-100 selection, this makes the reported accuracy a fit to the test set rather than an unbiased estimate of model performance. The authors should separate model selection from evaluation and report the seed distribution without label reorientation based on test performance.
- [§4 / Abstract] The claim that the model "rediscovered the central dogma" is not established by the presented evidence. The pretraining corpus includes DNA and protein sequences, so the model could exploit statistical associations such as organism-specific codon usage or amino-acid composition without learning which codon encodes which amino acid. To support the rediscovery claim, the paper would need a control that removes composition cues, a comparison to nontrivial alignment-based baselines, or a direct probe of the learned codon-amino-acid mapping.
minor comments (4)
- [§3.3] The section number 3.3 is used twice, for "Training Stability" and for "Best finetune Language"; renumber to avoid confusion.
- [Table 5] The row labeled "ja-ka" appears to be a typo; it should probably read "ja-ja" or "ja-ko."
- [Fig. 2 caption] The caption says the layer maps 768 hidden units to 115,000 units as the token vocabulary size, but the GPT-2 small tokenizer has 50,257 tokens; please clarify the actual vocabulary size.
- [Fig. 3 / Table 3 / Table 5] The Fig. 3 caption reports approximately 76% accuracy for gpt2-gene-mulit on dna_protein_pair_rand, while Table 3 reports 0.76 and Table 5 reports 0.77 for the en-en fine-tuning setting; reconcile these numbers or clarify which run is shown.
Circularity Check
The headline 81% is the best of 100 test-set-selected fine-tuning runs with post-hoc label swapping, so the reported 'zero-shot' prediction is partly fitted to the test set.
-
fitted input called prediction
[Section 3.3 (Best finetune Language), Table 5 caption and Table 6]
"Since fine-tuning performance is highly sensitive to random seed selection, we conducted 100 training runs for each dataset, with random seeds ranging from 1 to 100, and then selected the best-performing model for comparison (Table.5)."
The 81% reported in the abstract and conclusion is the maximum test accuracy over 100 independently seed-initialized fine-tuning runs, and also over the language-pair grid summarized in Table 6. Selecting the run with the highest test accuracy makes the test labels part of the model-selection procedure; the reported number is an extreme order statistic, not the accuracy of a pre-specified model. The paper's own Table 4 and Figure 4 show that only 9% of seeds exceed 70% on dna_protein_pair_rand, so 0.81 is a rare upper-tail draw. Calling this a zero-shot transfer success is equivalent to fitting a seed hyperparameter to the test set and re-labeling the fit as a prediction.
-
fitted input called prediction
[Section 2.4 Model Evaluation]
"Since this is a binary classification task, the meaning of classification IDs 0 and 1 may differ during transfer. Therefore, if the test accuracy is below 0.5, we can simply swap label IDs 0 and 1."
The label orientation is chosen after observing the test accuracy, using test labels to decide whether 0 or 1 denotes alignment. For each seed this yields max(acc, 1-acc), which guarantees at least chance-level performance and inflates a random run to an expected 0.75 even with no real signal. The direction of the label mapping is therefore fitted to the same labels that are later counted as correct predictions, so the reported accuracy is partly constructed from the test labels rather than predicted from the model alone.
full rationale
The central derivation is not circular in the sense of an equation reducing to its own input or a result borrowed from a self-citation: the paper trains and evaluates its own models, and reference [24] is background motivation rather than the evidence for the 81% claim. However, the key quantitative claim fails the 'fitted input called prediction' test. The 81% figure is selected as the best of 100 fine-tuning seeds, and also as the best of the fine-tuning language pairs in Table 6, while the binary-label orientation is decided by swapping labels whenever test accuracy is below 0.5. Both choices use the test labels to select the reported model and its mapping, so the headline number is an optimized test-set statistic rather than the accuracy of a fixed zero-shot model. Additional concerns about the test construction, such as using only the clearest positive ORFs and random-protein negatives with similarity below 0.45, weaken the inference that the model learned the genetic code, but those are experimental confounds rather than circular reductions. Overall, the paper has independent empirical content, but its marquee 'rediscover the central dogma' result is substantially fitted to the test set; this is partial circularity, not a full self-referential derivation.
Assumptions & free parameters
free parameters (4)
- fine-tuning random seed (best of 100) =
not reported (selected to maximize test accuracy)
- negative protein similarity threshold =
0.45
- positive example filtering =
approx. 3,000 of 8,533 original positives
- number of pre-training epochs =
3 to 5
assumptions (5)
- domain assumption PAWS-X sentence-pair similarity judgment is structurally analogous to DNA-protein alignment.
- domain assumption BPE tokenization of DNA and protein sequences preserves the information needed for alignment.
- ad hoc to paper The model can determine the boundary between concatenated DNA and protein sequences without a separator.
- ad hoc to paper A random protein with similarity <0.45 is a valid negative example for a DNA-protein alignment task.
- domain assumption The central dogma can be faithfully represented as binary DNA-protein pair matching.
Cite this review
Pith. "Pith review of Find Central Dogma Again: Leveraging Multilingual Transfer in Large Language Models." pith.science (2026). https://pith.science/paper/DZAJQS75
@misc{pith2026250206253,
author = {Pith},
title = {Pith review of: Find Central Dogma Again: Leveraging Multilingual Transfer in Large Language Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/DZAJQS75}},
note = {Machine review of arXiv:2502.06253}
}
read the original abstract
In recent years, large language models (LLMs) have achieved state-of-the-art results in various biological sequence analysis tasks, such as sequence classification, structure prediction, and function prediction. Similar to advancements in AI for other scientific fields, deeper research into biological LLMs has begun to focus on using these models to rediscover important existing biological laws or uncover entirely new patterns in biological sequences. This study leverages GPT-like LLMs to utilize language transfer capabilities to rediscover the genetic code rules of the central dogma. In our experimental design, we transformed the central dogma into a binary classification problem of aligning DNA sequences with protein sequences, where positive examples are matching DNA and protein sequences, and negative examples are non-matching pairs. We first trained a GPT-2 model from scratch using a dataset comprising protein sequences, DNA sequences, and sequences from languages such as English and Chinese. Subsequently, we fine-tuned the model using the natural language sentences similarity judgment dataset from PAWS-X. When tested on a dataset for DNA and protein sequence alignment judgment, the fine-tuned model achieved a classification accuracy of 81%. The study also analyzed factors contributing to this zero-shot capability, including model training stability and types of training data. This research demonstrates that LLMs can, through the transfer of natural language capabilities and solely relying on the analysis of sequences themselves, rediscover the central dogma without prior knowledge of it. This study bridges natural language and genetic language, opening a new door for AI-driven biological research.
Reference graph
Works this paper leans on
-
[1]
MachineTranslation–TheDNA→proteinencodingprocessisanalogoustotranslating asourcelanguageintoatargetlanguage
-
[2]
Semantic Similarity Texts – Similar to machine translation, this includes sentence pairs with equivalent meanings, such as two semantically similar English sentences or an English sentencepairedwithitssemanticallyequivalentFrenchtranslation
-
[3]
Speech-to-Text – The process of encoding DNA into proteins resembles converting speechsignalsintorecognizedtext
-
[4]
Code Generation – DNA-to-protein conversion follows strict rule-based mappings, similartothetransformationbetweennaturallanguageandprogrammingcode. Given the availability and diversity of datasets, we chose semantic similarity text datasets as candidates for language transfer learning. One of the most representative datasets in this category is PAWS-X, a ...
-
[5]
Multimodal Data Training During the pretraining phase, using a diverse set of training data—including multiple natural languages (not just English) as well as DNA andprotein sequences—enablesthe model to learna more comprehensive representation across different languages and symbolic systems. This approach allows the model not only to grasp English gramma...
-
[6]
Unified Tokenization and Encoding Methods (e.g., BPE) Using a unified tokenization (or encoding) method ensures consistencywhen converting different types of data (natural language and genetic sequences) into model inputs. For example, Byte Pair Encoding (BPE) can segment both English text and DNA/protein sequences into subwords or sub-sequences, forming ...
-
[7]
accidentalalignmentofthinkingpatterns
Fine-tuning on Natural Language Datasets with Structural Similarity to Genetic Sequence Tasks The structure of the dataset used for fine-tuning is critical for transfer effectiveness. Genetic sequence tasks often follow fixed patterns, such as DNA-protein alignment problems. Selecting a natural language dataset with a similar structure for fine-tuning all...
-
[8]
Cross-domain transfer is highly sensitive: Transferring a model between English and biological sequences requires finding those rare solutions that can accommodate both language structures. However, such solutions are extremely scarce in the optimization space andonlyappearunderafewspecificrandomseeds
Show all 38 references
-
[9]
crack the code
The alignment of thinking patterns is accidental: The model's English-based reasoning onlyproducesgoodtransferresultswhenit"happens"tooverlapwiththepatternsofDNAor protein sequences under certain conditions. This suggests that cross-domain transfer may require additional regul...
-
[10]
Training agents to autonomously search and optimize sequences to discover newfunctionalgenesorproteinstructures
CombiningPromptEngineeringandAgentsforBiologicalResearch Utilizing Prompt Engineering and Agents to enable large models to actively explorehiddenpatternsingeneticsequences.Forexample: Designing structured prompts for DNA or protein sequences to predict potentialfunctionalr...
-
[11]
Utilizing cross-modal AI to integrate imaging (e.g., protein structures), text, and sequence data for more precise disease prediction and personalized medicine analysis
ExpandingAIApplicationsinBiologyThroughCross-ModalLearning Integrating biological text data (e.g., research papers, patents) with experimental datatoenhancethemodel’sreasoningcapabilities.Forexample: Enabling large models to automatically read biomedical literature and gen...
-
[12]
Zhou, et al., Dnabert-2: Efficient foundation model and benchmark for multi-species genome,arXivpreprintarXiv:2306.15006(2023)
Z. Zhou, et al., Dnabert-2: Efficient foundation model and benchmark for multi-species genome,arXivpreprintarXiv:2306.15006(2023)
2023 arXiv
-
[13]
Nguyen, et al., Hyenadna: Long-range genomic sequence modeling at single nucleotide resolution,Advancesinneuralinformationprocessingsystems36(2024)
E. Nguyen, et al., Hyenadna: Long-range genomic sequence modeling at single nucleotide resolution,Advancesinneuralinformationprocessingsystems36(2024)
2024
-
[14]
F.Yang,etal.,scbertasalarge-scalepretraineddeeplanguagemodelforcelltypeannotation ofsingle-cellrna-seqdata,NatureMachineIntelligence4,852–866(2022)
2022
-
[15]
A. Elnaggar, et al., Prottrans: Toward understanding the language of life through selfsupervised learning, IEEE transactions on pattern analysis and machine intelligence 44, 7112–7127(2021)
2021
-
[16]
N.Brandes,D.Ofer,Y.Peleg,N.Rappoport,M.Linial,Proteinbert:auniversaldeeplearning modelofproteinsequenceandfunction,Bioinformatics38,2102–2110(2022)
2022
-
[17]
Lin, et al., Evolutionary-scale prediction of atomic-level protein structure with a language model,Science379,1123–1130(2023)
Z. Lin, et al., Evolutionary-scale prediction of atomic-level protein structure with a language model,Science379,1123–1130(2023)
2023
-
[18]
& Davuluri, R
Ji, Y., Zhou, Z., Liu, H. & Davuluri, R. V. DNABERT: Pre-trained bidirectional encoder representations from transformers model for DNA-language in genome. Bioinformatics 37, 2112–2120(2021).doi:doi.org/10.1093/bioinformatics/btab083
2021 doi
-
[19]
doi: https://doi.org/10.1101/2023.06.12.544594
Veniamin Fishman, et al.GENA-LM: A Family of Open-Source Foundational DNA Language Models for Long Sequences.BioRxiv 2023.06.12.544594. doi: https://doi.org/10.1101/2023.06.12.544594
2023 doi
-
[20]
Daoan Zhang, et al.DNAGPT: A Generalized Pretrained Tool for Multiple DNA Sequence AnalysisTasks.BioRxiv2023.07.11.548628;doi:https://doi.org/10.1101/2023.07.11.548628
2023 doi
-
[21]
Wang, B. et al. Pre-trained Language Models in Biomedical Domain: A Systematic Survey. ACM Comput. Surv.56,1–52(2023).doi:doi.org/10.1145/3611651
2023 doi
-
[22]
& Gao, G
Chen, Z., Wei, L. & Gao, G. Foundation models for bioinformatics. Quant. Biol. (2024) doi:10.1002/qub2.69
2024 doi
-
[24]
Nguyen E, Poli M, Durrant MG, Kang B, Katrekar D, Li DB, Bartie LJ, Thomas AW, King SH,BrixiG,SullivanJ.Sequencemodelinganddesignfrommoleculartogenomescalewith Evo.Science.2024Nov15;386(6723):eado9336
-
[25]
DNA language models are powerful predictors of genome-wide variant effects
Benegas G, Batra SS, Song YS. DNA language models are powerful predictors of genome-wide variant effects. Proceedings of the National Academy of Sciences. 2023 Oct 31;120(44):e2311219120
2023
-
[26]
ShaoB,YanJ.Along-context languagemodelfordecipheringandgenerating bacteriophage genomes.NatureCommunications.2024Oct30;15(1):9392
-
[27]
Language models enable zero-shot prediction of the effects of mutations on protein function
Meier J, Rao R, Verkuil R, Liu J, Sercu T, Rives A. Language models enable zero-shot prediction of the effects of mutations on protein function. Advances in neural information processingsystems.2021Dec6;34:29287-303
-
[28]
DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA
Patel A, Singhal A, Wang A, Pampari A, Kasowski M, Kundaje A. DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA. arXiv preprintarXiv:2412.05430.2024Dec6
- [29]
-
[30]
Alexis Conneau, Shijie Wu, Haoran Li, Luke Zettlemoyer, and Veselin Stoyanov. 2020. EmergingCross-lingualStructureinPretrainedLanguageModels.In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6022–6034, Online. AssociationforCompu...
2020
- [31]
-
[32]
Hetong Wang, Pasquale Minervini, and Edoardo Ponti. 2024. Probing the Emergence of Cross-lingual Alignment during LLM Training. In Findings of the Association for Computational Linguistics: ACL 2024, pages 12159–12173,Bangkok, Thailand. Association forComputationalLinguistics
2024
-
[33]
Junjie Hu, etc.XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingualGeneralization.arXiv(2020)https://arxiv.org/abs/2003.11080
2020 arXiv
-
[34]
Exploring the relationship between alignment and cross-lingual transfer in multilingual transformers
Gaschi F, Cerda P, Rastin P, Toussaint Y. Exploring the relationship between alignment and cross-lingual transfer in multilingual transformers. Findings of the Association for ComputationalLinguistics:ACL2023
-
[35]
LiangW.CanlinguistsbetterunderstandDNA?.arXivpreprintarXiv:2412.07678.2024Dec 10
-
[36]
LucaOne: Generalized Biological Foundation Model with Unified Nucleic Acid and Protein Language
Yong He, et al. LucaOne: Generalized Biological Foundation Model with Unified Nucleic Acid and Protein Language. BioRxiv 2024.05.10.592927; doi: https://doi.org/10.1101/2024.05.10.592927
2024 doi
-
[37]
Zeyuan Wang, etc. 2024. InstructProtein: Aligning Human and Protein Language via Knowledge Instruction. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1114–1136, Bangkok, Thailand. AssociationforComputa...
2024
-
[38]
paperhuggingface,https://huggingface.co/dnagpt/central_dogma Data availability section Sequence data that support the findings of this study have been deposited in huggingface.The fine-tuning/testdataislocatedathttps://huggingface.co/datasets/dnagpt/gene_lan_transfer,andthe pr...
-
[39]
A custom-constructed dataset, where protein sequences are randomly sampled from UniProtKB/Swiss-Prot, and their corresponding DNA sequences are retrieved. For non-aligned pairs, the DNA sequence remains unchanged, while the protein sequence is randomly sampled from the protein...
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.