REVIEW 4 major objections 4 minor 1 cited by
Large Language Model is Secretly a Protein Sequence Optimizer
T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read A general-purpose large language model, used as the mutation and crossover operator in a directed-evolution loop, can propose higher-fitness protein variants on the fly without any fine-tuning.
desk verdict Promising zero-shot LLM evolution, but the baseline compares fitness-conditioned prompts with blind random mutation, so the central claim isn't yet isolated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the LLM-as-proposer evolutionary loop. The LLM is treated as a black-box stochastic mutation/crossover operator: given a pair of parent sequences—with their fitness scores included in the prompt—it outputs a proposed variant, and if the model returns nothing valid, the algorithm falls back to random crossover or mutation. Selection is elitist top-k on the merged population, and constraints are enforced by rejection sampling plus natural-language instructions in the prompt. What carries the argument is the bias of the LLM's proposals: unlike uniform random edits, they cluster around plausible high-fitness substitutions, and this bias is what makes the search efficient on rugged fitness landscapes.
What would settle it
Run the same LLM-guided loop on a protein family that was not present in the LLM's pretraining corpus (for example, a newly designed enzyme with no close public homologs) and compare against random-mutation evolution; if the advantage disappears, the effect is memorization rather than optimization. Alternatively, remove or shuffle the fitness scores in the prompt and check whether proposal quality degrades.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that protein fitness optimization does not require a model explicitly trained on protein sequences or fitness labels: an instruction-tuned LLM prompted with the wild-type sequence, two parent sequences, and their fitness scores will propose variants whose fitness distribution is substantially better than random mutation or random recombination. The authors embed this LLM into an evolutionary algorithm—initialize a pool, sample parent pairs, ask the LLM for a crossover or mutation proposal, merge with the old pool, and keep the top-k by oracle fitness—and compare it against a classical evolutionary algorithm under identical population sizes, iterations, initial pools, and selection rules. On the three complex landscapes the LLM version consistently wins; on the near-linear GB1 and TrpB landscapes it is competitive and occasionally loses a seed. The paper further claims the same mechanism works when per-iteration Hamming distance is capped, when the total experimental budget is fixed, and when fitness and Hamming distance are optimized jointly as a two-objective Pareto problem.
Load-bearing premise
The paper's claim depends on the assumption that the LLM's proposals are driven by genuinely useful knowledge about what makes proteins fit, rather than by memorized training examples or by the fitness scores placed in the prompt.
Editorial extensions
If this is right
- An off-the-shelf LLM can be dropped into existing directed-evolution pipelines as a proposal generator, with no training or fine-tuning step.
- The method transfers across fitness oracles of very different types—exact wet-lab measurements, statistical-energy models, and learned surrogates—so it is not tied to one scoring function.
- Experimental budgets can be encoded directly as natural-language constraints (Hamming distance to wild type or to previously tested sequences), and the LLM respects them, reducing wasted experiments.
- Fitness and mutational distance can be optimized jointly, with the LLM discovering Pareto-frontier sequences comparable to or better than random evolution.
- On rugged, high-dimensional landscapes the advantage over random mutagenesis grows, suggesting LLM proposals are most useful precisely where classical directed evolution stalls.
Reading between the lines
- The paper does not ablate whether the LLM's success comes from reasoning about sequence-function relationships or from retrieving memorized high-fitness variants of these public benchmark proteins; a test on a protein family absent from pretraining data, or with fitness scores stripped from the prompt, would separate the two.
- If the mechanism is largely memorization, the method's utility would generalize only to proteins well represented in the LLM's training corpus; if it is genuine in-context reasoning, the same loop could be applied to newly designed or proprietary proteins.
- The prompt includes parent fitness values, so a plausible alternative explanation is that the LLM simply interpolates among high-scoring parents; this could be probed by giving the same parents without scores or with shuffled scores.
- The framework could be combined with active learning or surrogate-model guidance, where the LLM proposes candidates that the surrogate scores and the experimenter validates, potentially reducing wet-lab rounds further.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes an LLM-guided directed evolution framework in which Llama-3.1-8B-Instruct proposes new protein variants from pairs of parent sequences, after which the population is rank-selected by fitness. The framework is evaluated on five landscapes (GB1, TrpB, a synthetic SLIP landscape from PDB 3bfo, AAV, and GFP) under single-objective, Hamming-distance-constrained, budget-constrained, and multi-objective settings. The central claim is that a general-purpose LLM can act as a zero-shot mutation/crossover operator, outperforming a standard evolutionary algorithm with random mutation and crossover.
Significance. The paper addresses a timely and practically important question: whether a pretrained LLM can generate useful protein variants without fine-tuning. Its strengths include coverage of three oracle types (exact DMS, synthetic Potts-model, and ML-predicted fitness), a detailed prompt appendix (Appendix A.3), an iteration-count ablation (Table 6), and systematic treatment of constrained and multi-objective optimization. If the main claim were cleanly established, the framework would be a useful addition to ML-guided directed evolution. However, the principal comparison conflates two effects: the LLM receives parent fitness scores in its prompt while the EA baseline receives none, and one benchmark (GB1) shows the EA winning several reported columns. The retrieval-risk from public DMS data is also not addressed. These issues are load-bearing for the claim that the LLM is 'secretly' an optimizer, so the current evidence is suggestive but not yet conclusive.
major comments (4)
- [Appendix A.3, Table 2] The prompt shown in Appendix A.3 provides each parent sequence with an explicit fitness value (e.g., 'fitness score: 0.0018'), while the EA baseline receives only the sequence. This gives the LLM a trivial fitness-conditioning advantage: it can edit preferentially from the higher-fitness parent or preserve high-fitness residues, whereas random mutation and crossover cannot exploit fitness at proposal time. The central comparison in Table 2 therefore does not isolate LLM protein knowledge as the source of improvement. Please add an ablation that removes fitness scores from the prompt, and add a fitness-conditioned random baseline (e.g., mutation rate or parent-choice probability proportional to fitness) to test whether the observed gap persists without LLM knowledge.
- [Table 2, Section 4.2] On GB1 with population sizes 32 and 48, Table 2 reports EA Top-1 fitness of 5.38 +/- 1.77 and 4.88 +/- 0.33 versus Ours of 4.34 +/- 0.53 and 4.31 +/- 0.82. This contradicts the introduction's claim that the LLM proposals are 'much more efficient' in the single-objective setting. The explanation in Section 4.2 that 'EA only outperforms one of three random seeds' is not supported by the reported means and standard deviations; with three seeds and overlapping intervals, seed-level trajectories and a statistical test are needed before the cross-dataset claim can be accepted.
- [Algorithm 1, Section 3] The pseudocode's control flow is ambiguous. When the LLM returns None, the code appends C(s0,s1) but then continues to sample r and may append either M(s0) or 'proposed seq' (which is None) without an explicit guard. When the LLM returns a valid sequence, that sequence is only appended in the 'else' branch after a uniform random draw. As written, it is impossible to determine what fraction of offspring in the reported experiments actually come from the LLM rather than from the random crossover/mutation fallback. Please rewrite the pseudocode and report the average fraction of LLM-generated proposals that enter the offspring set per iteration.
- [Section 4.1, Table 1] GB1, TrpB, AAV, and GFP are public deep-mutational-scanning datasets, and the SLIP landscape is constructed from a public PDB MSA. It is plausible that Llama-3.1's pretraining corpus contains these sequences and associated fitness information, so the model may propose high-fitness variants by retrieval rather than by on-the-fly optimization. As a correctness-risk control, please report how often the LLM's proposed sequences coincide with known high-fitness variants in each benchmark, or evaluate on a synthetic landscape that did not exist before the model's training cutoff. This would make the retrieval risk measurable without requiring a full audit of the pretraining corpus.
minor comments (4)
- [Section 4.2, Table 4] The sentence that 'the maximum Hamming distance of sequences proposed by EA does not exceed 5' is used to explain identical EA rows for H=5 and H=10, but no evidence is provided. Please report the distribution of edit distances observed for EA proposals or otherwise support this claim.
- [Appendix A.3] In the prompt text, 'minimizing humming distance' should be 'Hamming distance'.
- [Table 1] The label 'AA V' should be 'AAV', and the meaning of 'N/A' in the space-size and target-range columns would benefit from a footnote.
- [Section 4.2] The statement that 'linear relationships between positions are less likely in more complex landscapes' is informal; consider reporting an epistasis measure or phrasing this as a testable hypothesis rather than a justification.
Circularity Check
No significant circularity; the empirical comparison is self-contained and does not reduce to its inputs by construction.
full rationale
The paper's central claim is empirical: pre-trained LLM proposals used inside a directed-evolution loop outperform a random-mutation/crossover evolutionary algorithm on fixed fitness oracles (DMS ground truth, SLIP, and ML fitness models). Neither the LLM nor the EA is fitted to the oracle; selection and ranking procedures are identical for both arms, and the reported fitness differences come from external evaluation, not from any identity between proposed sequences and oracle outputs. The only potentially circular-adjacent issue is that Appendix A.3 includes parent fitness scores in the LLM prompt, so the LLM can condition proposals on fitness while the EA baseline cannot; this makes the comparison uneven and the attribution of gains to pre-trained protein knowledge a confound risk, but it does not make the result equivalent to its inputs by construction. There are no load-bearing self-citations, imported uniqueness theorems, or ansatz-smuggling citations. Accordingly, the appropriate circularity finding is score 0, with the caveat that attribution and baseline fairness are separate correctness concerns.
Assumptions & free parameters
assumptions (3)
- domain assumption The fitness landscapes and ML oracles used are representative of real protein fitness landscapes, and the ML oracle predictions are accurate enough to rank variants.
- ad hoc to paper Llama-3.1-8B-Instruct's proposals are generated from reasoning about protein fitness, not from memorized high-fitness variants in its pre-training corpus.
- domain assumption Hamming distance is an appropriate measure of edit distance for the constraints and budget experiments.
Cite this review
Pith. "Pith review of Large Language Model is Secretly a Protein Sequence Optimizer." pith.science (2026). https://pith.science/paper/FZAEOBS2
@misc{pith2026250109274,
author = {Pith},
title = {Pith review of: Large Language Model is Secretly a Protein Sequence Optimizer},
year = {2026},
howpublished = {\url{https://pith.science/paper/FZAEOBS2}},
note = {Machine review of arXiv:2501.09274}
}
read the original abstract
We consider the protein sequence engineering problem, which aims to find protein sequences with high fitness levels, starting from a given wild-type sequence. Directed evolution has been a dominating paradigm in this field which has an iterative process to generate variants and select via experimental feedback. We demonstrate large language models (LLMs), despite being trained on massive texts, are secretly protein sequence optimizers. With a directed evolutionary method, LLM can perform protein engineering through Pareto and experiment-budget constrained optimization, demonstrating success on both synthetic and experimental fitness landscapes.
Figures
Figures from the paper (4 more)
Forward citations
Cited by 1 Pith paper
-
A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving
A literature survey that classifies LLM-based optimization research into modeling and solving, with solving divided into LLMs as optimizers, low-level components, and high-level managers.
Reference graph
Works this paper leans on
-
[1]
Design by directed evolution
Frances H Arnold. Design by directed evolution. Accounts of chemical research, 31 0 (3): 0 125--131, 1998
1998
-
[2]
Conditioning by adaptive sampling for robust design
David Brookes, Hahnbeom Park, and Jennifer Listgarten. Conditioning by adaptive sampling for robust design. In International conference on machine learning, pp.\ 773--782. PMLR, 2019
work page 2019
-
[3]
Deep diversification of an aav capsid protein by machine learning
Drew H Bryant, Ali Bashir, Sam Sinai, Nina K Jain, Pierce J Ogden, Patrick F Riley, George M Church, Lucy J Colwell, and Eric D Kelsic. Deep diversification of an aav capsid protein by machine learning. Nature Biotechnology, 39 0 (6): 0 691--696, 2021
work page 2021
-
[4]
Llms are highly-constrained biophysical sequence optimizers
Angelica Chen, Samuel D Stanton, Robert G Alberstein, Andrew M Watkins, Richard Bonneau, Vladimir Gligorijevi, Kyunghyun Cho, and Nathan C Frey. Llms are highly-constrained biophysical sequence optimizers. arXiv preprint arXiv:2410.22296, 2024
-
[5]
Flip: Benchmark tasks in fitness landscape inference for proteins
Christian Dallago, Jody Mou, Kadina E Johnston, Bruce J Wittmann, Nicholas Bhattacharya, Samuel Goldman, Ali Madani, and Kevin K Yang. Flip: Benchmark tasks in fitness landscape inference for proteins. bioRxiv, pp.\ 2021--11, 2021
2021
-
[6]
Deep mutational scanning: a new style of protein science
Douglas M Fowler and Stanley Fields. Deep mutational scanning: a new style of protein science. Nature methods, 11 0 (8): 0 801--807, 2014
work page 2014
-
[7]
Mutation effects predicted from sequence co-variation
Thomas A Hopf, John B Ingraham, Frank J Poelwijk, Charlotta PI Sch \"a rfe, Michael Springer, Chris Sander, and Debora S Marks. Mutation effects predicted from sequence co-variation. Nature biotechnology, 35 0 (2): 0 128--135, 2017
work page 2017
-
[8]
Biological sequence design with gflownets
Moksh Jain, Emmanuel Bengio, Alex Hernandez-Garcia, Jarrid Rector-Brooks, Bonaventure FP Dossou, Chanakya Ajit Ekbote, Jie Fu, Tianyu Zhang, Michael Kilgour, Dinghuai Zhang, et al. Biological sequence design with gflownets. In International Conference on Machine Learning, pp.\ 9786--9801. PMLR, 2022
2022
Show all 27 references
-
[9]
A combinatorially complete epistatic fitness landscape in an enzyme active site
Kadina E Johnston, Patrick J Almhjell, Ella J Watkins-Dulaney, Grace Liu, Nicholas J Porter, Jason Yang, and Frances H Arnold. A combinatorially complete epistatic fitness landscape in an enzyme active site. Proceedings of the National Academy of Sciences, 121 0 (32): 0 e24004...
2024
-
[10]
Highly accurate protein structure prediction with alphafold
John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Z \' dek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. nature, 596 0 (7873): 0 583--589, 2021
2021
-
[11]
Improving protein optimization with smoothed fitness landscapes
Andrew Kirjner, Jason Yim, Raman Samusevich, Shahar Bracha, Tommi S Jaakkola, Regina Barzilay, and Ila R Fiete. Improving protein optimization with smoothed fitness landscapes. In The Twelfth International Conference on Learning Representations, 2023
2023
-
[12]
Evolutionary-scale prediction of atomic-level protein structure with a language model
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science, 379 0 (6637): 0 1123--1130, 2023
2023
-
[13]
Generative design of functional metal complexes utilizing the internal knowledge of large language models
Jieyu Lu, Zhangde Song, Qiyuan Zhao, Yuanqi Du, Yirui Cao, Haojun Jia, and Chenru Duan. Generative design of functional metal complexes utilizing the internal knowledge of large language models. arXiv preprint arXiv:2410.18136, 2024
-
[14]
Proximal exploration for model-guided protein sequence design
Zhizhou Ren, Jiahan Li, Fan Ding, Yuan Zhou, Jianzhu Ma, and Jian Peng. Proximal exploration for model-guided protein sequence design. In International Conference on Machine Learning, pp.\ 18520--18536. PMLR, 2022
2022
-
[15]
Exploring protein fitness landscapes by directed evolution
Philip A Romero and Frances H Arnold. Exploring protein fitness landscapes by directed evolution. Nature reviews Molecular cell biology, 10 0 (12): 0 866--876, 2009
2009
-
[16]
Local fitness landscape of the green fluorescent protein
Karen S Sarkisyan, Dmitry A Bolotin, Margarita V Meer, Dinara R Usmanova, Alexander S Mishin, George V Sharonov, Dmitry N Ivankov, Nina G Bozhanova, Mikhail S Baranov, Onuralp Soylemez, et al. Local fitness landscape of the green fluorescent protein. Nature, 533 0 (7603): 0 39...
2016
-
[17]
Tuned fitness landscapes for benchmarking model-guided protein design
Neil Thomas, Atish Agarwala, David Belanger, Yun S Song, and Lucy J Colwell. Tuned fitness landscapes for benchmarking model-guided protein design. bioRxiv, pp.\ 2022--10, 2022
2022
-
[18]
Protein design by directed evolution guided by large language models
Thanh VT Tran and Truong Son Hy. Protein design by directed evolution guided by large language models. IEEE Transactions on Evolutionary Computation, 2024
2024
-
[19]
Efficient evolutionary search over chemical space with large language models
Haorui Wang, Marta Skreta, Cher-Tian Ser, Wenhao Gao, Lingkai Kong, Felix Strieth-Kalthoff, Chenru Duan, Yuchen Zhuang, Yue Yu, Yanqiao Zhu, et al. Efficient evolutionary search over chemical space with large language models. arXiv preprint arXiv:2406.16976, 2024
2024 arXiv
-
[20]
Adaptation in protein fitness landscapes is facilitated by indirect paths
Nicholas C Wu, Lei Dai, C Anders Olson, James O Lloyd-Smith, and Ren Sun. Adaptation in protein fitness landscapes is facilitated by indirect paths. Elife, 5: 0 e16965, 2016
2016
-
[21]
Active learning-assisted directed evolution
Jason Yang, Ravi G Lal, James C Bowden, Raul Astudillo, Mikhail A Hameedi, Sukhvinder Kaur, Matthew Hill, Yisong Yue, and Frances H Arnold. Active learning-assisted directed evolution. bioRxiv, pp.\ 2024--07, 2024
2024
-
[22]
Machine-learning-guided directed evolution for protein engineering
Kevin K Yang, Zachary Wu, and Frances H Arnold. Machine-learning-guided directed evolution for protein engineering. Nature methods, 16 0 (8): 0 687--694, 2019
2019
-
[23]
Protein language models learn evolutionary statistics of interacting sequence motifs
Zhidian Zhang, Hannah K Wayment-Steele, Garyk Brixi, Haobo Wang, Dorothee Kern, and Sergey Ovchinnikov. Protein language models learn evolutionary statistics of interacting sequence motifs. Proceedings of the National Academy of Sciences, 121 0 (45): 0 e2406285121, 2024
2024
-
[24]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
-
[25]
@esa (Ref
\@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...
-
[26]
\@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...
-
[27]
@open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.