Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Large Language Model is Secretly a Protein Sequence Optimizer

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Pith's one-line read A general-purpose large language model, used as the mutation and crossover operator in a directed-evolution loop, can propose higher-fitness protein variants on the fly without any fine-tuning.

desk verdict Promising zero-shot LLM evolution, but the baseline compares fitness-conditioned prompts with blind random mutation, so the central claim isn't yet isolated. read the letter →

arxiv 2501.09274 v2 pith:FZAEOBS2 submitted 2025-01-16 cs.LG cs.AIq-bio.QM

classification cs.LGcs.AIq-bio.QM
keywords proteinsequenceoptimizationdirectedevolutionlargelanguagemodelszero-shotmutationoperatorfitnesslandscapeParetodeepmutationalscanningevolutionaryalgorithm
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a general-purpose large language model (Llama-3.1-8B-Instruct), used with no fine-tuning, can act as the mutation and crossover operator inside a directed-evolution loop for protein sequence optimization. The method repeatedly asks the LLM to propose new variants from pairs of parent sequences, scores each proposal with a fitness oracle, and keeps the top-ranked candidates for the next round. Across five fitness landscapes—two exact experimental deep-mutational-scanning datasets (GB1, TrpB), one synthetic Potts-model landscape (Syn-3bfo), and two machine-learning oracles (AAV, GFP)—the LLM-guided search finds higher-fitness candidates than a standard evolutionary algorithm with random mutation and crossover, with the largest advantages on the more nonlinear landscapes. The paper also shows the same loop can obey Hamming-distance budgets, experimental iteration budgets, and multi-objective Pareto targets. If the claim holds, a capable text model can serve as a zero-shot sequence optimizer, potentially accelerating wet-lab directed evolution without training a domain-specific model.

What carries the argument

The central object is the LLM-as-proposer evolutionary loop. The LLM is treated as a black-box stochastic mutation/crossover operator: given a pair of parent sequences—with their fitness scores included in the prompt—it outputs a proposed variant, and if the model returns nothing valid, the algorithm falls back to random crossover or mutation. Selection is elitist top-k on the merged population, and constraints are enforced by rejection sampling plus natural-language instructions in the prompt. What carries the argument is the bias of the LLM's proposals: unlike uniform random edits, they cluster around plausible high-fitness substitutions, and this bias is what makes the search efficient on rugged fitness landscapes.

What would settle it

Run the same LLM-guided loop on a protein family that was not present in the LLM's pretraining corpus (for example, a newly designed enzyme with no close public homologs) and compare against random-mutation evolution; if the advantage disappears, the effect is memorization rather than optimization. Alternatively, remove or shuffle the fitness scores in the prompt and check whether proposal quality degrades.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that protein fitness optimization does not require a model explicitly trained on protein sequences or fitness labels: an instruction-tuned LLM prompted with the wild-type sequence, two parent sequences, and their fitness scores will propose variants whose fitness distribution is substantially better than random mutation or random recombination. The authors embed this LLM into an evolutionary algorithm—initialize a pool, sample parent pairs, ask the LLM for a crossover or mutation proposal, merge with the old pool, and keep the top-k by oracle fitness—and compare it against a classical evolutionary algorithm under identical population sizes, iterations, initial pools, and selection rules. On the three complex landscapes the LLM version consistently wins; on the near-linear GB1 and TrpB landscapes it is competitive and occasionally loses a seed. The paper further claims the same mechanism works when per-iteration Hamming distance is capped, when the total experimental budget is fixed, and when fitness and Hamming distance are optimized jointly as a two-objective Pareto problem.

Load-bearing premise

The paper's claim depends on the assumption that the LLM's proposals are driven by genuinely useful knowledge about what makes proteins fit, rather than by memorized training examples or by the fitness scores placed in the prompt.

Editorial extensions

If this is right

  • An off-the-shelf LLM can be dropped into existing directed-evolution pipelines as a proposal generator, with no training or fine-tuning step.
  • The method transfers across fitness oracles of very different types—exact wet-lab measurements, statistical-energy models, and learned surrogates—so it is not tied to one scoring function.
  • Experimental budgets can be encoded directly as natural-language constraints (Hamming distance to wild type or to previously tested sequences), and the LLM respects them, reducing wasted experiments.
  • Fitness and mutational distance can be optimized jointly, with the LLM discovering Pareto-frontier sequences comparable to or better than random evolution.
  • On rugged, high-dimensional landscapes the advantage over random mutagenesis grows, suggesting LLM proposals are most useful precisely where classical directed evolution stalls.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not ablate whether the LLM's success comes from reasoning about sequence-function relationships or from retrieving memorized high-fitness variants of these public benchmark proteins; a test on a protein family absent from pretraining data, or with fitness scores stripped from the prompt, would separate the two.
  • If the mechanism is largely memorization, the method's utility would generalize only to proteins well represented in the LLM's training corpus; if it is genuine in-context reasoning, the same loop could be applied to newly designed or proprietary proteins.
  • The prompt includes parent fitness values, so a plausible alternative explanation is that the LLM simply interpolates among high-scoring parents; this could be probed by giving the same parents without scores or with shuffled scores.
  • The framework could be combined with active learning or surrogate-model guidance, where the LLM proposes candidates that the surrogate scores and the experimenter validates, potentially reducing wet-lab rounds further.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes an LLM-guided directed evolution framework in which Llama-3.1-8B-Instruct proposes new protein variants from pairs of parent sequences, after which the population is rank-selected by fitness. The framework is evaluated on five landscapes (GB1, TrpB, a synthetic SLIP landscape from PDB 3bfo, AAV, and GFP) under single-objective, Hamming-distance-constrained, budget-constrained, and multi-objective settings. The central claim is that a general-purpose LLM can act as a zero-shot mutation/crossover operator, outperforming a standard evolutionary algorithm with random mutation and crossover.

Significance. The paper addresses a timely and practically important question: whether a pretrained LLM can generate useful protein variants without fine-tuning. Its strengths include coverage of three oracle types (exact DMS, synthetic Potts-model, and ML-predicted fitness), a detailed prompt appendix (Appendix A.3), an iteration-count ablation (Table 6), and systematic treatment of constrained and multi-objective optimization. If the main claim were cleanly established, the framework would be a useful addition to ML-guided directed evolution. However, the principal comparison conflates two effects: the LLM receives parent fitness scores in its prompt while the EA baseline receives none, and one benchmark (GB1) shows the EA winning several reported columns. The retrieval-risk from public DMS data is also not addressed. These issues are load-bearing for the claim that the LLM is 'secretly' an optimizer, so the current evidence is suggestive but not yet conclusive.

major comments (4)
  1. [Appendix A.3, Table 2] The prompt shown in Appendix A.3 provides each parent sequence with an explicit fitness value (e.g., 'fitness score: 0.0018'), while the EA baseline receives only the sequence. This gives the LLM a trivial fitness-conditioning advantage: it can edit preferentially from the higher-fitness parent or preserve high-fitness residues, whereas random mutation and crossover cannot exploit fitness at proposal time. The central comparison in Table 2 therefore does not isolate LLM protein knowledge as the source of improvement. Please add an ablation that removes fitness scores from the prompt, and add a fitness-conditioned random baseline (e.g., mutation rate or parent-choice probability proportional to fitness) to test whether the observed gap persists without LLM knowledge.
  2. [Table 2, Section 4.2] On GB1 with population sizes 32 and 48, Table 2 reports EA Top-1 fitness of 5.38 +/- 1.77 and 4.88 +/- 0.33 versus Ours of 4.34 +/- 0.53 and 4.31 +/- 0.82. This contradicts the introduction's claim that the LLM proposals are 'much more efficient' in the single-objective setting. The explanation in Section 4.2 that 'EA only outperforms one of three random seeds' is not supported by the reported means and standard deviations; with three seeds and overlapping intervals, seed-level trajectories and a statistical test are needed before the cross-dataset claim can be accepted.
  3. [Algorithm 1, Section 3] The pseudocode's control flow is ambiguous. When the LLM returns None, the code appends C(s0,s1) but then continues to sample r and may append either M(s0) or 'proposed seq' (which is None) without an explicit guard. When the LLM returns a valid sequence, that sequence is only appended in the 'else' branch after a uniform random draw. As written, it is impossible to determine what fraction of offspring in the reported experiments actually come from the LLM rather than from the random crossover/mutation fallback. Please rewrite the pseudocode and report the average fraction of LLM-generated proposals that enter the offspring set per iteration.
  4. [Section 4.1, Table 1] GB1, TrpB, AAV, and GFP are public deep-mutational-scanning datasets, and the SLIP landscape is constructed from a public PDB MSA. It is plausible that Llama-3.1's pretraining corpus contains these sequences and associated fitness information, so the model may propose high-fitness variants by retrieval rather than by on-the-fly optimization. As a correctness-risk control, please report how often the LLM's proposed sequences coincide with known high-fitness variants in each benchmark, or evaluate on a synthetic landscape that did not exist before the model's training cutoff. This would make the retrieval risk measurable without requiring a full audit of the pretraining corpus.
minor comments (4)
  1. [Section 4.2, Table 4] The sentence that 'the maximum Hamming distance of sequences proposed by EA does not exceed 5' is used to explain identical EA rows for H=5 and H=10, but no evidence is provided. Please report the distribution of edit distances observed for EA proposals or otherwise support this claim.
  2. [Appendix A.3] In the prompt text, 'minimizing humming distance' should be 'Hamming distance'.
  3. [Table 1] The label 'AA V' should be 'AAV', and the meaning of 'N/A' in the space-size and target-range columns would benefit from a footnote.
  4. [Section 4.2] The statement that 'linear relationships between positions are less likely in more complex landscapes' is informal; consider reporting an epistasis measure or phrasing this as a testable hypothesis rather than a justification.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the empirical comparison is self-contained and does not reduce to its inputs by construction.

full rationale

The paper's central claim is empirical: pre-trained LLM proposals used inside a directed-evolution loop outperform a random-mutation/crossover evolutionary algorithm on fixed fitness oracles (DMS ground truth, SLIP, and ML fitness models). Neither the LLM nor the EA is fitted to the oracle; selection and ranking procedures are identical for both arms, and the reported fitness differences come from external evaluation, not from any identity between proposed sequences and oracle outputs. The only potentially circular-adjacent issue is that Appendix A.3 includes parent fitness scores in the LLM prompt, so the LLM can condition proposals on fitness while the EA baseline cannot; this makes the comparison uneven and the attribution of gains to pre-trained protein knowledge a confound risk, but it does not make the result equivalent to its inputs by construction. There are no load-bearing self-citations, imported uniqueness theorems, or ansatz-smuggling citations. Accordingly, the appropriate circularity finding is score 0, with the caveat that attribution and baseline fairness are separate correctness concerns.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper does not introduce free parameters or new entities. Its load-bearing assumptions are about the validity of the oracles, the lack of training-data leakage, and the choice of edit distance.

assumptions (3)
  • domain assumption The fitness landscapes and ML oracles used are representative of real protein fitness landscapes, and the ML oracle predictions are accurate enough to rank variants.
    The paper relies on the SLIP synthetic model and two ML oracles (AAV, GFP) as stand-ins for true fitness, without reporting oracle accuracy.
  • ad hoc to paper Llama-3.1-8B-Instruct's proposals are generated from reasoning about protein fitness, not from memorized high-fitness variants in its pre-training corpus.
    The benchmark proteins are public and likely present in the LLM's training data; the paper neither demonstrates nor argues against contamination.
  • domain assumption Hamming distance is an appropriate measure of edit distance for the constraints and budget experiments.
    The paper uses Hamming distance in Eqs. (2) and (3); this is standard but not biologically validated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Large Language Model is Secretly a Protein Sequence Optimizer." pith.science (2026). https://pith.science/paper/FZAEOBS2

@misc{pith2026250109274,
  author       = {Pith},
  title        = {Pith review of: Large Language Model is Secretly a Protein Sequence Optimizer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FZAEOBS2}},
  note         = {Machine review of arXiv:2501.09274}
}
read the original abstract

We consider the protein sequence engineering problem, which aims to find protein sequences with high fitness levels, starting from a given wild-type sequence. Directed evolution has been a dominating paradigm in this field which has an iterative process to generate variants and select via experimental feedback. We demonstrate large language models (LLMs), despite being trained on massive texts, are secretly protein sequence optimizers. With a directed evolutionary method, LLM can perform protein engineering through Pareto and experiment-budget constrained optimization, demonstrating success on both synthetic and experimental fitness landscapes.

Figures

Figures reproduced from arXiv: 2501.09274 by the authors.

Figure 1
Figure 1. The overview of the optimization framework. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Pareto frontiers identified under constrained and budget-constrained optimization settings. [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Pareto frontiers identified under multi-objective optimizations. We display the Pareto [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The fitness heatmaps of first two, last two, last three, and full sequence on two datasets. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: The Pareto frontiers identified by EA and our method in both constrained and budget [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: The Pareto frontier found by our method via different task. S stand for [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Fitness score across all iterations for five datasets with three random seeds. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving

    cs.NE 2025-09 conditional novelty 4.0 of 10

    A literature survey that classifies LLM-based optimization research into modeling and solving, with solving divided into LLMs as optimizers, low-level components, and high-level managers.

Reference graph

Works this paper leans on

27 extracted references · 15 canonical work pages · cited by 1 Pith paper

  1. [1]

    Design by directed evolution

    Frances H Arnold. Design by directed evolution. Accounts of chemical research, 31 0 (3): 0 125--131, 1998

  2. [2]

    Conditioning by adaptive sampling for robust design

    David Brookes, Hahnbeom Park, and Jennifer Listgarten. Conditioning by adaptive sampling for robust design. In International conference on machine learning, pp.\ 773--782. PMLR, 2019

  3. [3]

    Deep diversification of an aav capsid protein by machine learning

    Drew H Bryant, Ali Bashir, Sam Sinai, Nina K Jain, Pierce J Ogden, Patrick F Riley, George M Church, Lucy J Colwell, and Eric D Kelsic. Deep diversification of an aav capsid protein by machine learning. Nature Biotechnology, 39 0 (6): 0 691--696, 2021

  4. [4]

    Llms are highly-constrained biophysical sequence optimizers

    Angelica Chen, Samuel D Stanton, Robert G Alberstein, Andrew M Watkins, Richard Bonneau, Vladimir Gligorijevi, Kyunghyun Cho, and Nathan C Frey. Llms are highly-constrained biophysical sequence optimizers. arXiv preprint arXiv:2410.22296, 2024

  5. [5]

    Flip: Benchmark tasks in fitness landscape inference for proteins

    Christian Dallago, Jody Mou, Kadina E Johnston, Bruce J Wittmann, Nicholas Bhattacharya, Samuel Goldman, Ali Madani, and Kevin K Yang. Flip: Benchmark tasks in fitness landscape inference for proteins. bioRxiv, pp.\ 2021--11, 2021

  6. [6]

    Deep mutational scanning: a new style of protein science

    Douglas M Fowler and Stanley Fields. Deep mutational scanning: a new style of protein science. Nature methods, 11 0 (8): 0 801--807, 2014

  7. [7]

    Mutation effects predicted from sequence co-variation

    Thomas A Hopf, John B Ingraham, Frank J Poelwijk, Charlotta PI Sch \"a rfe, Michael Springer, Chris Sander, and Debora S Marks. Mutation effects predicted from sequence co-variation. Nature biotechnology, 35 0 (2): 0 128--135, 2017

  8. [8]

    Biological sequence design with gflownets

    Moksh Jain, Emmanuel Bengio, Alex Hernandez-Garcia, Jarrid Rector-Brooks, Bonaventure FP Dossou, Chanakya Ajit Ekbote, Jie Fu, Tianyu Zhang, Michael Kilgour, Dinghuai Zhang, et al. Biological sequence design with gflownets. In International Conference on Machine Learning, pp.\ 9786--9801. PMLR, 2022

Show all 27 references
  1. [9]

    A combinatorially complete epistatic fitness landscape in an enzyme active site

    Kadina E Johnston, Patrick J Almhjell, Ella J Watkins-Dulaney, Grace Liu, Nicholas J Porter, Jason Yang, and Frances H Arnold. A combinatorially complete epistatic fitness landscape in an enzyme active site. Proceedings of the National Academy of Sciences, 121 0 (32): 0 e24004...

  2. [10]

    Highly accurate protein structure prediction with alphafold

    John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Z \' dek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold. nature, 596 0 (7873): 0 583--589, 2021

  3. [11]

    Improving protein optimization with smoothed fitness landscapes

    Andrew Kirjner, Jason Yim, Raman Samusevich, Shahar Bracha, Tommi S Jaakkola, Regina Barzilay, and Ila R Fiete. Improving protein optimization with smoothed fitness landscapes. In The Twelfth International Conference on Learning Representations, 2023

  4. [12]

    Evolutionary-scale prediction of atomic-level protein structure with a language model

    Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science, 379 0 (6637): 0 1123--1130, 2023

  5. [13]

    Generative design of functional metal complexes utilizing the internal knowledge of large language models

    Jieyu Lu, Zhangde Song, Qiyuan Zhao, Yuanqi Du, Yirui Cao, Haojun Jia, and Chenru Duan. Generative design of functional metal complexes utilizing the internal knowledge of large language models. arXiv preprint arXiv:2410.18136, 2024

  6. [14]

    Proximal exploration for model-guided protein sequence design

    Zhizhou Ren, Jiahan Li, Fan Ding, Yuan Zhou, Jianzhu Ma, and Jian Peng. Proximal exploration for model-guided protein sequence design. In International Conference on Machine Learning, pp.\ 18520--18536. PMLR, 2022

  7. [15]

    Exploring protein fitness landscapes by directed evolution

    Philip A Romero and Frances H Arnold. Exploring protein fitness landscapes by directed evolution. Nature reviews Molecular cell biology, 10 0 (12): 0 866--876, 2009

  8. [16]

    Local fitness landscape of the green fluorescent protein

    Karen S Sarkisyan, Dmitry A Bolotin, Margarita V Meer, Dinara R Usmanova, Alexander S Mishin, George V Sharonov, Dmitry N Ivankov, Nina G Bozhanova, Mikhail S Baranov, Onuralp Soylemez, et al. Local fitness landscape of the green fluorescent protein. Nature, 533 0 (7603): 0 39...

  9. [17]

    Tuned fitness landscapes for benchmarking model-guided protein design

    Neil Thomas, Atish Agarwala, David Belanger, Yun S Song, and Lucy J Colwell. Tuned fitness landscapes for benchmarking model-guided protein design. bioRxiv, pp.\ 2022--10, 2022

  10. [18]

    Protein design by directed evolution guided by large language models

    Thanh VT Tran and Truong Son Hy. Protein design by directed evolution guided by large language models. IEEE Transactions on Evolutionary Computation, 2024

  11. [19]

    Efficient evolutionary search over chemical space with large language models

    Haorui Wang, Marta Skreta, Cher-Tian Ser, Wenhao Gao, Lingkai Kong, Felix Strieth-Kalthoff, Chenru Duan, Yuchen Zhuang, Yue Yu, Yanqiao Zhu, et al. Efficient evolutionary search over chemical space with large language models. arXiv preprint arXiv:2406.16976, 2024

  12. [20]

    Adaptation in protein fitness landscapes is facilitated by indirect paths

    Nicholas C Wu, Lei Dai, C Anders Olson, James O Lloyd-Smith, and Ren Sun. Adaptation in protein fitness landscapes is facilitated by indirect paths. Elife, 5: 0 e16965, 2016

  13. [21]

    Active learning-assisted directed evolution

    Jason Yang, Ravi G Lal, James C Bowden, Raul Astudillo, Mikhail A Hameedi, Sukhvinder Kaur, Matthew Hill, Yisong Yue, and Frances H Arnold. Active learning-assisted directed evolution. bioRxiv, pp.\ 2024--07, 2024

  14. [22]

    Machine-learning-guided directed evolution for protein engineering

    Kevin K Yang, Zachary Wu, and Frances H Arnold. Machine-learning-guided directed evolution for protein engineering. Nature methods, 16 0 (8): 0 687--694, 2019

  15. [23]

    Protein language models learn evolutionary statistics of interacting sequence motifs

    Zhidian Zhang, Hannah K Wayment-Steele, Garyk Brixi, Haobo Wang, Dorothee Kern, and Sergey Ovchinnikov. Protein language models learn evolutionary statistics of interacting sequence motifs. Proceedings of the National Academy of Sciences, 121 0 (45): 0 e2406285121, 2024

  16. [24]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  17. [25]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  18. [26]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  19. [27]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.