Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

EvoLlama: Enhancing LLMs' Understanding of Proteins via Multimodal Structure and Sequence Representations

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read EvoLlama claims that adding a protein structure encoder and a protein sequence encoder to Llama-3, fused through a lightweight MLP projector, enhances the LLM's protein understanding beyond what sequence-only protein LLMs achieve.

desk verdict EvoLlama's headline Mol-Instructions gain is confounded by the Llama-3 vs Llama-2 decoder swap, but the PEER evidence and honest ablations make it worth a referee's time. read the letter →

arxiv 2412.11618 v1 pith:O33725ZI submitted 2024-12-16 cs.LG cs.AI

Nuowei Liu , Changzhi Sun , Tao Ji , Junfeng Tian , Jianxin Tang , Yuanbin Wu , Man Lan This is my paper · ORCID
classification cs.LGcs.AI
keywords proteinunderstandingmultimodalLLMlanguagemodelstructureencodersequenceinstructiontuningMol-InstructionsPEERbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EvoLlama claims that feeding an LLM both the amino acid sequence and the predicted 3D structure of a protein, encoded by pre-trained protein models and fused into the text embedding space, produces a protein-understanding model that is stronger and more sample-efficient than fine-tuning a text-only LLM on protein instructions. The paper builds a Llama-3 decoder with an ESM-2 sequence encoder and a ProteinMPNN structure encoder, connected by MLP projectors that add the two feature streams element-wise. In zero-shot evaluation on Mol-Instructions the model matches or beats several fully fine-tuned protein LLMs, and after supervised fine-tuning it exceeds Llama-2-7B-Chat by about six ROUGE-L points on the same benchmark. On the PEER protein property prediction tasks it reaches accuracies competitive with task-specific baselines. The paper's claim is that protein structure and sequence are complementary modalities a general-purpose LLM can absorb through a lightweight projector.

What carries the argument

The load-bearing object is the multimodal fusion path: ProteinMPNN's encoder produces per-residue structural features, ESM-2 produces per-residue sequence features, and two MLPs project each into the Llama-3 word-embedding space; the projected tokens are then combined by element-wise addition so the LLM attends over one protein token per residue instead of two. This element-wise fusion halves the number of protein tokens the decoder attends to, cutting inference latency by about 20%, while the two-stage training (optional projection tuning on Swiss-Prot descriptions, then supervised fine-tuning on verbalized Mol-Instructions and PEER tasks) teaches the projector to align protein features with language. The same architecture is shown to work with GearNet in place of ProteinMPNN, and with smaller ESM-2 encoders, which the paper uses to argue the design is plug-and-play.

What would settle it

A decisive check would be to train EvoLlama and an ESM-2-only variant with identical data and compute budget over many random seeds and tasks; if the full model's average ROUGE-L and PEER accuracy never exceed the sequence-only model by a margin larger than the reported 0.0125 ROUGE-L difference on Mol-Instructions, the structure branch's contribution would be negligible. More directly, running the same pipeline with experimentally determined crystal structures instead of AlphaFold-2/ESMFold predictions and observing no performance change would indicate the geometric features are not carrying genuine structural signal.

Watch

Extended reading notes

Core claim

The central claim is that a multimodal representation of proteins, combining an ESM-2 sequence embedding with a ProteinMPNN geometric embedding through simple MLP projection and element-wise addition, can be injected into a frozen Llama-3 decoder and substantially improve its ability to answer protein questions. The paper reports that EvoLlama (ProteinMPNN+ESM-2) achieves 0.52 average ROUGE-L on Mol-Instructions after supervised fine-tuning, versus 0.46 for Llama-2-7B-Chat fine-tuned on the full Mol-Instructions dataset, and that in zero-shot settings it outperforms other fine-tuned protein-oriented LLMs by 1% to 8%. On PEER, it beats or approaches prior state-of-the-art results on solubility, binary localization, and human protein-protein interaction prediction. The authors attribute the gain to the fused structure-sequence representation, noting that ablations removing ProteinMPNN or ESM-2 both hurt performance on Mol-Instructions and PEER.

Load-bearing premise

The claim depends on the assumption that AlphaFold-2 and ESMFold predicted structures are accurate enough that ProteinMPNN's geometric features add information beyond what ESM-2 sequence embeddings already capture, so that the structure branch is not merely redundant.

Editorial extensions

If this is right

  • After only projection tuning, EvoLlama can follow human protein instructions in zero-shot settings, reducing the need for task-specific fine-tuning.
  • The 6-point ROUGE-L gain over full-parameter fine-tuned Llama-2-7B-Chat suggests that frozen-LLM multimodal instruction tuning can beat full fine-tuning on protein text tasks while training far fewer parameters.
  • On PEER, the fused model approaches or beats task-specific baselines on several properties, indicating that a single protein LLM can replace separate classifiers for solubility, localization, and protein-protein interaction prediction.
  • Because the architecture is plug-and-play, swapping in different structure encoders or sequence encoders should extend the same recipe to other protein representations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own ablation numbers (removing ProteinMPNN drops Mol-Instructions ROUGE-L from 0.4800 to 0.4675 and PEER average from 62.38 to 61.14) suggest the structure branch contributes a small, consistent margin; whether that margin is worth its compute is an open question the paper does not fully settle.
  • The same MLP-projection plus element-wise fusion recipe could be applied to other biomolecular modalities such as small-molecule graphs or DNA sequences; the authors list this as future work.
  • The reported sensitivity to AlphaFold-2 versus ESMFold structures implies that zero-shot generalization depends on matching the structure encoder's feature distribution between training and inference; a testable extension would be fine-tuning on ESMFold structures only and measuring transfer back to AlphaFold-2 structures.
  • Because the fused model improves on Llama-3-8B-Instruct across all PEER tasks, the fusion gain may be largest when the base text decoder already has strong instruction-following; one could test whether the gain shrinks for weaker or stronger decoders.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes EvoLlama, a multimodal framework that connects a ProteinMPNN (or GearNet) structure encoder and an ESM-2 sequence encoder to a Llama-3-8B decoder through MLP projectors and element-wise addition of residue-level features. Training is performed in two stages: an optional projection tuning stage on Swiss-Prot-derived protein-text pairs, and a supervised fine-tuning stage on protein-oriented instructions from Mol-Instructions (PMol) and the PEER benchmark. The paper evaluates on four protein understanding tasks from Mol-Instructions and six protein property prediction tasks from PEER, reporting that EvoLlama outperforms fine-tuned protein-oriented LLMs in zero-shot settings and surpasses a state-of-the-art baseline after supervised fine-tuning on Mol-Instructions, while achieving competitive results on PEER.

Significance. The architecture is lightweight and modular, with only about 8% of parameters trainable, and the PEER results include a direct comparison to a fine-tuned text-only Llama-3-8B-Instruct model, which EvoLlama beats on all six tasks. This is a meaningful positive result for the value of protein encoders in LLM-based protein understanding. However, the Mol-Instructions headline claim is confounded by the choice of decoder and training data, and the ablation studies are run at a different training budget than the main model, so the specific contribution of the structural encoder remains unclear. The paper would be a solid empirical contribution if these controls were added and the claims were calibrated to the evidence.

major comments (4)
  1. [§5.1, Table 1 vs. Tables 4 and 7] The headline result that EvoLlama 'surpasses the state-of-the-art baseline with supervised fine-tuning by an average of 6%' (abstract; Table 1) compares EvoLlama, which uses a Llama-3-8B decoder and is trained on PMol plus PEER, with Llama-2-7B-Chat fine-tuned on the complete Mol-Instructions dataset. No text-only Llama-3-8B model fine-tuned on PMol or Mol-Instructions is reported, so the 0.5200 versus 0.4600 gap cannot be attributed to the protein encoders; it may reflect the stronger base decoder or the additional PEER training data. This control is essential because the PEER benchmark includes a text-only Llama-3-8B-Instruct baseline (Table 2) but Mol-Instructions does not.
  2. [§5.3, Tables 4 and 7] The ablation comparing EvoLlama with and without ProteinMPNN (0.4800 versus 0.4675) is run for 10K steps, as stated at the start of Section 5.3, while the headline model in Table 1 is trained for 25,000 steps according to Table 7. The paper does not provide a 25K-step ablation for the sequence-only model, so the 0.0125 difference may be an artifact of training budget. The structure encoder's contribution to the Mol-Instructions results is therefore not established at the training budget used for the main result.
  3. [§5.1, Table 1 and abstract] The abstract claims that EvoLlama 'outperform[s] other fine-tuned protein-oriented LLMs in zero-shot settings by an average of 1%-8%', but Table 1 reports EvoLlama (ProteinMPNN+ESM-2) zero-shot average ROUGE-L of 0.1400, which is lower than Alpaca (0.1625) and Baize (0.1750) fine-tuned on PMol. Only the GearNet variant (0.1700) exceeds those two baselines. The paper's own text in Section 5.1 says 'surpass or approach', so the abstract and introduction should be corrected to match the table.
  4. [§5.2 and Table 10] The sentence 'compared to Llama-3-8B-Instruct, used as a text decoder in our approach, EVOLLAMA improves performance on all tasks by incorporating the multimodal structure and sequence representations of proteins' attributes the improvement to the multimodal fusion. However, Table 10 shows that at the same 10K-step budget, the model without ProteinMPNN achieves 61.14 average on PEER, already 3.29 points above the fine-tuned text-only Llama-3-8B-Instruct average of 57.85 (Table 2). Adding ProteinMPNN improves the average by only 1.24 points and actually decreases subcellular localization (68.05 to 68.63 without the structure encoder) and fold classification (6.18 to 7.94). The evidence thus indicates that the sequence encoder, not the structural encoder, is the primary source of the gains over a text-only LLM.
minor comments (5)
  1. [Table 6] The task groupings appear swapped: 'Protein Understanding Tasks' is listed with PEER benchmark tasks (Solubility, Subcellular Localization, Binary Localization, Fold Classification, Yeast PPI, Human PPI), while 'Protein Property Prediction Tasks' is listed with Mol-Instructions (PMol) tasks. This makes the experimental setup confusing.
  2. [§5.2, first paragraph] The sentence 'Yeast PPI prediction ... and human PPI prediction ... are defined as binary localization tasks' should read 'binary classification tasks' to avoid confusion with the earlier 'binary localization' task.
  3. [§3.1, Eq. (1)] The notation Xa,i and Xa,<i in Equation (1) is not defined; the paper should specify that i indexes tokens of the answer sequence.
  4. [Figure 1] The figure contains the fragment 'AlphaFold ESMFold' without a connecting verb; the caption should state explicitly that structures are predicted by AlphaFold-2 during projection tuning and by ESMFold during supervised fine-tuning.
  5. [Abstract] The abstract states 'We will release our code in a future version'; for reproducibility it would be helpful to release the code, the exact data filtering procedure for the 369K Swiss-Prot subset, and the data splits at the time of publication.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: EvoLlama's reported gains are held-out benchmark measurements, not consequences of a fitted input or a self-citation chain.

full rationale

The paper's central claims are empirical evaluations on established benchmarks with standard held-out splits. The architecture in Sec. 3.1 is a combination of external pretrained encoders (ESM-2, ProteinMPNN) and a frozen Llama-3 decoder with MLP projectors; there is no fitted constant or derived quantity that is defined in terms of the reported outcome. Training uses standard autoregressive next-token likelihood on instruction-following data (Sec. 3.2, Eq. 1), and the projection-tuning templates from Swiss-Prot (Sec. 4) do not encode the PMol/PEER test answers. Results in Tables 1 and 2 are computed on test splits with ROUGE-L and accuracy, respectively, rather than being recovered by construction from the training objective or from a parameter that was fit to the same data. The ablation studies in Sec. 5.3 and Appendix E directly test whether removing the structure or sequence encoder changes performance, which is the appropriate empirical check for the multimodal contribution. The skeptical concern that the headline Mol-Instructions comparison is confounded by the decoder choice (Llama-3 vs Llama-2) and by additional PEER training data is a legitimate experimental-design criticism, but it is not circularity: the numbers could in principle have gone the other way, and the PEER results include a text-only Llama-3-8B-Instruct comparison. No load-bearing self-citation appears in the argument, no uniqueness theorem is imported from the authors' prior work, and no known result is renamed as a new derivation. The paper is self-contained against external benchmarks, so the honest finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No fitted constants appear in this work; the hyperparameters and template counts are engineering choices, and the 369K Swiss-Prot filter is a data-volume decision rather than a derived parameter. The paper relies on three domain assumptions about structure accuracy, projector alignment, and template transfer, none of which is formally proven.

assumptions (3)
  • domain assumption Predicted 3D structures from AlphaFold-2 and ESMFold are accurate enough for ProteinMPNN to encode useful biological information.
    Invoked in Sec 3.1 as the structure encoder input and acknowledged as a limitation; the small structure-ablation gains mean this assumption is load-bearing but weakly supported.
  • domain assumption The MLP projectors can map residue-level ESM-2 and ProteinMPNN features into Llama-3 embedding space without updating the LLM.
    Central to the architecture in Sec 3.1; the paper does not test whether frozen-LLM alignment degrades for longer proteins or unusual embeddings.
  • domain assumption Instruction templates verbalizing PEER and Mol tasks teach task knowledge that transfers to unseen prompts.
    Relied on for zero-shot claims in Sec 5.1 and 5.2; only 10 templates per task are used, with no analysis of template sensitivity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EvoLlama: Enhancing LLMs' Understanding of Proteins via Multimodal Structure and Sequence Representations." pith.science (2026). https://pith.science/paper/O33725ZI

@misc{pith2026241211618,
  author       = {Pith},
  title        = {Pith review of: EvoLlama: Enhancing LLMs' Understanding of Proteins via Multimodal Structure and Sequence Representations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O33725ZI}},
  note         = {Machine review of arXiv:2412.11618}
}
read the original abstract

Current Large Language Models (LLMs) for understanding proteins primarily treats amino acid sequences as a text modality. Meanwhile, Protein Language Models (PLMs), such as ESM-2, have learned massive sequential evolutionary knowledge from the universe of natural protein sequences. Furthermore, structure-based encoders like ProteinMPNN learn the structural information of proteins through Graph Neural Networks. However, whether the incorporation of protein encoders can enhance the protein understanding of LLMs has not been explored. To bridge this gap, we propose EvoLlama, a multimodal framework that connects a structure-based encoder, a sequence-based protein encoder and an LLM for protein understanding. EvoLlama consists of a ProteinMPNN structure encoder, an ESM-2 protein sequence encoder, a multimodal projector to align protein and text representations and a Llama-3 text decoder. To train EvoLlama, we fine-tune it on protein-oriented instructions and protein property prediction datasets verbalized via natural language instruction templates. Our experiments show that EvoLlama's protein understanding capabilities have been significantly enhanced, outperforming other fine-tuned protein-oriented LLMs in zero-shot settings by an average of 1%-8% and surpassing the state-of-the-art baseline with supervised fine-tuning by an average of 6%. On protein property prediction datasets, our approach achieves promising results that are competitive with state-of-the-art task-specific baselines. We will release our code in a future version.

Figures

Figures reproduced from arXiv: 2412.11618 by the authors.

Figure 1
Figure 1. Overall architecture and the training pipeline of the E [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. An example of the projection tuning data and [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Overview of the projection tuning data con [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: The prompt and response template of the projection tuning data. In the response template, [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: Overview of the supervised fine-tuning data [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: The prompt and response template of PEER benchmark in the supervised fine-tuning data. [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Examples of modifications to the original prompts in Mol-Instructions. [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Examples of computing ROUGE-L score on protein understanding tasks. [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Effect of protein sequence encoder sizes. The [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: Case study of performance on protein understanding tasks (Domain/motif prediction). [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 11
Figure 11. Figure 11: Case study of performance on protein understanding tasks (Catalytic activity prediction). [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PDFBench: A Benchmark for De novo Protein Design from Function

    cs.LG 2025-05 conditional novelty 6.0 of 10

    The paper presents PDFBench, a unified benchmark with 16 metrics and a new post-2025 protein test set, and finds that evaluation choices such as retrieval strategy or supported keywords can dominate model rankings.

Reference graph

Works this paper leans on

57 extracted references · 24 canonical work pages · cited by 1 Pith paper

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Hadi Abdine, Michail Chatzianastasis, Costas Bouyioukos, and Michalis Vazirgiannis. 2024. Prot2text: Multimodal protein’s function generation with gnns and transformers. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 10757--10765

    work page 2024

  4. [4]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  5. [5]

    Jos \'e Juan Almagro Armenteros, Casper Kaae S nderby, S ren Kaae S nderby, Henrik Nielsen, and Ole Winther. 2017. Deeploc: prediction of protein subcellular localization using deep learning. Bioinformatics, 33(21):3387--3395

    work page 2017

  6. [6]

    Nadav Brandes, Dan Ofer, Yam Peleg, Nadav Rappoport, and Michal Linial. 2022. Proteinbert: a universal deep-learning model of protein sequence and function. Bioinformatics, 38(8):2102--2110

    work page 2022

  7. [7]

    The UniProt Consortium. 2021. Uniprot: the universal protein knowledgebase in 2021. Nucleic acids research, 49(D1):D480--D489

    work page 2021

  8. [8]

    The UniProt Consortium. 2023. Uniprot: the universal protein knowledgebase in 2023. Nucleic acids research, 51(D1):D523--D531

    work page 2023

Show all 57 references
  1. [9]

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. 2024. Instructblip: Towards general-purpose vision-language models with instruction tuning. Advances in Neural Information Processing Systems, 36

    2024

  2. [10]

    Justas Dauparas, Ivan Anishchenko, Nathaniel Bennett, Hua Bai, Robert J Ragotte, Lukas F Milles, Basile IM Wicky, Alexis Courbet, Rob J de Haas, Neville Bethel, et al. 2022. Robust deep learning--based protein sequence design using proteinmpnn. Science, 378(6615):49--56

    2022

  3. [11]

    Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. 2021. Glm: General language model pretraining with autoregressive blank infilling. arXiv preprint arXiv:2103.10360

    2021 arXiv

  4. [12]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  5. [13]

    Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, et al. 2021. Prottrans: Toward understanding the language of life through self-supervised learning. IEEE transactions on pa...

    2021

  6. [14]

    Yin Fang, Xiaozhuan Liang, Ningyu Zhang, Kangwei Liu, Rui Huang, Zhuo Chen, Xiaohui Fan, and Huajun Chen. 2023. Mol-instructions: A large-scale biomolecular instruction dataset for large language models. arXiv preprint arXiv:2306.08018

    2023 arXiv

  7. [15]

    Zhi-Ping Feng and Chun-Ting Zhang. 2000. Prediction of membrane protein types based on the hydrophobic index of amino acids. Journal of protein chemistry, 19:269--275

    2000

  8. [16]

    Naomi K Fox, Steven E Brenner, and John-Marc Chandonia. 2014. Scope: Structural classification of proteins—extended, integrating scop and astral data and classification of new structures. Nucleic acids research, 42(D1):D304--D309

    2014

  9. [17]

    Vladimir Gligorijevi \'c , P Douglas Renfrew, Tomasz Kosciolek, Julia Koehler Leman, Daniel Berenberg, Tommi Vatanen, Chris Chandler, Bryn C Taylor, Ian M Fisk, Hera Vlamakis, et al. 2021. Structure-based protein function prediction using graph convolutional networks. Nature c...

    2021

  10. [18]

    Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2021. Domain-specific language model pretraining for biomedical natural language processing. ACM Transactions on Computing for Healthcare (HEALTH), 3(1):1--23

    2021

  11. [19]

    Han Guo, Mingjia Huo, Ruiyi Zhang, and Pengtao Xie. 2023. Proteinchat: Towards achieving chatgpt-like functionalities on protein 3d structures. Authorea Preprints

    2023

  12. [20]

    Yanzhi Guo, Lezheng Yu, Zhining Wen, and Menglong Li. 2008. Using support vector machine combined with auto covariance to predict protein--protein interactions from protein sequences. Nucleic acids research, 36(9):3025--3030

    2008

  13. [21]

    Tomas Hayes, Roshan Rao, Halil Akin, Nicholas J Sofroniew, Deniz Oktay, Zeming Lin, Robert Verkuil, Vincent Q Tran, Jonathan Deaton, Marius Wiggert, et al. 2024. Simulating 500 million years of evolution with a language model. bioRxiv, pages 2024--07

    2024

  14. [22]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770--778

    2016

  15. [23]

    Sepp Hochreiter and J \"u rgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735--1780

    1997

  16. [24]

    Jie Hou, Badri Adhikari, and Jianlin Cheng. 2018. Deepsf: deep convolutional neural network for mapping protein sequences to folds. Bioinformatics, 34(8):1295--1303

    2018

  17. [25]

    John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Z \' dek, Anna Potapenko, et al. 2021. Highly accurate protein structure prediction with alphafold. nature, 596(7873):583--589

    2021

  18. [26]

    Sameer Khurana, Reda Rawi, Khalid Kunji, Gwo-Yu Chuang, Halima Bensmail, and Raghvendra Mall. 2018. Deepsol: a deep learning framework for sequence-based protein solubility prediction. Bioinformatics, 34(15):2605--2613

    2018

  19. [27]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74--81

    2004

  20. [28]

    Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Sal Candido, et al. 2022. Language models of protein sequences at the scale of evolution enable accurate structure prediction. BioRxiv, 2022:500902

    2022

  21. [29]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2024 a . Visual instruction tuning. Advances in neural information processing systems, 36

    2024

  22. [30]

    Tianyu Liu, Yijia Xiao, Xiao Luo, Hua Xu, W Jim Zheng, and Hongyu Zhao. 2024 b . Geneverse: A collection of open-source multimodal large language models for genomic and proteomic research. arXiv preprint arXiv:2406.15534

    2024 arXiv

  23. [31]

    Liuzhenghao Lv, Zongying Lin, Hao Li, Yuyang Liu, Jiaxi Cui, Calvin Yu-Chian Chen, Li Yuan, and Yonghong Tian. 2024. Prollama: A protein large language model for multi-task protein language processing. arXiv preprint arXiv:2402.16445

    2024 arXiv

  24. [32]

    OpenAI. 2022. Introducing chatgpt. https://openai.com/blog/chatgpt/. Accessed: 2024-03-10

    2022

  25. [33]

    Xiao-Yong Pan, Ya-Nan Zhang, and Hong-Bin Shen. 2010. Large-scale prediction of human protein- protein interactions from amino acid sequence based on latent topic features. Journal of proteome research, 9(10):4992--5001

    2010

  26. [34]

    Qizhi Pei, Lijun Wu, Kaiyuan Gao, Xiaozhuan Liang, Yin Fang, Jinhua Zhu, Shufang Xie, Tao Qin, and Rui Yan. 2024. Biot5+: Towards generalized biological understanding with iupac integration and multi-task tuning. arXiv preprint arXiv:2402.17810

    2024 arXiv

  27. [35]

    Qizhi Pei, Wei Zhang, Jinhua Zhu, Kehan Wu, Kaiyuan Gao, Lijun Wu, Yingce Xia, and Rui Yan. 2023. Biot5: Enriching cross-modal integration in biology with chemical knowledge and natural language associations. arXiv preprint arXiv:2310.07276

    2023 arXiv

  28. [36]

    Suraj Peri, J Daniel Navarro, Ramars Amanchy, Troels Z Kristiansen, Chandra Kiran Jonnalagadda, Vineeth Surendranath, Vidya Niranjan, Babylakshmi Muthusamy, TKB Gandhi, Mads Gronborg, et al. 2003. Development of human protein reference database as an initial platform for appro...

    2003

  29. [37]

    Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C Lawrence Zitnick, Jerry Ma, et al. 2021. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proceedings of the Nat...

    2021

  30. [38]

    Vijayakumar Saravanan and Namasivayam Gautham. 2015. Harnessing computational biology for exact linear b-cell epitope prediction: a novel amino acid composition-based feature descriptor. Omics: a journal of integrative biology, 19(10):648--658

    2015

  31. [39]

    Amir Shanehsazzadeh, David Belanger, and David Dohan. 2020. Is transfer learning necessary for protein landscape prediction? arXiv preprint arXiv:2011.03443

    2020 arXiv

  32. [40]

    Jin Su, Chenchen Han, Yuyang Zhou, Junjie Shan, Xibin Zhou, and Fajie Yuan. 2023. Saprot: Protein language modeling with structure-aware vocabulary. bioRxiv, pages 2023--10

    2023

  33. [41]

    Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. 2022. Galactica: A large language model for science. arXiv preprint arXiv:2211.09085

    2022 arXiv

  34. [42]

    Tloen. 2023. Alpaca-lora. https://github.com/tloen/alpaca-lora

    2023

  35. [43]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023 a . Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  36. [44]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023 b . Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  37. [45]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30

    2017

  38. [46]

    Chao Wang, Hehe Fan, Ruijie Quan, and Yi Yang. 2024 a . Protchatgpt: Towards understanding proteins with large language models. arXiv preprint arXiv:2402.09649

    2024 arXiv

  39. [47]

    Zeyuan Wang, Qiang Zhang, Keyan Ding, Ming Qin, Xiang Zhuang, Xiaotong Li, and Huajun Chen. 2024 b . https://doi.org/10.18653/v1/2024.acl-long.62 Instructprotein: Aligning human and protein language via knowledge instruction . In Proceedings of the 62nd Annual Meeting of the A...

    2024 doi

  40. [48]

    Kevin E Wu, Howard Chang, and James Zou. 2024. Proteinclip: enhancing protein language models with natural language. bioRxiv, pages 2024--05

    2024

  41. [49]

    Yijia Xiao, Edward Sun, Yiqiao Jin, Qifan Wang, and Wei Wang. 2024. Proteingpt: Multimodal llm for protein property prediction and structure understanding. arXiv preprint arXiv:2408.11363

    2024 arXiv

  42. [50]

    Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley. 2023 a . Baize: An open-source chat model with parameter-efficient tuning on self-chat data. arXiv preprint arXiv:2304.01196

    2023 arXiv

  43. [51]

    Minghao Xu, Xinyu Yuan, Santiago Miret, and Jian Tang. 2023 b . Protst: Multi-modality learning of protein sequences and biomedical texts. In International Conference on Machine Learning, pages 38749--38767. PMLR

    2023

  44. [52]

    Minghao Xu, Zuobai Zhang, Jiarui Lu, Zhaocheng Zhu, Yangtian Zhang, Ma Chang, Runcheng Liu, and Jian Tang. 2022. Peer: a comprehensive and multi-task benchmark for protein sequence understanding. Advances in Neural Information Processing Systems, 35:35156--35173

    2022

  45. [53]

    Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, et al. 2022. Glm-130b: An open bilingual pre-trained model. arXiv preprint arXiv:2210.02414

    2022 arXiv

  46. [54]

    Ningyu Zhang, Zhen Bi, Xiaozhuan Liang, Siyuan Cheng, Haosen Hong, Shumin Deng, Jiazhang Lian, Qiang Zhang, and Huajun Chen. 2022 a . Ontoprotein: Protein pretraining with gene ontology embedding. arXiv preprint arXiv:2201.11147

    2022 arXiv

  47. [55]

    Zuobai Zhang, Chuanrui Wang, Minghao Xu, Vijil Chenthamarakshan, Aur \'e lie Lozano, Payel Das, and Jian Tang. 2023. A systematic study of joint representation learning on protein sequences and structures. arXiv preprint arXiv:2303.06275

    2023 arXiv

  48. [56]

    Zuobai Zhang, Minghao Xu, Arian Jamasb, Vijil Chenthamarakshan, Aurelie Lozano, Payel Das, and Jian Tang. 2022 b . Protein representation learning by geometric structure pretraining. arXiv preprint arXiv:2203.06125

    2022 arXiv

  49. [57]

    Le Zhuo, Zewen Chi, Minghao Xu, Heyan Huang, Heqi Zheng, Conghui He, Xian-Ling Mao, and Wentao Zhang. 2024. Protllm: An interleaved protein-language llm with protein-as-word pre-training. arXiv preprint arXiv:2403.07920

    2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.