Pith. sign in

REVIEW 2 major objections 3 minor 46 references

EpiBench: Can LLMs Understand Epitopes for Antibody Drug Discovery?

T0 review · 2 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read EpiBench: LLMs stay near random on precise epitope tasks.

desk verdict Solid, useful benchmark with a real gap: the 'partial epitope signals' claim needs a length-conditioned random baseline before it can carry the weight. read the letter →

arxiv 2608.06022 v1 pith:I3CSABHW submitted 2026-08-06 cs.CL q-bio.GN

Zirui Wang , Jiaqi Wang , Qinghan Wang , Yuzhi Xu , Gang Du , Tingjun Hou , Odin Zhang This is my paper · ORCID
classification cs.CLq-bio.GN
keywords epitopereasoningantibodydrugdiscoverylargelanguagemodelssequence-basedbenchmarkclosed-bookevaluationescapeassessmentbinningfunctional
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EpiBench asks whether general-purpose large language models can reason about epitopes directly from antigen and antibody sequences, with no structural data and no identifying metadata. It organizes 1,609 samples into five connected decisions, from targetable-region discovery to antibody escape assessment, each with experimentally grounded labels and automatically scorable answers. Across nine LLMs, the benchmark finds partial coarse signals but near-chance performance wherever residue-level or antibody-specific precision is required: Task 1 AUROC ranges from 50.3 to 54.6 against a random baseline of 50.0, and Task 5 escape accuracy ranges from 41.4 to 54.7 against 50.0. Where specialized sequence models exist, they beat every LLM (Task 1 60.4 vs best LLM 56.9; Task 2 40.2 vs 33.6; Task 3 70.8 vs 67.3). The paper concludes that current LLMs are not yet reliable for epitope-centered antibody development and positions EpiBench as a diagnostic testbed rather than a leaderboard.

What carries the argument

The carrying mechanism is EpiBench itself, a closed-book, sequence-only evaluation pipeline that converts epitope decisions into automatically scorable prompts. Each task uses a deterministic output format: residue-index sets for Task 1, multiple-choice for Tasks 2 and 4, and binary judgments for Tasks 3 and 5, with balanced answer distributions, shuffled candidates, matched antibody-similarity strata for binning, and mutation-level matching for escape. Labels are built from AsEP/SAbDab structural contacts (Tasks 1–3), IEDB functional B-cell assay evidence (Task 4), and deep mutational scanning escape measurements (Task 5). The five tasks are intentionally connected so that no single shortcut can solve the whole workflow, and the controlled sampling is designed to remove answer-position, class-prior, antibody-similarity, and mutation-identity artifacts.

What would settle it

Recompute Task 1 RegR@50 with a random-residue baseline matched to each sample's label coverage and antigen length; if random selection matches or approaches LLM recall on long antigens, the claim that LLMs capture partial epitope signals loses its support.

Watch

Extended reading notes

Core claim

The central claim is that current LLMs capture partial epitope-related signals but remain limited in antibody-specific sequence grounding, long-context residue localization, and biologically grounded reasoning. The evidence is a sequence-only, closed-book benchmark whose five tasks mirror the antibody development workflow: targetable region discovery, antibody-conditioned epitope identification, epitope binning, functional epitope assessment, and antibody escape assessment. Labels come from structural antibody–antigen contacts, curated B-cell functional assays, and deep mutational scanning escape measurements, while prompts strip identifiers to block lookup shortcuts. The paper reports that no general LLM reaches the specialized-model baseline on any task where such a baseline exists, and that several tasks sit at or near random accuracy.

Load-bearing premise

The load-bearing premise is that the Task 1 label — the union of antibody contact footprints filtered to cover at most half the antigen — accurately represents the complete set of targetable residues; if that label is too broad, the measured recall does not demonstrate genuine residue-level epitope understanding.

Editorial extensions

If this is right

  • If the benchmark's results hold, general-purpose LLMs are not yet usable for per-residue epitope localization: Task 1 AUROC stays within 0.3 to 4.6 points of random across all nine models.
  • Specialized sequence models remain the reference point for epitope prediction, since BepiPred-3.0 and EpiPred-derived baselines exceed every evaluated LLM on the tasks where they apply.
  • Explicit chain-of-thought reasoning helps some models on targetable-region discovery but does not reliably improve antibody-conditioned, functional, or escape decisions.
  • Model ranking is task-dependent, so no single best LLM for antibody discovery emerges from this evaluation.
  • Task 1 recall collapses from 81.3 on antigens under 200 residues to 12.8 on antigens over 800 residues, identifying long-context residue localization as a specific bottleneck.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The partial-signal conclusion would be stronger with a random-recall baseline conditioned on each Task 1 sample's label coverage and antigen length; without it, part of the observed RegR@50 could simply reflect the breadth of the union ground truth.
  • Because Task 3 matches same-bin and different-bin pairs by CDR similarity, the binning scores may partly reflect remaining sequence-level cues; an ablation that removes CDR regions from the prompts would isolate whether models use interface information or similarity heuristics.
  • The same benchmark design could be extended to protein-specific language models, which might localize residues better than text-pretrained general LLMs even under identical prompts.
  • The paper reports only zero-shot results, so it does not bound how much one-shot or fine-tuned exposure on EpiBench could close the gap to specialized models.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. EpiBench introduces a closed-book, sequence-only benchmark of 1,609 curated samples for evaluating whether LLMs can reason about epitopes across five connected antibody-discovery decisions: targetable region discovery, antibody-conditioned epitope identification, epitope binning, functional epitope assessment, and escape assessment. Labels are grounded in external evidence sources (AsEP and SAbDab structural contacts, IEDB functional B-cell assays, and Cao et al. deep mutational scanning). Nine general-purpose LLMs are evaluated under a unified zero-shot, explicit-reasoning protocol and compared with random baselines and specialized models where available. The paper reports that current LLMs capture partial epitope-related signals but remain limited in long-context residue localization, antibody-specific sequence grounding, and biologically grounded reasoning, and it provides length-stratified, reasoning-format, and failure-mode analyses.

Significance. If its conclusions hold, EpiBench is a valuable diagnostic resource. The evaluation is unusually careful for an LLM benchmark: answer distributions are balanced and verified (Appendix D.2), Task 3 pairs are matched by CDR similarity, Task 5 samples are mutation-matched, prompts are de-identified, and the failure-mode case study in Appendix C.3 concretely shows a coordinate-misalignment error. The negative half of the central claim—that LLMs are near chance on per-residue epitope discrimination and escape assessment, and are outperformed by specialized models where comparisons exist—is credible and practically relevant. The positive half, however, rests primarily on Task 1 RegionRecall@50, which is not yet calibrated against length-conditioned random behavior; this is the main load-bearing weakness.

major comments (2)
  1. [Section 4.1, Table 2, Figure 4; Appendix B.2.1] The claim that LLMs 'capture partial epitope-related signals' is principally supported by Task 1 RegionRecall@50 (Table 2, observation (1)), but this metric is reported without a length-conditioned random baseline. For uniform random selection of 50 positions, the expected RegR@50 is 50/L, where L is the antigen length, so the random expectation ranges from roughly 25–50% for antigens under 200 residues to about 6% for antigens over 800 residues; Table 2 marks the random RegR@50 as N/A, and Figure 4 plots the observed decline from 81.3 to 12.8 without this reference. Because Algorithm 1 filters only by |U|/|S| > 0.5 over the mapped surface (line 11), the union label for short antigens can be comparable to the fixed 50-residue prediction budget, raising the achievable recall ceiling by construction. Please add length-stratified random and surface-accessibility heuristic baselines, report label-size and coverage statistics per length bin, and re-derive observation (1) and the abstract's 'partial signals' wording from the calibrated comparison.
  2. [Section 4.1, observation (2); Appendix B.2.3] The paper attributes Task 3's above-random performance partly to 'pairwise sequence-similarity cues' and states that the task 'may admit' such cues, yet Appendix B.2.3 says same- and different-bin pairs are sampled within CDR-similarity quantile bins so that positive and negative pairs have matched antibody similarity profiles. If the matching is effective, the cue-based explanation is unsupported; if it is not, the matching control is insufficient. Please quantify the residual discriminability of CDR or germline similarity for the bin labels (for example, the AUROC of a similarity score on the released pairs) and adjust the interpretation of Task 3 accordingly.
minor comments (3)
  1. [Figures 1–4] Figure 1 contains the typo 'Resjdueprediction', and Figures 2–4 render task names as 'T ask1' and similar in some axis labels; these should be corrected in the final version.
  2. [Table 2 and Figure 8] Several near-random accuracies (for example, Task 5 scores of 41.4–54.7 versus the 50.0 random baseline, and Task 2 scores of 24.0–33.6 versus 25.0) are interpreted as 'at or near chance' without confidence intervals or significance tests; with 260–360 samples per task, bootstrap or standard-error reporting would make these interpretations more robust.
  3. [Appendix C.2] The prompt examples say 'numbering omitted' while the evaluation prompts are described as position-numbered; please clarify that the appendix excerpts are truncated for readability and that the actual prompts include residue numbering, since Task 1 and the coordinate-misalignment case study depend on it.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: EpiBench's labels are externally grounded and no fitted quantity is repackaged as a prediction.

full rationale

EpiBench is a benchmark-construction and evaluation paper, not a parameter-fitting paper. Its labels come from external experimental sources: AsEP and SAbDab structural contacts for Tasks 1–3, IEDB B-cell assay records for Task 4, and Cao et al. deep mutational scanning measurements for Task 5. None of these labels are defined in terms of the evaluated LLMs' outputs, and no fitted parameter is later reported as a prediction. The fixed thresholds used in construction, such as the Task 5 normalized escape cutoff of 0.50/0.05 and the Task 3 Jaccard binning threshold of 0.75, are label-construction choices inherited from external assay conventions rather than learned quantities that reappear as results. The only self-citation by the present authors, reference [44], appears in a related-work enumeration of structure-based epitope predictors and is not load-bearing for any experimental claim. The paper's own stated limitation, that pretraining memorization of public biological sequences cannot be fully eliminated, is a leakage caveat, not a circular step. The most substantive concern is the absence of a length-conditioned random baseline for Task 1's RegR@50: under uniform random selection of 50 residues, expected recall is approximately 50*k/L, and the union label can cover a large fraction of short antigens, so the reported 81.3 recall on antigens under 200 residues may partly reflect label breadth. That is a benchmark-validity and statistical-control weakness, but it is not circularity: the label is not constructed from model predictions, no equation makes RegR@50 equivalent to an input, and the 'partial epitope signals' conclusion is not justified by a self-citation chain. The failure-mode taxonomy in Section 4.4 is also descriptive and explicitly hedged ('These observations do not imply that the detected patterns are the only causes of failure'), so it does not smuggle the conclusion into the scoring. No specific circular reduction can be exhibited, and the honest finding is no circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

EpiBench introduces no fitted parameters in the derivation sense; its central claim is an empirical evaluation result, not a derived formula. However, several hand-chosen thresholds define the labels and therefore shape task difficulty: the Task 1 prediction budget k=50, the Task 5 escape/non-escape split (0.50/0.05 on per-antibody normalized DMS scores), the Task 3 binning Jaccard threshold (0.75), and the 0.5 overlap coefficient for candidate grouping. The benchmark rests on four domain assumptions inherited from external evidence sources: the 4.5 Å structural contact definition of epitopes (AsEP), the validity of union-of-footprints labels for targetable regions, the mapping of IEDB assay records to mechanism categories and residue coordinates, and the transfer of single-study SARS-CoV-2 DMS measurements to a general escape-assessment construct. No new physical or model entities are postulated; the five task constructs map to established biological concepts.

free parameters (4)
  • Task 1 prediction budget k = 50 residues regardless of antigen length
    Hand-chosen output budget for RegR@50. Fixed at 50 while antigen length spans under 200 to over 800 residues, so longer antigens have lower achievable recall; directly shapes the length-stratified decline in Section 4.2.
  • Task 5 escape threshold = escape if normalized score >= 0.50; non-escape if <= 0.05
    Binary split on per-antibody normalized DMS escape scores; intermediate cases discarded (Appendix B.2.5). Threshold placement determines label balance and task difficulty.
  • Task 3 binning Jaccard threshold = J >= 0.75 -> same bin
    Adopted from Waury et al. [41]; pairs with partial overlap are discarded. Threshold choice defines the binning ground truth.
  • Candidate grouping overlap coefficient = 0.5 overlap
    Used to group mapped footprints into candidate regions and patches (Tasks 1-2); affects distractor construction and label granularity.
assumptions (5)
  • domain assumption Epitope residues are antigen surface residues with heavy atoms within 4.5 Å of antibody CDR heavy atoms
    Contact definition inherited from AsEP [22] and used for all structural labels (Tasks 1-3). It is a geometric proxy for binding and may miss indirect or allosteric contributions.
  • domain assumption The union of representative antibody footprints, mapped to the reference antigen and capped at 50% surface coverage, is a valid targetable-region ground truth
    Algorithm 1 collapses footprints from distinct antibody clusters into a single label per antigen; known epitopes may be incomplete, which the paper acknowledges (Appendix B.2.1).
  • domain assumption IEDB assay records map reliably to mechanism categories and to unique residue coordinates in the UniProt-linked antigen
    Task 4 requires mapping linear peptides to parent sequences and assay names to inhibition, protection, or cytotoxicity (Appendix B.2.4); this mapping is lossy and inherits IEDB annotation quality.
  • domain assumption Per-antibody normalized DMS scores from Cao et al. 2022 support a binary escape/non-escape split
    Task 5 labels derive from one study of SARS-CoV-2 RBD antibodies (Appendix B.2.5); the construct of antibody escape assessment is tested on a single antigen family.
  • domain assumption MMseqs2 clustering at 95% identity with 0.8 coverage removes redundancy without erasing epitope diversity
    Curation step in Algorithm 1; the clustering aggressiveness shapes the antigen set and the union labels.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EpiBench: Can LLMs Understand Epitopes for Antibody Drug Discovery?." pith.science (2026). https://pith.science/paper/I3CSABHW

@misc{pith2026260806022,
  author       = {Pith},
  title        = {Pith review of: EpiBench: Can LLMs Understand Epitopes for Antibody Drug Discovery?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I3CSABHW}},
  note         = {Machine review of arXiv:2608.06022}
}
read the original abstract

Epitopes determine where antibodies bind antigens and shape downstream therapeutic properties such as functional blockade and escape resistance, making epitope understanding central to antibody drug discovery. Although large language models (LLMs) have shown strong biomedical reasoning ability, it remains unclear whether they can infer epitope information directly from antigen and antibody sequences. Existing epitope resources typically focus on isolated prediction tasks or rely on specialized structural settings, while general protein benchmarks do not evaluate epitope-centered decisions across the antibody development workflow. To address this gap, we introduce EpiBench, a closed-book, sequence-based, and automatically scorable benchmark for evaluating epitope reasoning in LLMs. EpiBench contains 1,609 curated samples grounded in structural antibody--antigen contacts, curated functional B-cell assays, and deep mutational scanning escape measurements. It covers five connected tasks: targetable region discovery, antibody-conditioned epitope identification, epitope binning, functional epitope assessment, and antibody escape assessment, with controlled sampling to reduce shortcut-based evaluation artifacts. We evaluate nine general-purpose LLMs and analyze their behavior through task-specific baselines, antigen length stratification, explicit-reasoning comparison, and failure-mode inspection. The results show that current LLMs capture partial epitope-related signals but remain limited in antibody-specific sequence grounding, long-context residue localization, and biologically grounded reasoning. Therefore, EpiBench provides a diagnostic testbed for measuring and improving sequence-aware biomedical LLMs toward reliable LLM-assisted antibody discovery.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 38 canonical work pages

  1. [1]

    CHIMERA-Bench: A Benchmark Dataset for Epitope-Specific Antibody Design

    Mansoor Ahmed, Nadeem Taj, Imdad Ullah Khan, Hemanth Venkateswara, and Murray Patterson. Chimera-bench: A benchmark dataset for epitope-specific antibody design.arXiv preprint arXiv:2603.13431, 2026

    work page Pith review arXiv 2026

  2. [2]

    Identification of conformational b-cell epitopes in an antigen from its primary sequence.Immunome research, 6(1):6, 2010

    Hifzur Rahman Ansari and Gajendra PS Raghava. Identification of conformational b-cell epitopes in an antigen from its primary sequence.Immunome research, 6(1):6, 2010

    work page 2010

  3. [3]

    Yunlong Cao, Ayijiang Yisimayi, Fanchong Jian, Weiliang Song, Tianhe Xiao, Lei Wang, Shuo Du, Jing Wang, Qianqian Li, Xiaosu Chen, et al. Ba. 2.12. 1, ba. 4 and ba. 5 escape antibodies elicited by omicron infection. Nature, 608(7923):593–602, 2022. 9

    work page 2022

  4. [4]

    Proteinglue multi-task benchmark suite for self-supervised protein modeling.Scientific Reports, 12(1):16047, 2022

    Henriette Capel, Robin Weiler, Maurits Dijkstra, Reinier Vleugels, Peter Bloem, and K Anton Feenstra. Proteinglue multi-task benchmark suite for self-supervised protein modeling.Scientific Reports, 12(1):16047, 2022

    work page 2022

  5. [5]

    Flab: Benchmarking deep learning methods for antibody fitness prediction.BioRxiv, pp

    Michael Chungyoun, Jeffrey Ruffolo, and Jeffrey Gray. Flab: Benchmarking deep learning methods for antibody fitness prediction.BioRxiv, pp. 2024–01, 2024

    work page 2024

  6. [6]

    Bepipred-3.0: Improved b-cell epitope prediction using protein language models.Protein Science, 31(12):e4497, 2022

    Joakim Nøddeskov Clifford, Magnus Haraldson Høie, Sebastian Deleuran, Bjoern Peters, Morten Nielsen, and Paolo Marcatili. Bepipred-3.0: Improved b-cell epitope prediction using protein language models.Protein Science, 31(12):e4497, 2022

    work page 2022

  7. [7]

    Flip: Benchmark tasks in fitness landscape inference for proteins.bioRxiv, pp

    Christian Dallago, Jody Mou, Kadina E Johnston, Bruce J Wittmann, Nicholas Bhattacharya, Samuel Goldman, Ali Madani, and Kevin K Yang. Flip: Benchmark tasks in fitness landscape inference for proteins.bioRxiv, pp. 2021–11, 2021

    work page 2021

  8. [8]

    Flip2: Expanding protein fitness landscape benchmarks for real-world machine learning applications

    Kieran Didi, Sarah Alamdari, Alex Xijie Lu, Bruce James Wittmann, Kadina E Johnston, Ava P Amini, Ali Madani, Maya Czeneszew, Christian Dallago, and Kevin K Yang. Flip2: Expanding protein fitness landscape benchmarks for real-world machine learning applications. InForty-third International Conference on Machine Learning, 2026

    work page 2026

Show all 46 references
  1. [9]

    Sabdab: the structural antibody database.Nucleic acids research, 42(D1):D1140–D1146, 2014

    James Dunbar, Konrad Krawczyk, Jinwoo Leem, Terry Baker, Angelika Fuchs, Guy Georges, Jiye Shi, and Charlotte M Deane. Sabdab: the structural antibody database.Nucleic acids research, 42(D1):D1140–D1146, 2014

    2014

  2. [10]

    Predicting linear b-cell epitopes using string kernels

    Yasser EL-Manzalawy, Drena Dobbs, and Vasant Honavar. Predicting linear b-cell epitopes using string kernels. Journal of Molecular Recognition: An Interdisciplinary Journal, 21(4):243–255, 2008

    2008

  3. [11]

    others prottrans: Toward understanding the language of life through self-supervised learning.IEEE transactions on pattern analysis and machine intelligence, 44(10): 7112–7127, 2021

    Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang, Llion Jones, Tom Gibbs, Tamas Feher, Christoph Angerer, Martin Steinegger, et al. others prottrans: Toward understanding the language of life through self-supervised learning.IEEE transactions on pa...

    2021

  4. [12]

    Induction of hepatitis a virus-neutralizing antibody by a virus-specific synthetic peptide.Journal of virology, 55(3):836–839, 1985

    Emilio A Emini, JOSEPH V Hughes, DoS Perlow, and J Boger. Induction of hepatitis a virus-neutralizing antibody by a virus-specific synthetic peptide.Journal of virology, 55(3):836–839, 1985

    1985

  5. [13]

    Mol-instructions: A large-scale biomolecular instruction dataset for large language models

    Yin Fang, Xiaozhuan Liang, Ningyu Zhang, Kangwei Liu, Rui Huang, Zhuo Chen, Xiaohui Fan, and Huajun Chen. Mol-instructions: A large-scale biomolecular instruction dataset for large language models. InInternational Conference on Learning Representations, volume 2024, pp. 48221–...

    2024

  6. [14]

    Enhancing protein mutation effect prediction through a retrieval-augmented framework

    Ruihan Guo, Rui Wang, Ruidong Wu, Zhizhou Ren, Jiahan Li, Shitong Luo, Zuofan Wu, Qiang Liu, Jian Peng, and Jianzhu Ma. Enhancing protein mutation effect prediction through a retrieval-augmented framework. In Advances in Neural Information Processing Systems, volume 37, pp. 49...

    2024

  7. [15]

    Prediction of residues in discontinuous b-cell epitopes using protein 3d structures.Protein Science, 15(11):2558–2567, 2006

    Pernille Haste Andersen, Morten Nielsen, and OLE Lund. Prediction of residues in discontinuous b-cell epitopes using protein 3d structures.Protein Science, 15(11):2558–2567, 2006

    2006

  8. [16]

    Artificial intelligence foundation for therapeutic science.Nature chemical biology, 18(10):1033–1036, 2022

    Kexin Huang, Tianfan Fu, Wenhao Gao, Yue Zhao, Yusuf Roohani, Jure Leskovec, Connor W Coley, Cao Xiao, Jimeng Sun, and Marinka Zitnik. Artificial intelligence foundation for therapeutic science.Nature chemical biology, 18(10):1033–1036, 2022

    2022

  9. [17]

    A semi-empirical method for prediction of antigenic determinants on protein antigens.FEBS letters, 276(1-2):172–174, 1990

    Ashok S Kolaskar and Prasad C Tongaonkar. A semi-empirical method for prediction of antigenic determinants on protein antigens.FEBS letters, 276(1-2):172–174, 1990

    1990

  10. [18]

    Measuring faithfulness in chain-of-thought reasoning

    Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, et al. Measuring faithfulness in chain-of-thought reasoning. arXiv preprint arXiv:2307.13702, 2023

    2023 arXiv

  11. [19]

    Improved method for predicting linear b-cell epitopes.Immunome research, 2(1):2, 2006

    Jens Erik Pontoppidan Larsen, Ole Lund, and Morten Nielsen. Improved method for predicting linear b-cell epitopes.Immunome research, 2(1):2, 2006

    2006

  12. [20]

    Hotspot-driven peptide design via multi-fragment autoregressive extension

    Jiahan Li, Tong Chen, Shitong Luo, Chaoran Cheng, Jiaqi Guan, Ruihan Guo, Sheng Wang, Ge Liu, Jian Peng, and Jianzhu Ma. Hotspot-driven peptide design via multi-fragment autoregressive extension. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  13. [21]

    Evolutionary-scale prediction of atomic-level protein structure with a language model.Science, 379(6637):1123–1130, 2023

    Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model.Science, 379(6637):1123–1130, 2023. 10

    2023

  14. [22]

    Asep: Benchmarking deep learning methods for antibody-specific epitope prediction.Advances in Neural Information Processing Systems, 37:11700–11734, 2024

    Chunan Liu, Lilian Denzler, Yihong Chen, Andrew Martin, and Brooks Paige. Asep: Benchmarking deep learning methods for antibody-specific epitope prediction.Advances in Neural Information Processing Systems, 37:11700–11734, 2024

    2024

  15. [23]

    What makes chain-of-thought prompting effective? a counterfactual study

    Aman Madaan, Katherine Hermann, and Amir Yazdanbakhsh. What makes chain-of-thought prompting effective? a counterfactual study. InFindings of the Association for Computational Linguistics: EMNLP 2023, pp. 1448–1535, 2023

    2023

  16. [24]

    Large language models generate functional protein sequences across diverse families.Nature biotechnology, 41(8):1099–1106, 2023

    Ali Madani, Ben Krause, Eric R Greene, Subu Subramanian, Benjamin P Mohr, James M Holton, Jose Luis Olmos Jr, Caiming Xiong, Zachary Z Sun, Richard Socher, et al. Large language models generate functional protein sequences across diverse families.Nature biotechnology, 41(8):10...

    2023

  17. [25]

    Exploring large protein language models in constrained evaluation scenarios within the flip benchmark.arXiv preprint arXiv:2501.18223, 2025

    Manuel F Mollon, Joaquin Gonzalez-Rodriguez, Alicia Lozano-Diez, Daniel Ramos, and Doroteo T Toledano. Exploring large protein language models in constrained evaluation scenarios within the flip benchmark.arXiv preprint arXiv:2501.18223, 2025

    2025 arXiv

  18. [26]

    Progen2: exploring the boundaries of protein language models.Cell systems, 14(11):968–978, 2023

    Erik Nijkamp, Jeffrey A Ruffolo, Eli N Weinstein, Nikhil Naik, and Ali Madani. Progen2: exploring the boundaries of protein language models.Cell systems, 14(11):968–978, 2023

    2023

  19. [27]

    Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval

    Pascal Notin, Mafalda Dias, Jonathan Frazer, Javier Marchena-Hurtado, Aidan N Gomez, Debora Marks, and Yarin Gal. Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval. InInternational Conference on Machine Learning, pp. 16990–17...

    2022

  20. [28]

    Proteingym: Large-scale benchmarks for protein fitness prediction and design.Advances in neural information processing systems, 36:64331–64379, 2023

    Pascal Notin, Aaron Kollasch, Daniel Ritter, Lood Van Niekerk, Steffanie Paul, Han Spinner, Nathan Rollins, Ada Shaw, Rose Orenbuch, Ruben Weitzman, et al. Proteingym: Large-scale benchmarks for protein fitness prediction and design.Advances in neural information processing sy...

    2023

  21. [29]

    Ellipro: a new structure-based tool for the prediction of antibody epitopes.BMC bioinformatics, 9(1):514, 2008

    Julia Ponomarenko, Huynh-Hoa Bui, Wei Li, Nicholas Fusseder, Philip E Bourne, Alessandro Sette, and Bjoern Peters. Ellipro: a new structure-based tool for the prediction of antibody epitopes.BMC bioinformatics, 9(1):514, 2008

    2008

  22. [30]

    Evaluating protein transfer learning with tape.Biorxiv, pp

    Roshan Rao, Nicholas Bhattacharya, Neil Thomas, Yan Duan, Xi Chen, John Canny, Pieter Abbeel, and Yun S Song. Evaluating protein transfer learning with tape.Biorxiv, pp. 676825, 2019

    2019

  23. [31]

    Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C Lawrence Zitnick, Jerry Ma, et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences.Proceedings of the national a...

    2021

  24. [32]

    Prediction of continuous b-cell epitopes in an antigen using recurrent neural network.Proteins: Structure, Function, and Bioinformatics, 65(1):40–48, 2006

    Sudipto Saha and Gajendra Pal Singh Raghava. Prediction of continuous b-cell epitopes in an antigen using recurrent neural network.Proteins: Structure, Function, and Bioinformatics, 65(1):40–48, 2006

    2006

  25. [33]

    Large language models encode clinical knowledge.arXiv preprint arXiv:2212.13138, 2022

    Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. Large language models encode clinical knowledge.arXiv preprint arXiv:2212.13138, 2022

    2022 arXiv

  26. [34]

    Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets.Nature biotechnology, 35(11):1026–1028, 2017

    Martin Steinegger and Johannes Söding. Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets.Nature biotechnology, 35(11):1026–1028, 2017

    2017

  27. [35]

    Seppa: a computational server for spatial epitope prediction of protein antigens.Nucleic acids research, 37(suppl_2): W612–W616, 2009

    Jing Sun, Di Wu, Tianlei Xu, Xiaojing Wang, Xiaolian Xu, Lin Tao, YX Li, and Zhi-Wei Cao. Seppa: a computational server for spatial epitope prediction of protein antigens.Nucleic acids research, 37(suppl_2): W612–W616, 2009

    2009

  28. [36]

    Peta: evaluating the impact of protein transfer learning with sub-word tokenization on downstream applications.Journal of Cheminformatics, 16(1):92, 2024

    Yang Tan, Mingchen Li, Ziyi Zhou, Pan Tan, Huiqun Yu, Guisheng Fan, and Liang Hong. Peta: evaluating the impact of protein transfer learning with sub-word tokenization on downstream applications.Journal of Cheminformatics, 16(1):92, 2024

    2024

  29. [37]

    Atom3d: Tasks on molecules in three dimensions

    Raphael John Lamarre Townshend, Martin Vögele, Patricia Adriana Suriana, Alexander Derry, Alexander Powers, Yianni Laloudakis, Sidhika Balachandar, Bowen Jing, Brandon M Anderson, Stephan Eismann, et al. Atom3d: Tasks on molecules in three dimensions. InThirty-fifth Conference...

    2021

  30. [38]

    2025 ginkgo datapoints antibody developability competition outcomes: limited model performance and a call for data standardization.MAbs, 18(1):2634216, 2026

    Lood van Niekerk, Joshua Moller, Seth Ritter, Porfirio Quintero-Cadena, Rich Cohen, Georgia Channing, Michael Chungyoun, Laura Rand, Alexander Smith, Aanal Bhatt, et al. 2025 ginkgo datapoints antibody developability competition outcomes: limited model performance and a call f...

    2025

  31. [39]

    The immune epitope database (iedb): 2024 update.Nucleic Acids Research, 53(D1):D436–D443, 2025

    Randi Vita, Nina Blazeska, Daniel Marrama, IEDB Curation Team Members Shackelford Deborah Zalman Leora Foos Gabriele Zarebski Laura Chan Kenneth Reardon Brian Fitzpatrick Sidne Busse Matthew Coleman Sara Sedwick Caitlin Edwards Lindy MacFarlane Catriona Ennis Marcus, Sebastian...

    2024

  32. [40]

    Uni-rna: universal pre-trained models revolutionize rna research.bioRxiv, pp

    Xi Wang, Ruichu Gu, Zhiyuan Chen, Yongge Li, Xiaohong Ji, Guolin Ke, and Han Wen. Uni-rna: universal pre-trained models revolutionize rna research.bioRxiv, pp. 2023–07, 2023

    2023

  33. [41]

    Comparison of sequence-and structure-based antibody clustering approaches on simulated repertoire sequencing data.PLoS Computational Biology, 21(5):e1013057, 2025

    Katharina Waury, Stefan Lelieveld, Sanne Abeln, and Henk-Jan van den Ham. Comparison of sequence-and structure-based antibody clustering approaches on simulated repertoire sequencing data.PLoS Computational Biology, 21(5):e1013057, 2025

    2025

  34. [42]

    Fafe: Immune complex modeling with geodesic distance loss on noisy group frames

    Ruidong Wu, Ruihan Guo, Rui Wang, Shitong Luo, Yue Xu, Jiahan Li, Jianzhu Ma, Qiang Liu, Yunan Luo, and Jian Peng. Fafe: Immune complex modeling with geodesic distance loss on noisy group frames. InProceedings of the 41st International Conference on Machine Learning, volume 23...

    2024

  35. [43]

    Peer: a comprehensive and multi-task benchmark for protein sequence understanding.Advances in Neural Information Processing Systems, 35:35156–35173, 2022

    Minghao Xu, Zuobai Zhang, Jiarui Lu, Zhaocheng Zhu, Yangtian Zhang, Ma Chang, Runcheng Liu, and Jian Tang. Peer: a comprehensive and multi-task benchmark for protein sequence understanding.Advances in Neural Information Processing Systems, 35:35156–35173, 2022

    2022

  36. [44]

    Yuzhi Xu, Wei Xia, Chao Zhang, Xinxin Liu, Cheng-Wei Ju, Xuhang Dai, Pujun Xie, Yuanqing Wang, Guangyong Chen, and John Z. H. Zhang. Sake-pp: A spatial-attention equivariant network for accurate ranking of protein- protein interaction models.JACS Au, 6(5):2846–2856, 2026

    2026

  37. [45]

    Benchmark for antibody binding affinity maturation and design.arXiv e-prints, pp

    Xinyan Zhao, Yi-Ching Tang, Akshita Singh, Victor J Cantu, KwanHo An, Junseok Lee, Adam E Stogsdill, Ashwin Kumar Ramesh, Zhiqiang An, Xiaoqian Jiang, et al. Benchmark for antibody binding affinity maturation and design.arXiv e-prints, pp. arXiv–2506, 2025

    2025

  38. [46]

    Respond with ONLY a JSON object and no other text

    Le Zhuo, Zewen Chi, Minghao Xu, He-Yan Huang, Jianan Zhao, Heqi Zheng, Conghui He, Xian-Ling Mao, and Wentao Zhang. Protllm: An interleaved protein-language llm with protein-as-word pre-training. InProceedings of the 62nd Annual Meeting of the Association for Computational Lin...

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.