Pith. sign in

REVIEW 4 major objections 7 minor 52 references

Protein Inverse Folding From Structure Feedback

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that fine-tuning an inverse folding model with Direct Preference Optimization (DPO) on preferences derived from a folding model improves both sequence recovery and structural similarity of designed proteins, raising…

desk verdict A solid DPO-for-inverse-folding study with a likely test-set-adaptation artifact in its headline multi-round claim and an ESMFold-circular evaluation loop. read the letter →

arxiv 2506.03028 v1 pith:SVU57ISD submitted 2025-06-03 cs.LG q-bio.BM

Junde Xu , Zijun Gao , Xinyi Zhou , Jie Hu , Xingyi Cheng , Le Song , Guangyong Chen , Pheng-Ann Heng
show 1 more author
Jiezhong Qiu
This is my paper · ORCID
classification cs.LGq-bio.BM
keywords inversefoldingproteindesigndirectpreferenceoptimizationstructurefeedbackTM-ScoreESMFolditerativerefinementsequencerecovery
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that an inverse folding model can design better sequences for a target structure by fine-tuning it with Direct Preference Optimization (DPO) on pairwise preferences derived from a protein folding model. Candidate sequences sampled from the model are folded in silico, scored by TM-Score against the target structure, and split into chosen and rejected halves; the model is then trained to prefer the better-folding half. On the CATH 4.2 benchmark, this improves average structural similarity of designed sequences from 0.77 to 0.81 TM-Score while also improving sequence recovery over the baseline. Iterating the procedure on ten hard structures raises their average TM-Score from 0.39 to 0.70, a 79.5% relative gain. The significance, if true, is that protein sequence design can be improved without experimental feedback, using only structure prediction as the reward signal.

What carries the argument

The engine of the method is a self-generated preference dataset built by folding-model feedback, combined with the DPO objective and an SFT regularizer. For each target structure T, the inverse folding model samples N candidate sequences; ESMFold predicts each candidate's structure; TM-Align computes each predicted structure's TM-Score against T; the top half of sequences are labeled chosen and the bottom half rejected. The fine-tuning loss is L = λ·SFT loss on chosen sequences + DPO loss, where DPO maximizes the log-ratio πθ(S_w|T)/πref(S_w|T) minus the corresponding log-ratio for rejected sequences, and λ controls the balance. In iterative training, each round reinitializes the reference model to the current model before generating the next preference dataset.

What would settle it

Take the sequences designed by InstructPLM-DPO and by the baseline for the same CATH 4.2 structures, and score their predicted structures with an independent structure predictor such as AlphaFold (or test a sample experimentally). If the TM-Score advantage of the DPO model shrinks or disappears under the independent oracle, the reported gains reflect fitting ESMFold rather than improved foldability. A simpler version: check whether ESMFold's per-sequence TM-Score ranking agrees with AlphaFold's ranking on the same candidate sequences; substantial disagreement would falsify the transfer assumption.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that preference optimization on structural feedback is a viable post-training step for inverse folding. The authors show that DPO fine-tuning of InstructPLM on self-generated sequences labeled by ESMFold-predicted TM-Scores improves both recovery rate (from 53.58% to 55.21% on CATH 4.2) and average TM-Score (from 0.77 to 0.81), and that including rejected low-TM sequences in the loss matters more than only training on chosen ones. They further show that iterative multi-round DPO, where each round's model generates its own preference data and becomes the reference model for the next round, substantially improves ten difficult structures, and that simply increasing the number of contrastive samples per structure degrades structural fidelity while increasing training steps helps up to a saturation point. The intended conclusion is that the model learns structure-related preferences beyond sequence identity, producing sequences that fold more accurately even when they deviate from the wild-type sequence.

Load-bearing premise

The load-bearing premise is that ESMFold-predicted structures are a valid, unbiased oracle for true foldability, so optimizing the TM-Score of ESMFold predictions transfers to real protein design.

Editorial extensions

If this is right

  • A fully in silico feedback loop can replace at least part of the experimental screening step in inverse folding workflows.
  • The same DPO-plus-SFT recipe can be applied to other inverse folding or protein language models, since only sampling, folding, and TM-Score labeling are required.
  • Rejected samples (low-TM-Score predictions) are the main driver of structural improvement, especially for structures where the baseline design is poor, so preference-data quality may matter more than dataset size.
  • Multi-round refinement can push difficult design targets substantially closer to the target fold, though one hard case (1mkn.A) remains unsolved.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 0.77-to-0.81 TM-Score gain is measured with the same ESMFold model that provides the training labels; whether the improvement transfers to AlphaFold or to wet-lab folding is an open question the paper does not address.
  • Because the model is rewarded for high TM-Score against the target, it may learn to exploit systematic blind spots of ESMFold, such as over-stable or self-complementary sequences that score well in silico but fail in vitro.
  • The scaling result that more contrastive samples can hurt TM-Score suggests a future direction of active or hard-negative mining to select preference pairs, rather than random sampling.
  • The method is a general optimization-from-feedback template: any scoreable structural property, such as stability, binding, or function, could be swapped into the same DPO loop.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes an in silico preference optimization pipeline for protein inverse folding. Given a target structure, candidate sequences are sampled from an inverse folding model (InstructPLM or ProteinMPNN); each sequence is folded with ESMFold, and TM-Scores against the target structure are used to construct 'chosen' and 'rejected' pairs. The inverse folding model is then fine-tuned with the DPO objective plus an SFT regularization term. The authors report improved sequence recovery and TM-Scores on CATH 4.2, TS50, and TS500, with an average TM-Score increase from 0.77 to 0.81 on the CATH 4.2 test set, and a 79.5% relative TM-Score gain in an iterative multi-round setting on 10 difficult CATH 4.2 structures. The paper includes ablations on the SFT weight lambda, number of contrastive samples, and training steps, and it discloses a modified protocol for ProteinMPNN.

Significance. If the claims hold, the paper offers a scalable, fully in silico method to improve inverse folding models using structure feedback, which is an active and important direction. The experiments are internally consistent and include useful ablations; the authors also honestly disclose the ProteinMPNN modification and the scaling limitations. However, the central evaluation is weakened by two issues: the same ESMFold model generates the preference labels and scores the final structures, creating a reward-overfitting risk; and the multi-round 79.5% gain is reported on structures that appear to be the very ones used for iterative training. These issues affect the interpretation of the headline results but are addressable with additional experiments and clarification, so the paper merits a major revision rather than outright rejection.

major comments (4)
  1. [§3, Eq. (8) and §4.2] The preference labels are computed with ESMFold (Tn = g(Sn), cn = TM-Score(Tn, T)) and the final evaluation in §4.2 also uses ESMFold for all predicted structures. This circularity means the reported improvements in TM-Score may reflect the model learning to exploit ESMFold's specific scoring biases rather than a genuine improvement in foldability. Please evaluate the designed sequences with an independent structure predictor (e.g., AlphaFold2) or, at minimum, report a correlation analysis between ESMFold and AlphaFold2 TM-Scores on the designed sequences. Without such a check, the claim of 'enhanced structure similarity' is not established beyond ESMFold-specific reward alignment.
  2. [§4.5, Fig. 4 and Tab. 4] The 10 'difficult structures' are selected from the CATH 4.2 test set, and the iterative DPO loop (Sec. 3) constructs preference data D_t^pair from the current model's generations for the structures being optimized. The text does not state that these 10 test structures are excluded from the training set, and the natural reading is that each of the 20 rounds fine-tunes on sequences generated for exactly these 10 targets. If so, the reported mean TM-Score increase from 0.39 to 0.70 (79.5%) is an adaptive fit to the evaluation set, not evidence of generalization. Please clarify which structures are used to construct D_t^pair in each round; if the test structures are included, the abstract's iterative-gain claim must be re-evaluated on held-out structures or a separate validation split.
  3. [§4.5 and Tab. 4] The multi-round protocol uses N=200 sequences per structure, 20 rounds, and only 200 training steps per round, with no validation-based early stopping or model selection described. Because the reference model is reinitialized to the current policy each round, this is not the standard DPO setup and the iteration may converge to an overfit solution on the 10 chosen structures. Please report the progression of TM-Score on a separate holdout set during the iterative rounds, and show that the improvements are not due to memorization of the specific structures used for training.
  4. [§4.1 and Appendix A.1.4] The ProteinMPNN variant deviates substantially from the main method: wild-type sequences are added as chosen samples, and only sequences with TM-Score below a threshold (tr = 0.8) are used for training, rather than the top/bottom 50% split of Eq. (8). This means the ProteinMPNN-DPO result does not test the same preference-optimization approach as InstructPLM-DPO, and the statement in §4.1 that 'both DPO models yield consistent gains' should be qualified. Please either present ProteinMPNN results under the original protocol (if feasible) or explicitly frame the ProteinMPNN experiments as a variant with a modified preference dataset.
minor comments (7)
  1. [§3, Eq. (7)] The inequality in Eq. (7) is misprinted: it reads c_k(1) ≥ c_k(1) ≥ ... instead of c_k(1) ≥ c_k(2) ≥ ... . Please correct the indexing.
  2. [§3, Eq. (10)] The term 'super-parameter' should be 'hyperparameter'.
  3. [§4.3, Fig. 2] The linear regression reports R² = 0.19, which is a weak association; the claim of a 'positive correlation' should be accompanied by a confidence interval or p-value to indicate whether the slope is statistically distinguishable from zero.
  4. [§4.2, Table 2] The TM-Score differences (e.g., 0.77 vs. 0.81) are reported without error bars or standard deviations across the 10 sampled sequences per structure. Please provide variance estimates or statistical significance tests to support that the improvements are not within sampling noise.
  5. [Abstract and §4.5] The phrase 'an average TM-Score increase of 79.5%' is a relative increase; please clarify in the abstract and text that this is a percentage improvement relative to the baseline (from 0.39 to 0.70), since readers may otherwise interpret it as an absolute increase of 79.5 percentage points.
  6. [§3, Iterative Training] The notation uses π_t^θ for the model at iteration t, but θ is not formally defined in that section; clarify that θ denotes the model parameters and that the reference model is set to the weights of π_t^θ at each round.
  7. [§3, Iterative Training] The phrase 'interactive training framework' should be 'iterative training framework' for consistency with the rest of the paper.

Circularity Check

1 steps flagged · score 6.0 of 10

Iterative DPO's headline 79.5% TM-Score gain is measured on the same 10 CATH 4.2 test structures used to build each round's preference data, so the multi-round improvement is a train-set result rather than a held-out prediction.

  1. fitted input called prediction [Sec. 4.5 (Iterative Protein Design); Sec. 3 (Iterative Training); Appendix A.4]
    "To evaluate the capacity for iterative improvement, we selected 10 difficult structures from the CATH 4.2 test set to assess the performance limits of our approach. ... For each iteration t, the model πtθ will first generate and construct preference data Dt_pair as described in Sec. 3, the model is then updated to πt+1θ with the DPO and SFT loss (Eq. 10) on generated dataset Dt_pair. ... we monitor the running TM-Scores and recoveries of all 10 structures during training in Fig. 6."

    The paper never states that the 10 hard test structures were excluded from the iterative preference-data construction; the only training loop described (Sec. 3) builds D_t^pair from the current model's generations for structures T, and the multi-round experiment is reported for exactly these 10 structures, whose TM-Scores are tracked 'during training' in Fig. 6. Therefore the reported mean TM-Score increase from 0.39 to 0.70 (79.5%) is computed on the same structures whose ESMFold-derived preference labels drove the fine-tuning in each of the 20 rounds.

full rationale

The single-round experiments are not circular in the construction-level sense: InstructPLM-DPO and ProteinMPNN-DPO are fine-tuned on preference data from the CATH 4.2 training split and evaluated on the held-out test split (Sec. 4.1, 4.2), so the recovery and TM-Score gains there are genuine held-out numbers. Using ESMFold both to create preference labels and to score the final structures is an oracle-choice limitation that raises external-validity risk, but it is not a definitional equivalence because the training and test structures are disjoint and ESMFold is an independent external predictor. The clear circularity is in Sec. 4.5: the 10 difficult structures selected from the test set for the iterative experiment coincide with the structures on which each round's D_t^pair is constructed (Fig. 6 monitors these 10 structures 'during training'), and no held-out set is reported for multi-round DPO. The 79.5% gain therefore reflects optimization on the evaluation targets, not a predictive result. Score 6 reflects that this test-set adaptation invalidates the central multi-round claim while leaving the single-round benchmark results independent.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central method introduces no new physical entities. The main free choices are loss weights and sampling scales (lambda, beta, N, tr), and the key domain assumption is that ESMFold TM-Scores are a valid designability reward. The ledger shows the paper buys its central result with a surrogate oracle and several hand-chosen hyperparameters, not with a parameter-free derivation.

free parameters (4)
  • lambda (SFT loss weight) = 10, 1, 0 (ablated)
    Controls the balance between chosen-sample SFT loss and DPO loss in Eq. 10; ablation shows performance depends on it.
  • beta (DPO temperature) = 0.5
    Set to 0.5 for all experiments; standard DPO hyperparameter chosen by hand.
  • N (contrastive samples per structure) = 20 (single-round), 200 (multi-round)
    Number of sampled sequences used to build preference pairs; scaling experiments show more samples can degrade TM-Score.
  • tr (ProteinMPNN threshold) = 0.8
    Threshold used in the modified ProteinMPNN protocol to restrict training to low-TM-Score predictions and add wild-type as chosen.
assumptions (5)
  • domain assumption TM-Score of ESMFold-predicted structures against the target is a valid reward for sequence designability.
    Used to label chosen/rejected pairs (Sec. 3, Eq. 8) and as the evaluation metric (Sec. 4.2); no independent predictor or wet-lab check.
  • domain assumption Top/bottom 50% split of sampled sequences by TM-Score produces meaningful preference pairs.
    Eqs. 7-8; assumes any pair with a higher TM-Score is reliably preferable, even when both scores are low.
  • domain assumption Adding an SFT term to the DPO loss (Eq. 10) preserves the validity of DPO and prevents degeneration.
    Motivated by mathematical reasoning tasks [33]; no proof or analysis that the regularized loss still optimizes preferences.
  • domain assumption Self-generated preference data from the model's own samples is a stable training signal.
    Sec. 4.1 reports ProteinMPNN fails to converge on self-generated data, so this assumption is violated for smaller models; wild-type supervision was added.
  • ad hoc to paper Standard DPO derivation remains valid when the reference model is reinitialized each round.
    Sec. 3 iterative training; reference model is reset to current pi_t each round, a modification not covered by the original DPO derivation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Protein Inverse Folding From Structure Feedback." pith.science (2026). https://pith.science/paper/SVU57ISD

@misc{pith2026250603028,
  author       = {Pith},
  title        = {Pith review of: Protein Inverse Folding From Structure Feedback},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SVU57ISD}},
  note         = {Machine review of arXiv:2506.03028}
}
read the original abstract

The inverse folding problem, aiming to design amino acid sequences that fold into desired three-dimensional structures, is pivotal for various biotechnological applications. Here, we introduce a novel approach leveraging Direct Preference Optimization (DPO) to fine-tune an inverse folding model using feedback from a protein folding model. Given a target protein structure, we begin by sampling candidate sequences from the inverse-folding model, then predict the three-dimensional structure of each sequence with the folding model to generate pairwise structural-preference labels. These labels are used to fine-tune the inverse-folding model under the DPO objective. Our results on the CATH 4.2 test set demonstrate that DPO fine-tuning not only improves sequence recovery of baseline models but also leads to a significant improvement in average TM-Score from 0.77 to 0.81, indicating enhanced structure similarity. Furthermore, iterative application of our DPO-based method on challenging protein structures yields substantial gains, with an average TM-Score increase of 79.5\% with regard to the baseline model. This work establishes a promising direction for enhancing protein sequence design ability from structure feedback by effectively utilizing preference optimization.

Figures

Figures reproduced from arXiv: 2506.03028 by the authors.

Figure 1
Figure 1. Overview of our methods. A. We connect the inverse-folding process and folding process with DPO, utilizing the structure similarity (TM-Scores) predicted by the folding model to guide the optimization of the inverse-folding model. B. We classify generated sequences into chosen and rejected based on their TM-Scores. C. Multi-round DPO for iterative refinement of designed sequences. The underlying assumption is that p… view at source ↗
Figure 2
Figure 2. Per-structure changes in TM-score vs. changes in sequence recovery on the CATH 4.2 test set. Top left: Change in TM-Score (mean TM-Score over 10 predictions) vs. change in recovery (mean Recovery over 10 predictions) for each structure. Top right: TM-Score versus recovery of 10 sequences sampled from the baseline and the DPO model, samples selected for visualization are marked by circle. Bottom: Examples structure p… view at source ↗
Figure 3
Figure 3. Performance at different training scales. The upper row shows protein design perfor￾mance changes at different training steps, and the lower row shows protein design performance at different numbers of contrastive samples (N in Eq. 6) of each structure. 4.3 A Closer Look at Proteins Generated by DPO Model To evaluate the impact of our methods on the DPO model, we analyzed the relationship between the changes in stru… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Multi-round result on 10 hard structures selected from CATH 4.2 test set. The plot shows the TM-Scores of predicted structures from the final round. 4.4 DPO at Scale Scaling has been shown to improve performance in large language models and, more recently, in protein l…
Figure 5
Figure 5. Figure 5: Metrics of ProteinMPNN training process. [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: TM-Score and recovery rate changes across each round of iterative refinement. [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Sequence alignment of 1kvz.A. Baseline recovery rate: 0.36, TM-Score: 0.51. Finetuned [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Sequence alignment of 2ln7.A. Baseline recovery rate: 0.30, TM-Score: 0.36. Finetuned [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Sequence alignment of 3i5d.A. Baseline recovery rate: 0.57, TM-Score: 0.55. Finetuned [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Sequence alignment of 4ap5.A. Baseline recovery rate: 0.38, TM-Score: 0.37. Finetuned [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 21 canonical work pages

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Computational protein design.Nature Reviews Methods Primers, 5(1):13, 2025

    Katherine I Albanese, Sophie Barbe, Shunsuke Tagami, Derek N Woolfson, and Thomas Schiex. Computational protein design.Nature Reviews Methods Primers, 5(1):13, 2025

    2025

  3. [3]

    Constitutional ai: Harmlessness from ai feedback.arXiv preprint arXiv:2212.08073, 2022

    Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback.arXiv preprint arXiv:2212.08073, 2022

    arXiv 2022

  4. [4]

    Hallucination of multimodal large language models: A survey.arXiv preprint arXiv:2404.18930, 2024

    Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang, and Mike Zheng Shou. Hallucination of multimodal large language models: A survey.arXiv preprint arXiv:2404.18930, 2024

    arXiv 2024

  5. [5]

    Scaling unlocks broader generation and deeper functional understanding of proteins.bioRxiv, pages 2025–04, 2025

    Aadyot Bhatnagar, Sarthak Jain, Joel Beazer, Samuel C Curran, Alexander M Hoffnagle, Kyle Ching, Michael Martyn, Stephen Nayfach, Jeffrey A Ruffolo, and Ali Madani. Scaling unlocks broader generation and deeper functional understanding of proteins.bioRxiv, pages 2025–04, 2025

    work page 2025

  6. [6]

    Self-play fine-tuning converts weak language models to strong language models.arXiv preprint arXiv:2401.01335, 2024

    Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. Self-play fine-tuning converts weak language models to strong language models.arXiv preprint arXiv:2401.01335, 2024

    arXiv 2024

  7. [7]

    Deep reinforcement learning from human preferences.Advances in neural information processing systems, 30, 2017

    Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences.Advances in neural information processing systems, 30, 2017

    2017

  8. [8]

    Group equivariant convolutional networks

    Taco Cohen and Max Welling. Group equivariant convolutional networks. InInternational conference on machine learning, pages 2990–2999. PMLR, 2016

    2016

Show all 52 references
  1. [9]

    Robust deep learning–based protein sequence design using proteinmpnn.Science, 378(6615):49–56, 2022

    Justas Dauparas, Ivan Anishchenko, Nathaniel Bennett, Hua Bai, Robert J Ragotte, Lukas F Milles, Basile IM Wicky, Alexis Courbet, Rob J de Haas, Neville Bethel, et al. Robust deep learning–based protein sequence design using proteinmpnn.Science, 378(6615):49–56, 2022

    2022

  2. [10]

    Atomic context-conditioned protein sequence design using ligandmpnn.Nature Methods, pages 1–7, 2025

    Justas Dauparas, Gyu Rie Lee, Robert Pecoraro, Linna An, Ivan Anishchenko, Cameron Glasscock, and David Baker. Atomic context-conditioned protein sequence design using ligandmpnn.Nature Methods, pages 1–7, 2025

    2025

  3. [11]

    Amino acid pairing at the n-and c- termini of helical segments in proteins.PROTEINS: Structure, Function, and Bioinformatics, 70(1):188–196, 2008

    Nuno A Fonseca, Rui Camacho, and AL Magalhaes. Amino acid pairing at the n-and c- termini of helical segments in proteins.PROTEINS: Structure, Function, and Bioinformatics, 70(1):188–196, 2008

    2008

  4. [12]

    Zhangyang Gao, Cheng Tan, Xingran Chen, Yijie Zhang, Jun Xia, Siyuan Li, and Stan Z. Li. KW-design: Pushing the limit of protein design via knowledge refinement. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  5. [13]

    Alphadesign: A graph protein design method and benchmark on alphafolddb.arXiv preprint arXiv:2202.01079, 2022

    Zhangyang Gao, Cheng Tan, and Stan Z Li. Alphadesign: A graph protein design method and benchmark on alphafolddb.arXiv preprint arXiv:2202.01079, 2022

    2022 arXiv

  6. [14]

    Pifold: Toward effective and efficient protein inverse folding, 2022

    Zhangyang Gao, Cheng Tan, and Stan Z Li. Pifold: Toward effective and efficient protein inverse folding, 2022

    2022

  7. [15]

    Simulating 500 million years of evolution with a language model.Science, page eads0018, 2025

    Thomas Hayes, Roshan Rao, Halil Akin, Nicholas J Sofroniew, Deniz Oktay, Zeming Lin, Robert Verkuil, Vincent Q Tran, Jonathan Deaton, Marius Wiggert, et al. Simulating 500 million years of evolution with a language model.Science, page eads0018, 2025

    2025

  8. [16]

    Learning inverse folding from millions of predicted structures.International Conference on Machine Learning, pages 8946–8970, 2022

    Chloe Hsu, Robert Verkuil, Jason Liu, Zeming Lin, Brian Hie, Tom Sercu, Adam Lerer, and Alexander Rives. Learning inverse folding from millions of predicted structures.International Conference on Machine Learning, pages 8946–8970, 2022. 10

    2022

  9. [17]

    Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  10. [18]

    Generative models for graph-based protein design.Advances in neural information processing systems, 32, 2019

    John Ingraham, Vikas Garg, Regina Barzilay, and Tommi Jaakkola. Generative models for graph-based protein design.Advances in neural information processing systems, 32, 2019

    2019

  11. [19]

    Learn- ing from protein structure with geometric vector perceptrons.arXiv preprint arXiv:2009.01411, 2020

    Bowen Jing, Stephan Eismann, Patricia Suriana, Raphael JL Townshend, and Ron Dror. Learn- ing from protein structure with geometric vector perceptrons.arXiv preprint arXiv:2009.01411, 2020

    2009 arXiv

  12. [20]

    Highly accurate protein structure prediction with alphafold.nature, 596(7873):583–589, 2021

    John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ron- neberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold.nature, 596(7873):583–589, 2021

    2021

  13. [21]

    Sequence-similar, structure-dissimilar protein pairs in the pdb.Proteins: Structure, Function, and Bioinformatics, 71(2):891–902, 2008

    Mickey Kosloff and Rachel Kolodny. Sequence-similar, structure-dissimilar protein pairs in the pdb.Proteins: Structure, Function, and Bioinformatics, 71(2):891–902, 2008

    2008

  14. [22]

    Generalized biomolecular modeling and design with rosettafold all-atom.Science, 384(6693):eadl2528, 2024

    Rohith Krishna, Jue Wang, Woody Ahern, Pascal Sturmfels, Preetham Venkatesh, Indrek Kalvet, Gyu Rie Lee, Felix S Morey-Burrows, Ivan Anishchenko, Ian R Humphreys, et al. Generalized biomolecular modeling and design with rosettafold all-atom.Science, 384(6693):eadl2528, 2024

    2024

  15. [23]

    Rlaif: Scaling reinforcement learning from human feedback with ai feedback.arXiv preprint arXiv:2309.00267, 2023

    Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, et al. Rlaif: Scaling reinforcement learning from human feedback with ai feedback.arXiv preprint arXiv:2309.00267, 2023

    2023 arXiv

  16. [24]

    Evolutionary-scale prediction of atomic-level protein structure with a language model.Science, 379(6637):1123–1130, 2023

    Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model.Science, 379(6637):1123–1130, 2023

    2023

  17. [25]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  18. [26]

    Progen: Language modeling for protein generation

    Ali Madani, Bryan McCann, Nikhil Naik, Nitish Shirish Keskar, Namrata Anand, Raphael R Eguchi, Po-Ssu Huang, and Richard Socher. Progen: Language modeling for protein generation. arXiv preprint arXiv:2004.03497, 2020

    2004 arXiv

  19. [27]

    Preference optimization of protein language models as a multi-objective binder design paradigm.arXiv preprint arXiv:2403.04187, 2024

    Pouria Mistani and Venkatesh Mysore. Preference optimization of protein language models as a multi-objective binder design paradigm.arXiv preprint arXiv:2403.04187, 2024

    2024 arXiv

  20. [28]

    Conditional language models enable the efficient design of proficient enzymes.bioRxiv, pages 2024–05, 2024

    Geraldene Munsamy, Ramiro Illanes-Vicioso, Silvia Funcillo, Ioanna T Nakou, Sebastian Lindner, Gavin Ayres, Lesley S Sheehan, Steven Moss, Ulrich Eckhard, Philipp Lorenz, et al. Conditional language models enable the efficient design of proficient enzymes.bioRxiv, pages 2024–05, 2024

    2024

  21. [29]

    Progen2: exploring the boundaries of protein language models.Cell systems, 14(11):968–978, 2023

    Erik Nijkamp, Jeffrey A Ruffolo, Eli N Weinstein, Nikhil Naik, and Ali Madani. Progen2: exploring the boundaries of protein language models.Cell systems, 14(11):968–978, 2023

    2023

  22. [30]

    Machine learning for functional protein design.Nature biotechnology, 42(2):216–228, 2024

    Pascal Notin, Nathan Rollins, Yarin Gal, Chris Sander, and Debora Marks. Machine learning for functional protein design.Nature biotechnology, 42(2):216–228, 2024

    2024

  23. [31]

    Cath–a hierarchic classification of protein domain structures.Structure, 5(8):1093–1109, 1997

    Christine A Orengo, Alex D Michie, Susan Jones, David T Jones, Mark B Swindells, and Janet M Thornton. Cath–a hierarchic classification of protein domain structures.Structure, 5(8):1093–1109, 1997

    1997

  24. [32]

    Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:27730–27744, 2022

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:27730...

    2022

  25. [33]

    Smaug: Fixing failure modes of preference optimisation with dpo-positive.arXiv preprint arXiv:2402.13228, 2024

    Arka Pal, Deep Karkhanis, Samuel Dooley, Manley Roberts, Siddartha Naidu, and Colin White. Smaug: Fixing failure modes of preference optimisation with dpo-positive.arXiv preprint arXiv:2402.13228, 2024. 11

    2024 arXiv

  26. [34]

    Improving inverse folding for peptide design with diversity-regularized direct preference optimization.arXiv preprint arXiv:2410.19471, 2024

    Ryan Park, Darren J Hsu, C Brian Roland, Maria Korshunova, Chen Tessler, Shie Mannor, Olivia Viessmann, and Bruno Trentini. Improving inverse folding for peptide design with diversity-regularized direct preference optimization.arXiv preprint arXiv:2410.19471, 2024

    arXiv

  27. [35]

    Instructplm: Aligning protein language models to follow protein structure instructions.bioRxiv, 2024

    Jiezhong Qiu, Junde Xu, Jie Hu, Hanqun Cao, Liya Hou, Zijun Gao, Xinyi Zhou, Anni Li, Xiujuan Li, Bin Cui, Fei Yang, Shuang Peng, Ning Sun, Fangyu Wang, Aimin Pan, Jie Tang, Jieping Ye, Junyang Lin, Jin Tang, Xingxu Huang, Pheng Ann Heng, and Guangyong Chen. Instructplm: Align...

    2024

  28. [36]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024

    2024

  29. [37]

    The use of alphafold for in silico exploration of drug targets in the parasite trypanosoma cruzi.Frontiers in Cellular and Infection Microbiology, 12:944748, 2022

    Albert Ros-Lucas, Nieves Martinez-Peinado, Jaume Bastida, Joaquim Gascón, and Julio Alonso- Padilla. The use of alphafold for in silico exploration of drug targets in the parasite trypanosoma cruzi.Frontiers in Cellular and Infection Microbiology, 12:944748, 2022

    2022

  30. [38]

    Ruffolo, Aadyot Bhatnagar, Joel Beazer, Stephen Nayfach, Jordan Russ, Emily Hill, Riffat Hussain, Joseph Gallagher, and Ali Madani

    Jeffrey A. Ruffolo, Aadyot Bhatnagar, Joel Beazer, Stephen Nayfach, Jordan Russ, Emily Hill, Riffat Hussain, Joseph Gallagher, and Ali Madani. Adapting protein language models for structure-conditioned design.bioRxiv, 2024

    2024

  31. [39]

    How good are alphafold models for docking-based virtual screening?Iscience, 26(1), 2023

    Valeria Scardino, Juan I Di Filippo, and Claudio N Cavasotto. How good are alphafold models for docking-based virtual screening?Iscience, 26(1), 2023

    2023

  32. [40]

    Protein structure determination from pseudocontact shifts using rosetta.Journal of molecular biology, 416(5):668–677, 2012

    Christophe Schmitz, Robert Vernon, Gottfried Otting, David Baker, and Thomas Huber. Protein structure determination from pseudocontact shifts using rosetta.Journal of molecular biology, 416(5):668–677, 2012

    2012

  33. [41]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  34. [42]

    Surfing the biocatalysis wave to new applications changed to: From nature to industry: Harnessing enzymes for biocatalysis, as per request of editor.Science, 382(6673), 2023

    Radka Snajdrova, Rebecca Buller, Stefan Lutz, Jeffrey Moore, Romas Kazlauskas, and Uwe Bornscheuer. Surfing the biocatalysis wave to new applications changed to: From nature to industry: Harnessing enzymes for biocatalysis, as per request of editor.Science, 382(6673), 2023

    2023

  35. [43]

    Guiding generative protein language models with reinforcement learning.arXiv preprint arXiv:2412.12979, 2024

    Filippo Stocco, Maria Artigues-Lleixa, Andrea Hunklinger, Talal Widatalla, Marc Guell, and Noelia Ferruz. Guiding generative protein language models with reinforcement learning.arXiv preprint arXiv:2412.12979, 2024

    2024

  36. [44]

    Global-context aware generative protein design.ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5, 2023

    Cheng Tan, Zhangyang Gao, Jun Xia, Bozhen Hu, and Stan Z Li. Global-context aware generative protein design.ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5, 2023

    2023

  37. [45]

    Evolution of function in protein superfamilies, from a structural perspective.Journal of molecular biology, 307(4):1113–1143, 2001

    Annabel E Todd, Christine A Orengo, and Janet M Thornton. Evolution of function in protein superfamilies, from a structural perspective.Journal of molecular biology, 307(4):1113–1143, 2001

    2001

  38. [46]

    Aligning protein generative models with experimental fitness via direct preference optimization.bioRxiv, pages 2024–05, 2024

    Talal Widatalla, Rafael Rafailov, and Brian Hie. Aligning protein generative models with experimental fitness via direct preference optimization.bioRxiv, pages 2024–05, 2024

    2024

  39. [47]

    Self-play preference optimization for language model alignment.arXiv preprint arXiv:2405.00675, 2024

    Yue Wu, Zhiqing Sun, Huizhuo Yuan, Kaixuan Ji, Yiming Yang, and Quanquan Gu. Self-play preference optimization for language model alignment.arXiv preprint arXiv:2405.00675, 2024

    2024 arXiv

  40. [48]

    Hallucidoctor: Mitigating hallucinatory toxicity in visual instruction data

    Qifan Yu, Juncheng Li, Longhui Wei, Liang Pang, Wentao Ye, Bosheng Qin, Siliang Tang, Qi Tian, and Yueting Zhuang. Hallucidoctor: Mitigating hallucinatory toxicity in visual instruction data. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, ...

    2024

  41. [49]

    Self-rewarding language models.arXiv preprint arXiv:2401.10020, 2024

    Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Sainbayar Sukhbaatar, Jing Xu, and Jason Weston. Self-rewarding language models.arXiv preprint arXiv:2401.10020, 2024. 12

    2024 arXiv

  42. [50]

    Evaluating large language models at evaluating instruction following.arXiv preprint arXiv:2310.07641, 2023

    Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. Evaluating large language models at evaluating instruction following.arXiv preprint arXiv:2310.07641, 2023

    2023 arXiv

  43. [51]

    Tm-align: a protein structure alignment algorithm based on the tm-score.Nucleic acids research, 33(7):2302–2309, 2005

    Yang Zhang and Jeffrey Skolnick. Tm-align: a protein structure alignment algorithm based on the tm-score.Nucleic acids research, 33(7):2302–2309, 2005

    2005

  44. [52]

    Structure- informed language models are protein designers.bioRxiv, pages 2023–02, 2023

    Zaixiang Zheng, Yifan Deng, Dongyu Xue, Yi Zhou, Fei Ye, and Quanquan Gu. Structure- informed language models are protein designers.bioRxiv, pages 2023–02, 2023. 13 A Technical Appendices and Supplementary Material A.1 Implementation Details A.1.1 Generation Configs for Evalua...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.