REVIEW 3 major objections 4 minor 40 references
Beyond Simple Concatenation: Fairly Assessing PLM Architectures for Multi-Chain Protein-Protein Interactions Prediction
T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Pooled and cross-attention adapters beat concatenation for protein-protein affinity prediction.
desk verdict Useful curated PPI affinity benchmark, but the 'consistently outperform' claim is contradicted by the paper's own numbers. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the two-step pooling hierarchy and the cross-attention residual block. HP's two-level attention pooler (intra-chain then inter-chain) keeps each chain's representation distinct before forming a partner-level vector, avoiding the dilution that follows pooling a long concatenated chain sequence. PAD's shared multi-head cross-attention lets each partner's residues attend to the other partner's residues, then adds that context back and sums the pooled partner embeddings, an explicit inductive bias for intermolecular interaction. The same 1D global attention pooling is reused in all architectures, so the comparisons isolate the aggregation strategy.
What would settle it
Retrain the four architectures on the original PPB-Affinity entries without SEQRES residue recovery and without averaging affinities; if HP and PAD no longer consistently beat EC and SC, the claimed architectural advantage is an artifact of the dataset reconstruction rather than a general property of the models.
Extended reading notes
Core claim
The authors claim that for adapting PLMs to multi-chain PPI affinity prediction, concatenating chains before or after encoding is systematically worse than architectures that preserve chain identity and model ligand-receptor interaction explicitly. HP encodes each chain separately, pools per-chain embeddings, then pools across chains; PAD applies cross-attention between ligand and receptor hidden states, adds the attention output back residually, pools, and sums the two partner embeddings instead of concatenating. Across ProtT5, ESM2, Ankh, Ankh2, and ESM3, with both full fine-tuning and frozen-feature ConvBERT heads, HP and PAD occupy the top tier while EC and SC rarely do, up to a 12% Spearman improvement.
Load-bearing premise
The evaluation rests on the reconstructed PPB-Affinity dataset: missing residues were filled from SEQRES records and affinity values were averaged across methods, and the paper itself notes these repairs create inconsistencies with the original PDB structures, so the architecture ranking is only as valid as that reconstruction.
Editorial extensions
If this is right
- For sequence-only PPI affinity pipelines, switching from concatenation to HP or PAD should improve ranking of candidate binders by Spearman correlation, by up to 12% relative in the tested setups.
- Full PLM fine-tuning is not always worth the cost: for ESM2, ESM3, and ProtT5, a frozen PLM with a ConvBERT head matches or nearly matches fine-tuning, while for Ankh and Ankh2 fine-tuning is needed.
- No single PLM is uniformly best; ProtT5 has the highest average correlation, but ESM2-650M and Ankh2-Ext1 win in specific HP setups, and larger ESM2-3B does not beat ESM2-650M.
- The curated dataset with ≤30% sequence identity splitting should reduce leakage, so models evaluated on it are being tested for generalization rather than memorization.
Reading between the lines
- If the architecture ranking holds on other affinity benchmarks, the same adapter designs could be adopted by tools that currently use concatenation.
- The HP advantage should be largest for complexes with more than one chain per partner; a per-subset breakdown by number of chains would test this directly and is not reported in the paper.
- The competitive frozen-feature ConvBERT results suggest that for very large PLMs, avoiding fine-tuning may make sophisticated adapters affordable in practice.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript introduces a curated version of the PPB-Affinity dataset for sequence-based multi-chain protein-protein interaction (PPI) binding affinity prediction, with a pipeline that corrects annotation errors, fills missing residues from SEQRES records, removes short chains, consolidates duplicates, and applies a stringent ≤30% sequence identity split. The authors then compare four PLM adaptation architectures—embedding concatenation (EC), sequence concatenation (SC), hierarchical pooling (HP), and pooled attention addition (PAD)—under full fine-tuning and a frozen-PLM ConvBERT head across several PLMs (ProtT5, ESM2 variants, Ankh, Ankh2, and ESM3). The central claim is that HP and PAD consistently outperform conventional concatenation methods, achieving up to 12% improvement in Spearman correlation. The paper also releases its code and dataset.
Significance. If fully supported, the paper would provide useful practical guidance for adapting protein language models to multi-chain PPI affinity prediction, and the released dataset with a low-leakage split would be a valuable community resource. The study's strengths include its reproducible pipeline and public code, the breadth of the PLM/training-scheme matrix, and the explicit attention to data leakage. However, the headline claim is not supported by the paper's own full results in Supplementary Table 3; the evidence supports a more modest conclusion that HP and PAD often help but not consistently, and that gains depend heavily on the PLM and training scheme.
major comments (3)
- [Abstract and Section 3, Table 3] The claim that HP and PAD 'consistently outperform' EC and SC is contradicted by the paper's full results. For example, ESM2-3B FT-SC reaches test Spearman 0.44±0.01, tying FT-PAD (0.44±0.01) and exceeding FT-HP (0.41±0.02) and FT-EC (0.41±0.02); ESM2-650M FT-SC (0.43±0.03) exceeds FT-PAD (0.39±0.02); Ankh2-Ext1 CV-SC (0.41±0.02) ties CV-PAD (0.41±0.02) and beats CV-HP (0.39±0.02); and Ankh2-Ext2 CV-SC (0.41±0.02) beats CV-HP (0.39±0.03). The 'up to 12%' increase appears to be a single best contrast (ProtT5 CV-PAD 0.48 vs CV-EC 0.43), not a systematic effect. The abstract and Section 3 should be revised to state that HP and PAD tend to help in many settings, with the specific favorable and unfavorable configurations identified.
- [Section 3, Table 3] No significance tests are reported for the architecture comparisons. With only three seeds and the reported standard deviations, many differences are likely within noise (e.g., ESM3-SM-Open CV-SC 0.35±0.02 vs CV-HP 0.35±0.01 vs CV-PAD 0.36±0.04). Since the central claim is about consistent superiority, the authors should report paired tests across seeds (or a mixed-effects model with architecture as a factor and PLM/setup interactions) and effect sizes. The current presentation of means without inference does not establish the claimed ranking.
- [Sections 2.1.2 and 4] The benchmark's validity rests on the reconstructed sequences: missing residues are filled from SEQRES records, duplicate affinity values are averaged across experimental methods, and Section 4 concedes that these repairs created inconsistencies with original PDB structures. Because every architecture comparison occurs on this reconstructed dataset, the possibility of sequence-label mismatches or artifacts is load-bearing. The authors should provide validation analyses, such as the distribution of recovered residues, a comparison of model performance with and without affected entries, or a consistency check of reconstructed sequences against independent structural information. Without such checks, the dataset's reliability for benchmarking is not fully established.
minor comments (4)
- [Table 3 caption] The caption begins 'Table presents...' and should read 'The table presents...' for grammatical consistency.
- [Abstract and Section 2.3] The term 'ConvBERT' is used in the abstract but introduced only in Section 2.3; please define or briefly describe it at first use.
- [Figure 2] The description of marginal means would be clearer if it stated whether they are unweighted averages across all cells; this affects how readers interpret the per-PLM and per-setup summaries.
- [Section 3] The 'fairly assessing' framing would be strengthened by including at least one external baseline (e.g., a published PPI affinity prediction method evaluated on the same split), even if it is not the main focus of the architecture comparison.
Circularity Check
Empirical benchmark study with no load-bearing circular derivation; the only self-citation is non-load-bearing.
full rationale
This paper is an empirical benchmark study, not a derivation. The central claim that HP and PAD architectures outperform EC and SC for PLM-based PPI affinity prediction is established by training models on a train split and measuring held-out test Spearman correlation from three seeds. No architecture parameter is fitted to the test split; validation-based early stopping is standard and does not reduce the test metric to a training input. The dataset construction, including the <=30% sequence identity split via MMseqs2, is an external preprocessing step whose split definition uses sequence identity and PDB ID coherence, not the affinity labels, so it is not self-definitional. The only self-citation is the choice of ConvBERT as the frozen-feature head: 'we employed a ConvBERT model [38] as the downstream head, selected for its previously reported state-of-the-art efficacy [12],' where [12] is the Ankh paper by overlapping authors. This is a baseline/head selection, not an input that forces the architecture ranking; the HP/PAD comparison is computed independently under both full fine-tuning and ConvBERT settings, and the relative ordering could have gone either way. The abstract's 'up to 12%' language is a selected maximum contrast and may overstate consistency, but that is a correctness or overclaim concern, not circularity. No equation or fitted parameter is reused as a prediction, and no uniqueness theorem or ansatz is imported from the authors' prior work to forbid alternatives. Therefore no load-bearing circular step is present.
Assumptions & free parameters
free parameters (3)
- Sequence identity threshold =
30%
- Chain length cutoff =
40 residues
- Affinity consolidation rule =
Average after dropping Unknown/Other methods
assumptions (3)
- domain assumption SEQRES-based gap filling preserves label fidelity
- domain assumption The concatenated-sequence split prevents information leakage sufficient for fair architecture comparison
- domain assumption Spearman correlation on this test set captures prediction quality
Cite this review
Pith. "Pith review of Beyond Simple Concatenation: Fairly Assessing PLM Architectures for Multi-Chain Protein-Protein Interactions Prediction." pith.science (2026). https://pith.science/paper/F6QSQ5WG
@misc{pith2026250520036,
author = {Pith},
title = {Pith review of: Beyond Simple Concatenation: Fairly Assessing PLM Architectures for Multi-Chain Protein-Protein Interactions Prediction},
year = {2026},
howpublished = {\url{https://pith.science/paper/F6QSQ5WG}},
note = {Machine review of arXiv:2505.20036}
}
read the original abstract
Protein-protein interactions (PPIs) are fundamental to numerous cellular processes, and their characterization is vital for understanding disease mechanisms and guiding drug discovery. While protein language models (PLMs) have demonstrated remarkable success in predicting protein structure and function, their application to sequence-based PPI binding affinity prediction remains relatively underexplored. This gap is often attributed to the scarcity of high-quality, rigorously refined datasets and the reliance on simple strategies for concatenating protein representations. In this work, we address these limitations. First, we introduce a meticulously curated version of the PPB-Affinity dataset of a total of 8,207 unique protein-protein interaction entries, by resolving annotation inconsistencies and duplicate entries for multi-chain protein interactions. This dataset incorporates a stringent, less than or equal to 30%, sequence identity threshold to ensure robust splitting into training, validation, and test sets, minimizing data leakage. Second, we propose and systematically evaluate four architectures for adapting PLMs to PPI binding affinity prediction: embeddings concatenation (EC), sequences concatenation (SC), hierarchical pooling (HP), and pooled attention addition (PAD). These architectures were assessed using two training methods: full fine-tuning and a lightweight approach employing ConvBERT heads over frozen PLM features. Our comprehensive experiments across multiple leading PLMs (ProtT5, ESM2, Ankh, Ankh2, and ESM3) demonstrated that the HP and PAD architectures consistently outperform conventional concatenation methods, achieving up to 12% increase in terms of Spearman correlation. These results highlight the necessity of sophisticated architectural designs to fully exploit the capabilities of PLMs for nuanced PPI binding affinity prediction.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
Pascal Braun and Anne-Claude Gingras. History of protein–protein interactions: From egg- white to complex networks.Proteomics, 12(10):1478–1498, 2012
work page 2012
-
[2]
Haiying Lu, Qiaodan Zhou, Jun He, Zhongliang Jiang, Cheng Peng, Rongsheng Tong, and Jianyou Shi. Recent advances in the development of protein–protein interactions modulators: mechanisms and clinical trials.Signal transduction and targeted therapy, 5(1):213, 2020
work page 2020
-
[3]
Shuyan Li, Lili Xi, Chengqi Wang, Jiazhong Li, Beilei Lei, Huanxiang Liu, and Xiaojun Yao. A novel method for protein-ligand binding affinity prediction and the related descriptors exploration.Journal of computational chemistry, 30(6):900–909, 2009
work page 2009
-
[4]
Ensembling methods for protein-ligand binding affinity prediction
Jiffriya Mohamed Abdul Cader, MA Hakim Newton, Julia Rahman, Akmal Jahan Mohamed Ab- dul Cader, and Abdul Sattar. Ensembling methods for protein-ligand binding affinity prediction. Scientific Reports, 14(1):24447, 2024
work page 2024
-
[5]
Jingtian Zhao, Yang Cao, and Le Zhang. Exploring the computational methods for protein- ligand binding site prediction.Computational and structural biotechnology journal, 18:417–426, 2020
work page 2020
-
[6]
Sangmin Seo, Jonghwan Choi, Sanghyun Park, and Jaegyoon Ahn. Binding affinity prediction for protein–ligand complex using deep attention mechanism based on intermolecular interactions. BMC bioinformatics, 22:1–15, 2021
work page 2021
-
[7]
Uniref clusters: a comprehensive and scalable alternative for improving sequence similarity searches.Bioinformatics, 31(6):926–932, 2015
Baris E Suzek, Yuqi Wang, Hongzhan Huang, Peter B McGarvey, Cathy H Wu, and UniProt Consortium. Uniref clusters: a comprehensive and scalable alternative for improving sequence similarity searches.Bioinformatics, 31(6):926–932, 2015
2015
-
[8]
Martin Steinegger, Milot Mirdita, and Johannes Söding. Protein-level assembly increases protein sequence recovery from metagenomic samples manyfold.Nature methods, 16(7):603–606, 2019
work page 2019
Show all 40 references
-
[9]
Alexander Rives, Joshua Meier, Tom Sercu, Siddharth Goyal, Zeming Lin, Jason Liu, Demi Guo, Myle Ott, C Lawrence Zitnick, Jerry Ma, et al. Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences.Proceedings of the National A...
2021
-
[10]
Prottrans: Towards cracking the language of life’s code through 500 self-supervised deep learning and high performance computing [j]
Ahmed Elnaggar, M Heinzinger, C Dallago, et al. Prottrans: Towards cracking the language of life’s code through 500 self-supervised deep learning and high performance computing [j]. IEEE Trans, 685, 2020
2020
-
[11]
Evolutionary-scale prediction of atomic-level protein structure with a language model.Science, 379(6637):1123–1130, 2023
Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, et al. Evolutionary-scale prediction of atomic-level protein structure with a language model.Science, 379(6637):1123–1130, 2023
2023
-
[12]
Ankh: Optimized protein language model unlocks general-purpose modelling.arXiv preprint arXiv:2301.06568, 2023
Ahmed Elnaggar, Hazem Essam, Wafaa Salah-Eldin, Walid Moustafa, Mohamed Elkerdawy, Charlotte Rochereau, and Burkhard Rost. Ankh: Optimized protein language model unlocks general-purpose modelling.arXiv preprint arXiv:2301.06568, 2023
2023 arXiv
-
[13]
ankh2-ext1 (revision 286cb6e), 2025
Elnaggar Lab. ankh2-ext1 (revision 286cb6e), 2025. URL https://huggingface.co/ ElnaggarLab/ankh2-ext1. 10
2025
-
[14]
ankh2-ext2 (revision 4c155ee), 2025
Elnaggar Lab. ankh2-ext2 (revision 4c155ee), 2025. URL https://huggingface.co/ ElnaggarLab/ankh2-ext2
2025
-
[15]
Sofroniew, Deniz Oktay, Zeming Lin, Robert Verkuil, Vincent Q
Thomas Hayes, Roshan Rao, Halil Akin, Nicholas J. Sofroniew, Deniz Oktay, Zeming Lin, Robert Verkuil, Vincent Q. Tran, Jonathan Deaton, Marius Wiggert, Rohil Badkundri, Irhum Shafkat, Jun Gong, Alexander Derry, Raul S. Molina, Neil Thomas, Yousuf A. Khan, Chetan Mishra, Caroly...
2024
-
[16]
Peer: a comprehensive and multi-task benchmark for protein sequence understanding.Advances in Neural Information Processing Systems, 35:35156–35173, 2022
Minghao Xu, Zuobai Zhang, Jiarui Lu, Zhaocheng Zhu, Yangtian Zhang, Ma Chang, Runcheng Liu, and Jian Tang. Peer: a comprehensive and multi-task benchmark for protein sequence understanding.Advances in Neural Information Processing Systems, 35:35156–35173, 2022
2022
-
[17]
D-script translates genome to phenome with sequence-based, structure-aware, genome-scale predictions of protein-protein interactions.Cell Systems, 12(10):969–982, 2021
Samuel Sledzieski, Rohit Singh, Lenore Cowen, and Bonnie Berger. D-script translates genome to phenome with sequence-based, structure-aware, genome-scale predictions of protein-protein interactions.Cell Systems, 12(10):969–982, 2021
2021
-
[18]
Topsy- turvy: integrating a global view into sequence-based ppi prediction.Bioinformatics, 38 (Supplement_1):i264–i272, 2022
Rohit Singh, Kapil Devkota, Samuel Sledzieski, Bonnie Berger, and Lenore Cowen. Topsy- turvy: integrating a global view into sequence-based ppi prediction.Bioinformatics, 38 (Supplement_1):i264–i272, 2022
2022
-
[19]
Saprot: Protein language modeling with structure-aware vocabulary.bioRxiv, pages 2023–10, 2023
Jin Su, Chenchen Han, Yuyang Zhou, Junjie Shan, Xibin Zhou, and Fajie Yuan. Saprot: Protein language modeling with structure-aware vocabulary.bioRxiv, pages 2023–10, 2023
2023
-
[20]
Multifaceted protein–protein interaction prediction based on siamese residual rcnn.Bioinformatics, 35(14):i305–i314, 2019
Muhao Chen, Chelsea J-T Ju, Guangyu Zhou, Xuelu Chen, Tianran Zhang, Kai-Wei Chang, Carlo Zaniolo, and Wei Wang. Multifaceted protein–protein interaction prediction based on siamese residual rcnn.Bioinformatics, 35(14):i305–i314, 2019
2019
-
[21]
Mu- tation effect estimation on protein–protein interactions using deep contextualized representation learning.NAR genomics and bioinformatics, 2(2):lqaa015, 2020
Guangyu Zhou, Muhao Chen, Chelsea JT Ju, Zheng Wang, Jyun-Yu Jiang, and Wei Wang. Mu- tation effect estimation on protein–protein interactions using deep contextualized representation learning.NAR genomics and bioinformatics, 2(2):lqaa015, 2020
2020
-
[22]
Ddmut-ppi: predicting effects of mutations on protein–protein interactions using graph-based deep learning
Yunzhuo Zhou, YooChan Myung, Carlos HM Rodrigues, and David B Ascher. Ddmut-ppi: predicting effects of mutations on protein–protein interactions using graph-based deep learning. Nucleic Acids Research, 52(W1):W207–W214, 2024
2024
-
[23]
Ppb-affinity: Protein-protein binding affinity dataset for ai-based protein drug discovery.Scientific Data, 11(1):1–11, 2024
Huaqing Liu, Peiyi Chen, Xiaochen Zhai, Ku-Geng Huo, Shuxian Zhou, Lanqing Han, and Guoxin Fan. Ppb-affinity: Protein-protein binding affinity dataset for ai-based protein drug discovery.Scientific Data, 11(1):1–11, 2024
2024
-
[24]
Datasets: A community library for natural language processing.arXiv preprint arXiv:2109.02846, 2021
Quentin Lhoest, Albert Villanova Del Moral, Yacine Jernite, Abhishek Thakur, Patrick V on Platen, Suraj Patil, Julien Chaumond, Mariama Drame, Julien Plu, Lewis Tunstall, et al. Datasets: A community library for natural language processing.arXiv preprint arXiv:2109.02846, 2021
2021 arXiv
-
[25]
Skempi 2.0: an updated benchmark of changes in protein–protein binding energy, kinetics and thermodynamics upon mutation.Bioinformatics, 35(3):462–469, 2019
Justina Jankauskait˙e, Brian Jiménez-García, Justas Dapk¯unas, Juan Fernández-Recio, and Iain H Moal. Skempi 2.0: an updated benchmark of changes in protein–protein binding energy, kinetics and thermodynamics upon mutation.Bioinformatics, 35(3):462–469, 2019
2019
-
[26]
Proteinflow: a python library to pre-process protein structure data for deep learning applications
Elizaveta Kozlova, Arthur Valentin, Aous Khadhraoui, and Daniel Nakhaee-Zadeh Gutierrez. Proteinflow: a python library to pre-process protein structure data for deep learning applications. bioRxiv, pages 2023–09, 2023
2023
-
[27]
Deeploc: prediction of protein subcellular localization using deep learning
José Juan Almagro Armenteros, Casper Kaae Sønderby, Søren Kaae Sønderby, Henrik Nielsen, and Ole Winther. Deeploc: prediction of protein subcellular localization using deep learning. Bioinformatics, 33(21):3387–3395, 2017
2017
-
[28]
Deeploc 2.0: multi-label subcellular localization prediction using protein language models.Nucleic acids research, 50(W1):W228–W234, 2022
Vineet Thumuluri, José Juan Almagro Armenteros, Alexander Rosenberg Johansen, Henrik Nielsen, and Ole Winther. Deeploc 2.0: multi-label subcellular localization prediction using protein language models.Nucleic acids research, 50(W1):W228–W234, 2022. 11
2022
-
[29]
Mutabind2: predicting the impacts of single and multiple mutations on protein-protein interactions.Iscience, 23(3), 2020
Ning Zhang, Yuting Chen, Haoyu Lu, Feiyang Zhao, Roberto Vera Alvarez, Alexander Goncearenco, Anna R Panchenko, and Minghui Li. Mutabind2: predicting the impacts of single and multiple mutations on protein-protein interactions.Iscience, 23(3), 2020
2020
-
[30]
Saambe-3d: predicting effect of mutations on protein–protein interactions
Swagata Pahari, Gen Li, Adithya Krishna Murthy, Siqi Liang, Robert Fragoza, Haiyuan Yu, and Emil Alexov. Saambe-3d: predicting effect of mutations on protein–protein interactions. International journal of molecular sciences, 21(7):2563, 2020
2020
-
[31]
Revealing data leakage in protein interaction benchmarks.arXiv preprint arXiv:2404.10457, 2024
Anton Bushuiev, Roman Bushuiev, Jiri Sedlar, Tomas Pluskal, Jiri Damborsky, Stanislav Mazurenko, and Josef Sivic. Revealing data leakage in protein interaction benchmarks.arXiv preprint arXiv:2404.10457, 2024
2024 arXiv
-
[32]
Quantification of biases in predictions of protein–protein binding affinity changes upon mutations.Briefings in bioinformatics, 25(1): bbad491, 2024
Matsvei Tsishyn, Fabrizio Pucci, and Marianne Rooman. Quantification of biases in predictions of protein–protein binding affinity changes upon mutations.Briefings in bioinformatics, 25(1): bbad491, 2024
2024
-
[33]
Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets.Nature biotechnology, 35(11):1026–1028, 2017
Martin Steinegger and Johannes Söding. Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets.Nature biotechnology, 35(11):1026–1028, 2017
2017
-
[34]
Robust deep learning–based protein sequence design using proteinmpnn.Science, 378(6615):49–56, 2022
Justas Dauparas, Ivan Anishchenko, Nathaniel Bennett, Hua Bai, Robert J Ragotte, Lukas F Milles, Basile IM Wicky, Alexis Courbet, Rob J de Haas, Neville Bethel, et al. Robust deep learning–based protein sequence design using proteinmpnn.Science, 378(6615):49–56, 2022
2022
-
[35]
Evaluating protein transfer learning with tape.Advances in neural information processing systems, 32, 2019
Roshan Rao, Nicholas Bhattacharya, Neil Thomas, Yan Duan, Peter Chen, John Canny, Pieter Abbeel, and Yun Song. Evaluating protein transfer learning with tape.Advances in neural information processing systems, 32, 2019
2019
-
[36]
Deepsol: a deep learning framework for sequence-based protein solubility prediction
Sameer Khurana, Reda Rawi, Khalid Kunji, Gwo-Yu Chuang, Halima Bensmail, and Raghven- dra Mall. Deepsol: a deep learning framework for sequence-based protein solubility prediction. Bioinformatics, 34(15):2605–2613, 2018
2018
-
[37]
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. V on Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors,Advances in Neural ...
2017
-
[38]
Con- vbert: Improving bert with span-based dynamic convolution.Advances in Neural Information Processing Systems, 33:12837–12848, 2020
Zi-Hang Jiang, Weihao Yu, Daquan Zhou, Yunpeng Chen, Jiashi Feng, and Shuicheng Yan. Con- vbert: Improving bert with span-based dynamic convolution.Advances in Neural Information Processing Systems, 33:12837–12848, 2020
2020
-
[39]
PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation
Jason Ansel, Edward Yang, Horace He, Natalia Gimelshein, Animesh Jain, Michael V oznesen- sky, Bin Bao, Peter Bell, David Berard, Evgeni Burovski, Geeta Chauhan, Anjali Chourdia, Will Constable, Alban Desmaison, Zachary DeVito, Elias Ellison, Will Feng, Jiong Gong, Michael Gsc...
2024
-
[40]
Accurate structure prediction of biomolecular interactions with alphafold 3.Nature, 630(8016):493–500, 2024
Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ronneberger, Lindsay Willmore, Andrew J Ballard, Joshua Bambrick, et al. Accurate structure prediction of biomolecular interactions with alphafold 3.Nature, 630(8016):493–500, 2024. 12 A...
2024
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.