Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Flash Invariant Point Attention

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read FlashIPA makes Invariant Point Attention scale linearly in memory.

desk verdict The lifted attention reformulation is real and useful, but the wall-clock linear scaling claim overreaches because compute stays O(L^2). read the letter →

arxiv 2505.11580 v1 pith:V7SWDBCX submitted 2025-05-16 cs.LG cs.AIq-bio.BM

Andrew Liu , Axel Elaldi , Nicholas T Franklin , Nathan Russell , Gurinder S Atwal , Yih-En A Ban , Olivia Viessmann This is my paper · ORCID
classification cs.LGcs.AIq-bio.BM
keywords invariantpointattentionFlashSE(3)invariancepairrepresentationfactorizationk-nearest-neighbordistogramproteinbackbonegenerationRNAstructurelinearmemoryscaling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FlashIPA rewrites Invariant Point Attention (IPA) so the entire attention update becomes a single inner product on lifted query and key vectors, allowing FlashAttention to run it without ever materializing the quadratic attention matrix. The paper claims this gives linear scaling in GPU memory and wall-clock time with sequence length, while matching or exceeding the validation performance of standard IPA on retrained protein and RNA generative models. A factorized pair representation — $z_{ij} = z1_i^{\top} z2_j$ with rank $r=2$ plus a $k=20$ nearest-neighbor distogram — is the approximation that makes the rewrite possible. If the claim holds, IPA-based structure models can train on full-length biomolecules instead of cropped chains.

What carries the argument

The load-bearing identity expands the squared-norm term in the IPA logit, $\|T_i \circ q_i^p - T_j \circ k_j^p\|^2 = \|T_i \circ q_i^p\|^2 - 2 (T_i \circ q_i^p)\cdot(T_j \circ k_j^p) + \|T_j \circ k_j^p\|^2$, and absorbs the cross term into a dot product between rotated query points and rotated key points. The lifted query $\hat{q}^h_i$ stacks the scalar query, the rotated point queries, their squared norms, a vector of ones, and the first pair factor; the lifted key $\hat{k}^h_j$ stacks the scaled scalar key, scaled rotated point keys, scaled squared norms, scaled negative constants, and the second pair factor. With $z_{ij} = z1_i^{\top} z2_j$ and a $k$-NN distogram, this turns the whole update into FlashAttention on lifted vectors, which is what carries the memory and runtime savings while retaining the SE(3)-invariant structure.

What would settle it

Take a structure model whose accuracy depends on long-range contacts, train it with FlashIPA's rank-2/k-20 factorized pair representation and with the original dense IPA under identical budgets, and compare on a benchmark rich in inter-domain or multi-chain contacts; if the FlashIPA model's accuracy (e.g., sc-RMSD or TM-score) falls noticeably behind at any length where both fit in memory, the 'matches or exceeds' claim is falsified for that regime. Alternatively, measure FlashIPA wall-clock at L=50k and L=100k on the same GPU; if runtime quadruples when length doubles, the linear wall-clock finding would be limited to the tested range.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the IPA update can be rewritten exactly as standard dot-product attention on 'lifted' vectors, so the entire geometric logit — the scalar query-key product, the squared distances between rotated point queries and keys, and the pair bias — collapses into a single inner product between an augmented query and an augmented key. With the pair tensor factorized as $z_{ij} = z1_i^{\top} z2_j$ (the authors use rank $r=2$) and the distogram restricted to the $k=20$ nearest neighbors with positional encodings, the layer becomes ordinary FlashAttention on these lifted vectors, never materializing the $O(L^2)$ attention matrix. The paper reports that retrained protein and RNA flow-matching models match or exceed the original IPA's validation performance, that training without the usual 512-residue cutoff yields lower protein self-consistency RMSD, and that structures of thousands of residues can be generated, while memory usage and wall-clock time scale approximately linearly in sequence length.

Load-bearing premise

The factorized pair representation — rank $r=2$ and a $k=20$ nearest-neighbor distogram — must capture enough of what the dense pair tensor contributes; if a downstream task genuinely needs long-range or dense pairwise features, FlashIPA will not match standard IPA's performance.

Editorial extensions

If this is right

  • IPA-based protein and RNA generative models can be trained on full-length chains without cropping; the paper demonstrates protein chains up to 8.8k residues and RNA up to 4,417 nucleotides.
  • FoldFlow trained with FlashIPA on all monomer data reaches lower self-consistency RMSD than either standard IPA or FlashIPA trained with the 512-residue cutoff.
  • RNA-FrameFlow with FlashIPA matches the original model's validity, diversity, and novelty scores at roughly a quarter of the compute cost, including a single-GPU training run.
  • With the pair factorization and k-NN distogram, local geometry is captured more efficiently; the paper reports steric-clash losses decreasing faster for FlashIPA than IPA.
  • The reformulation is drop-in compatible with existing IPA-based repositories, so other models in the IPA ecosystem can adopt it directly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the paper's own limitation states that compute remains $O(L^2)$ due to the softmax, the 'linear wall-clock' is an engineering finding for tested lengths and hardware, not an asymptotic FLOP reduction; at sufficiently large $L$ the quadratic compute term should eventually dominate.
  • The $k=20$ nearest-neighbor distogram suggests FlashIPA will be most faithful on tasks where local contacts dominate; structure prediction tasks that hinge on long-range contacts, such as inter-domain interfaces, may be where dense pair representations are more likely to be needed.
  • The same factorization trick could be applied to other quadratic geometric bias terms, such as triangular attention in AF-3-style models, potentially extending linear-memory training to full multi-chain complexes.
  • A natural next experiment is to check whether the rank-2 factorization ceiling couples with the FlashAttention head-dimension limit: raising the head-dimension cap would permit higher rank and denser pair information, which may close any residual gap against quadratic IPA on contact-heavy tasks.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces FlashIPA, a factorized reformulation of Invariant Point Attention (IPA) that rewrites the softmax logits in Equation (3) as a single inner product over lifted query/key/value vectors, allowing the use of FlashAttention kernels. The authors claim that FlashIPA achieves linear scaling in GPU memory and wall-clock time with sequence length, matches or exceeds standard IPA performance, and enables training on biomolecular structures of thousands of residues. They demonstrate this by retraining FoldFlow for proteins and RNA-FrameFlow for RNAs, and by providing an open-source implementation. The paper also discusses factorization of the pair representation using low-rank factors and k-nearest-neighbor distograms.

Significance. If the claims hold, FlashIPA could substantially lower the cost of training IPA-based models on full-length biomolecules, which would be an important practical contribution to structural biology and geometric deep learning. The algebraic derivation is transparent and correct: the expansion of the squared norm in Equation (3) and the subsequent lifting of queries, keys, and values are sound. The memory-scaling experiments support the claim of approximately O(L) GPU memory usage for FlashIPA in the tested range. The open-source release is a concrete strength that will facilitate adoption and reproducibility. However, the paper's headline wall-clock scaling claim is overstated relative to its own limitations section, and the performance comparisons are partly confounded by architectural and training-schedule differences. The core idea is promising but needs more careful qualification and additional experiments before the broad claims can be accepted.

major comments (3)
  1. [Abstract; §2.3–§2.4; §4.1] The claim of 'linear scaling in GPU memory and wall-clock time with sequence length' conflates I/O complexity with FLOP complexity. FlashAttention does not change the O(L^2) FLOP count of the softmax argument in Equation (3); §4.1 explicitly states that 'the underlying compute cost is still O(L^2) due to the softmax.' The memory-scaling fit in Fig. 2A supports a linear memory trend for FlashIPA in the tested range, but no analogous runtime scaling fit, exponent, or error bars are reported for Fig. 2B. Since the asymptotic wall-clock time must be quadratic, the abstract's unqualified statement is not supportable. Please either restrict the claim to 'linear memory and I/O with substantial constant-factor speedups in the tested finite-length range' or provide an asymptotic runtime analysis that justifies a linear-time claim.
  2. [§3.3] The FoldFlow comparison is confounded by architectural and training-schedule differences. The FlashIPA model uses hidden dimension 128 with 5 blocks and a linear effective batch-size schedule, while the original model uses hidden dimension 256 with 4 blocks and a quadratic effective batch-size schedule. The text first states 'We kept model and train parameters identical' and then immediately lists two adjustments, which is contradictory. The faster convergence and improved sc-RMSD in Fig. 3A cannot be attributed to FlashIPA alone, since the larger effective batch size alone would be expected to accelerate convergence. Please provide a matched-hyperparameter ablation (same hidden size, same number of blocks, same batch-size schedule) or clearly frame the comparison as 'FlashIPA with re-tuned settings' rather than as evidence that FlashIPA matches or exceeds IPA under identical conditions.
  3. [§3.4; Table 1] The RNA-FrameFlow results do not fully support the abstract's claim that FlashIPA 'matches or exceeds standard IPA performance.' In Table 1, the FlashIPA variants show lower mean validity (0.38 vs 0.42 and 0.41 vs 0.42) and noticeably lower diversity (0.08 vs 0.15 for the single-GPU run) than the original model. The paper describes these as 'comparable,' which is a reasonable interpretation, but this is not a demonstration of exceeding IPA. Please either temper the abstract to say 'matches in most settings' or provide statistical significance testing or additional evidence that these differences are within noise.
minor comments (5)
  1. [§3.1; Fig. 2B] Fig. 2B shows wall-clock runtime but does not include a fitted scaling curve, exponent, or error bars. Please add a fitted model (e.g., a power-law fit) and report error bars so the reader can assess the scaling behavior quantitatively.
  2. [§2.4; Algorithm 1] The definitions of the lifted vectors in Algorithm 1 use NQuery for queries and keys and NValue for values, but the text preceding the algorithm uses Nquery and Nvalue inconsistently. Please check the subscripts and ensure the dimensions in the concat operations are consistent with the head-dimension constraint c+5Nquery+rdz and c+3Nvalue+rdz.
  3. [§3.3] The statement 'FlashAttention becomes incompatible with DDP at that dimension' is unexplained. Please provide details or a reference; otherwise readers cannot evaluate whether this is a fundamental limitation or an implementation-specific issue.
  4. [§2.5] The choice of k=20 nearest neighbors for the distogram and rank r=2 for the pair-factorization is reported, but no sensitivity analysis is given. Consider reporting performance for a small range of k and r values to demonstrate that the results are not sensitive to these hyperparameters.
  5. [Throughout] The manuscript contains several typos and grammatical errors: 'approixmate' (Section 3.1), 'strucutre' (Fig. 3), 'lenghth' (Fig. 5), 'permit usage' (Section 4), 'recasted' (Section 1). Please proofread carefully.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: FlashIPA's factorized attention is a self-contained algebraic reformulation, and the performance claims rest on external retraining benchmarks rather than on fitted inputs.

full rationale

The derivation chain in FlashIPA is self-contained algebra plus an explicit approximation, not a reduction of predictions to inputs. Section 2.4 rewrites the original IPA softmax argument from Eq. (3) by expanding the sum of squared norms and factorizing the pair representation as z_ij = z1_i^T z2_j, then lifting q, k, v into higher-dimensional vectors so the update becomes a plain inner product amenable to FlashAttention (Algorithm 1). This is an algebraic equivalence; no fitted constant is later reported as a predicted result. The low-rank factorization (rank 2) and k=20 nearest-neighbor distogram are tuned on validation performance in Section 3.2, which is ordinary model selection rather than circular reasoning. The performance claims are established by retraining FoldFlow and RNA-FrameFlow with the original IPA implementations, matching the original authors' hyperparameters, so 'matches or exceeds' is an external empirical benchmark. The scaling claims do contain an internal inconsistency: the abstract and introduction state linear scaling in wall-clock time, while Section 4.1 concedes that 'the underlying compute cost is still O(L^2) due to the softmax,' meaning the observed wall-clock linearity is an I/O-bound finite-range effect rather than an asymptotic FLOP reduction. That is an overstatement about asymptotic complexity, but it is not circularity: the memory and runtime measurements are empirical fits, not quantities forced by construction. There is no self-citation used as load-bearing evidence, no imported uniqueness theorem, and no ansatz disguised as an external result. Therefore the paper's central derivation is not circular.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard FlashAttention assumptions, SE(3)-invariance of the original IPA, and the empirical premise that pair representations can be low-rank approximated. The factorization rank and k-NN cutoff are tuned on validation loss, so they are free parameters of the method.

free parameters (3)
  • factorization rank r = 2
    Section 3.2: rank in {1,2} tested on FoldFlow and RNA-FrameFlow convergence; rank 2 chosen to match or surpass IPA loss curves.
  • nearest-neighbor distance cutoff k = 20
    Section 3.2: pair distograms replaced by k=20 nearest neighbors; chosen empirically to match IPA performance.
  • effective batch-size multiplier = 20,000 x n_GPUs / N
    Section 3.3: linear effective batch-size schedule selected so FlashIPA training uses similar GPU memory as original IPA; affects convergence speed and comparisons.
assumptions (4)
  • standard math Algebraic expansion of the squared-distance term in eq. (3) is valid, so the lifted qhat/khat dot product reproduces the original attention logits up to a per-pair constant and the factorized bias.
    Section 2.4, Algorithm 1; this is the core derivation, checked by expansion; no external proof needed.
  • domain assumption Original IPA is SE(3)-invariant and FlashIPA inherits invariance when the pair features are invariant.
    Section 3.1; numerical invariance test only, deviation <1e-3, and no full proof is supplied for the factorized variant.
  • domain assumption Pair representations in common IPA implementations are redundant enough to be approximated by low-rank factors and k-NN distances.
    Section 2.5; empirical motivation, not proven in general.
  • domain assumption FlashAttention implementations accept the lifted head size and compute exact softmax in tiled form.
    Section 2.4; the method depends on a FlashAttention kernel that supports the required head dimension, with noted max-head-dim limits in Section 4.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Flash Invariant Point Attention." pith.science (2026). https://pith.science/paper/V7SWDBCX

@misc{pith2026250511580,
  author       = {Pith},
  title        = {Pith review of: Flash Invariant Point Attention},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V7SWDBCX}},
  note         = {Machine review of arXiv:2505.11580}
}
read the original abstract

Invariant Point Attention (IPA) is a key algorithm for geometry-aware modeling in structural biology, central to many protein and RNA models. However, its quadratic complexity limits the input sequence length. We introduce FlashIPA, a factorized reformulation of IPA that leverages hardware-efficient FlashAttention to achieve linear scaling in GPU memory and wall-clock time with sequence length. FlashIPA matches or exceeds standard IPA performance while substantially reducing computational costs. FlashIPA extends training to previously unattainable lengths, and we demonstrate this by re-training generative models without length restrictions and generating structures of thousands of residues. FlashIPA is available at https://github.com/flagshippioneering/flash_ipa.

Figures

Figures reproduced from arXiv: 2505.11580 by the authors.

Figure 1
Figure 1. The protein back￾bone frame: The Cα is the centre. The vectors spanning Cα − N and Cα − C define the third axis via Gram–Schmidt. The oxygen atom O is parameterized via the torsion angle ψ around the Cα − C axis. Meaningful frames for biomolecules are commonly defined on the backbone structures. In proteins, each residue frame is typically determined by three of the four atoms from the backbone, namely the alpha car… view at source ↗
Figure 2
Figure 2. We use a polynomial fit (green and red dotted lines), and find an approixmate GPU memory scaling [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 2
Figure 2. Scaling as a function of input sequence length on a single-sample batch forward pass. [A] GPU memory usage in GB. Original IPA scaled approximately quadratically with sequence length (y [MB] = 2.4 × 10−3 · L 2 + 1.4 · 10−2 · L), FlashIPA follows a linear trend (y [MB] = −7 · 10−12 · L 2 + 7.5 · 10−2 · L). [B] Wall-clock time in seconds. by the original authors (embedding sizes, hidden dimensions, number of heads, et… view at source ↗
Figures from the paper (8 more)
Figure 3
Figure 3. Figure 3: FoldFlow self-consistency validation after 200k optimization steps. A) The sc-RMSD of the FlashIPA (red, green) models is consistent or better than the original IPA model (blue). Extending training to larger structures with FlashIPA further improves sc-RMSD. B) Three e…
Figure 5
Figure 5. Figure 5: We observe comparable scores between all models. In particular, the single-GPU training matches [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 4
Figure 4. Figure 4: Scaling of FlashIPA versus IPA for RNA generation using RNA-FrameFlow model, with a number of diffusion timestep NT = 50. [A] Impact of the generated sequence length on the generation runtime, using a batch size of 1. [B] Impact of the generated batch size on the gener…
Figure 5
Figure 5. Figure 5: RNA-FrameFlow generated RNAs results. [A] Comparison of the scTM score depending on the generated RNA sequence length and the IPA module used. The RNA-FrameFlow and RNA-FrameFlow + FlashIPA models are both trained on only short sequences ≤ 150, while the All data model…
Figure 6
Figure 6. Figure 6: Distribution of protein residue counts of the proteins resolved in the PDB (Figure reproduced and adapted from the data at RCSB PDB Statistics: Sequence length distribution; https://www.rcsb.org/stats/distribution-residue-count; accessed 6 May 2025). 11 [PITH_FULL_IMA…
Figure 7
Figure 7. Figure 7: Loss behaviour for FoldFlow model training. 12 [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Distribution of nucleotide residue counts of the RNA in the RNASolo2 dataset, filtering out the short structures of length < 40 nucleotides. The training cut-off of RNA Flow discards all sequences of length > 150, accounting for 30% of the dataset. A.6 RNA-FrameFlow tr…
Figure 9
Figure 9. Figure 9: Loss behaviour for RNA-FrameFlow model training. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Design-CP: Context Parallelism for Design of Protein Nanoparticles

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Context-parallel inference for RFdiffusion 3 enables end-to-end all-atom design of large symmetric protein nanoparticles on multi-GPU hardware without retraining.

Reference graph

Works this paper leans on

32 extracted references · 17 canonical work pages · cited by 1 Pith paper

  1. [1]

    Highly accurate protein structure prediction with alphafold.nature, 596(7873):583–589, 2021

    John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold.nature, 596(7873):583–589, 2021

    2021

  2. [2]

    Protein complex prediction with alphafold-multimer.biorxiv, pages 2021–10, 2021

    Richard Evans, Michael O’Neill, Alexander Pritzel, Natasha Antropova, Andrew Senior, Tim Green, Augustin Žídek, Russ Bates, Sam Blackwell, Jason Yim, et al. Protein complex prediction with alphafold-multimer.biorxiv, pages 2021–10, 2021

    2021

  3. [3]

    Openfold: Retraining alphafold2 yields new insights into its learning mechanisms and capacity for generalization.Nature Methods, 21(8):1514–1524, 2024

    Gustaf Ahdritz, Nazim Bouatta, Christina Floristean, Sachin Kadyan, Qinghui Xia, William Gerecke, Timothy J O’Donnell, Daniel Berenberg, Ian Fisk, Niccolò Zanichelli, et al. Openfold: Retraining alphafold2 yields new insights into its learning mechanisms and capacity for generalization.Nature Methods, 21(8):1514–1524, 2024. 8 Flash Invariant Point Attenti...

    work page 2024

  4. [4]

    Language models of protein sequences at the scale of evolution enable accurate structure prediction.BioRxiv, 2022:500902, 2022

    Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Sal Candido, et al. Language models of protein sequences at the scale of evolution enable accurate structure prediction.BioRxiv, 2022:500902, 2022

    2022

  5. [5]

    Accurate rna 3d structure prediction using a language model-based deep learning approach.Nature Methods, pages 1–12, 2024

    Tao Shen, Zhihang Hu, Siqi Sun, Di Liu, Felix Wong, Jiuming Wang, Jiayang Chen, Yixuan Wang, Liang Hong, Jin Xiao, et al. Accurate rna 3d structure prediction using a language model-based deep learning approach.Nature Methods, pages 1–12, 2024

    2024

  6. [6]

    Se (3) diffusion model with application to protein backbone generation.arXiv preprint arXiv:2302.02277, 2023

    Jason Yim, Brian L Trippe, Valentin De Bortoli, Emile Mathieu, Arnaud Doucet, Regina Barzilay, and Tommi Jaakkola. Se (3) diffusion model with application to protein backbone generation.arXiv preprint arXiv:2302.02277, 2023

    arXiv 2023

  7. [7]

    Jason Yim, Andrew Campbell, Emile Mathieu, Andrew Y. K. Foong, Michael Gastegger, Jose Jimenez- Luna, Sarah Lewis, Victor Garcia Satorras, Bastiaan S. Veeling, Frank Noe, Regina Barzilay, and Tommi Jaakkola. Improved motif-scaffolding with SE(3) flow matching.Transactions on Machine Learning Research, 2024

    work page 2024

  8. [8]

    Fast protein backbone generation with se (3) flow matching.arXiv preprint arXiv:2310.05297, 2023

    Jason Yim, Andrew Campbell, Andrew YK Foong, Michael Gastegger, José Jiménez-Luna, Sarah Lewis, Victor Garcia Satorras, Bastiaan S Veeling, Regina Barzilay, Tommi Jaakkola, et al. Fast protein backbone generation with se (3) flow matching.arXiv preprint arXiv:2310.05297, 2023

    arXiv

Show all 32 references
  1. [9]

    Sequence- augmented se (3)-flow matching for conditional protein backbone generation.Advances in neural information processing systems, 2024

    Guillaume Huguet, James Vuckovic, Kilian Fatras, Eric Thibodeau-Laufer, Pablo Lemos, Riashat Islam, Cheng-Hao Liu, Jarrid Rector-Brooks, Tara Akhound-Sadegh, Michael Bronstein, et al. Sequence- augmented se (3)-flow matching for conditional protein backbone generation.Advances...

    2024

  2. [10]

    Se (3)-stochastic flow matching for protein backbone generation

    Joey Bose, Tara Akhound-Sadegh, Guillaume Huguet, Kilian FATRAS, Jarrid Rector-Brooks, Cheng- Hao Liu, Andrei Cristian Nica, Maksym Korablyov, Michael M Bronstein, and Alexander Tong. Se (3)-stochastic flow matching for protein backbone generation. InThe Twelfth International ...

    2024

  3. [11]

    Framedipt: Se(3) diffusion model for protein structure inpainting.bioRxiv, 2023

    Cheng Zhang, Adam Leach, Thomas Makkink, Miguel Arbesú, Ibtissem Kadri, Daniel Luo, Liron Mizrahi, Sabrine Krichen, Maren Lang, Andrey Tovchigrechko, Nicolas Lopez Carranza, U˘ gur ¸ Sahin, Karim Beguir, Michael Rooney, and Yunguan Fu. Framedipt: Se(3) diffusion model for prot...

    2023

  4. [12]

    Proteus: exploring protein structure generation for enhanced designability and efficiency.bioRxiv, pages 2024–02, 2024

    Chentong Wang, Yannan Qu, Zhangzhi Peng, Yukai Wang, Hongli Zhu, Dachuan Chen, and Longxing Cao. Proteus: exploring protein structure generation for enhanced designability and efficiency.bioRxiv, pages 2024–02, 2024

    2024

  5. [13]

    Floating anchor diffusion model for multi-motif scaffolding

    Ke Liu, Weian Mao, Shuaike Shen, Xiaoran Jiao, Zheng Sun, Hao Chen, and Chunhua Shen. Floating anchor diffusion model for multi-motif scaffolding. 2024

    2024

  6. [14]

    Generating novel, designable, and diverse protein structures by equivariantly diffusing oriented residue clouds.arXiv preprint arXiv:2301.12485, 2023

    Yeqing Lin and Mohammed AlQuraishi. Generating novel, designable, and diverse protein structures by equivariantly diffusing oriented residue clouds.arXiv preprint arXiv:2301.12485, 2023

    2023 arXiv

  7. [15]

    De novo antibody design with se (3) diffusion.Journal of Computational Biology, 2024

    Daniel Cutting, Frédéric A Dreyer, David Errington, Constantin Schneider, and Charlotte M Deane. De novo antibody design with se (3) diffusion.Journal of Computational Biology, 2024

    2024

  8. [16]

    Generating highly designable proteins with geometric algebra flow matching

    Simon Wagner, Leif Seute, Vsevolod Viliuga, Nicolas Wolf, Frauke Gräter, and Jan Stühmer. Generating highly designable proteins with geometric algebra flow matching. InThirty-eighth Conference on Neural Information Processing Systems, 2024

    2024

  9. [17]

    P2dflow: A protein ensemble generative model with se(3) flow matching.Journal of Chemical Theory and Computation, 2025

    Yaowei Jin, Qi Huang, Ziyang Song, Mingyue Zheng, Dan Teng, and Qian Shi. P2dflow: A protein ensemble generative model with se(3) flow matching.Journal of Chemical Theory and Computation, 2025

    2025

  10. [18]

    Joshi, Alex Morehead, Arian R

    Rishabh Anand, Chaitanya K. Joshi, Alex Morehead, Arian R. Jamasb, Charles Harris, Simon Mathis, Kieran Didi, Bryan Hooi, and Pietro Liò. Rna-frameflow: Flow matching for de novo 3d rna backbone design.arXiv preprint arXiv:2406.13839, 2024

    2024 arXiv

  11. [19]

    lociparse: a locality-aware invariant point attention model for scoring rna 3d structures.Journal of Chemical Information and Modeling, 64(22):8655–8664, 2024

    Sumit Tarafder and Debswapna Bhattacharya. lociparse: a locality-aware invariant point attention model for scoring rna 3d structures.Journal of Chemical Information and Modeling, 64(22):8655–8664, 2024

    2024

  12. [20]

    Flashattention: Fast and memory- efficient exact attention with io-awareness.Advances in neural information processing systems, 35:16344– 16359, 2022

    Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory- efficient exact attention with io-awareness.Advances in neural information processing systems, 35:16344– 16359, 2022

    2022

  13. [21]

    Flashattention-2: Faster attention with better parallelism and work partitioning.International Conference on Learning Representations, 2024

    Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning.International Conference on Learning Representations, 2024. 9 Flash Invariant Point AttentionA Preprint

    2024

  14. [22]

    Mamba: Linear-time sequence modeling with selective state spaces.arXiv preprint arXiv:2312.00752, 2023

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces.arXiv preprint arXiv:2312.00752, 2023

    2023 arXiv

  15. [23]

    Robust deep learning–based protein sequence design using proteinmpnn.Science, 378(6615):49–56, 2022

    Justas Dauparas, Ivan Anishchenko, Nathaniel Bennett, Hua Bai, Robert J Ragotte, Lukas F Milles, Basile IM Wicky, Alexis Courbet, Rob J de Haas, Neville Bethel, et al. Robust deep learning–based protein sequence design using proteinmpnn.Science, 378(6615):49–56, 2022

    2022

  16. [24]

    Rnasolo: a repository of cleaned pdb-derived rna 3d structures.Bioinformatics, 38(14):3668–3670, 2022

    Bartosz Adamczyk, Maciej Antczak, and Marta Szachniuk. Rnasolo: a repository of cleaned pdb-derived rna 3d structures.Bioinformatics, 38(14):3668–3670, 2022

    2022

  17. [25]

    gRNAde: Geometric deep learning for 3d rna inverse design

    Chaitanya K Joshi, Arian R Jamasb, Ramon Viñas, Charles Harris, Simon V Mathis, Alex Morehead, Rishabh Anand, and Pietro Liò. gRNAde: Geometric deep learning for 3d rna inverse design. In International Conference on Learning Representations (ICLR), 2025

    2025

  18. [26]

    E2efold-3d: End-to-end deep learning method for accurate de novo rna 3d structure prediction.arXiv preprint arXiv:2207.01586, 2022

    Tao Shen, Zhihang Hu, Zhangzhi Peng, Jiayang Chen, Peng Xiong, Liang Hong, Liangzhen Zheng, Yixuan Wang, Irwin King, Sheng Wang, et al. E2efold-3d: End-to-end deep learning method for accurate de novo rna 3d structure prediction.arXiv preprint arXiv:2207.01586, 2022

    2022 arXiv

  19. [27]

    Accurate structure prediction of biomolecular interactions with alphafold 3.Nature, 630(8016):493–500, 2024

    Josh Abramson, Jonas Adler, Jack Dunger, Richard Evans, Tim Green, Alexander Pritzel, Olaf Ron- neberger, Lindsay Willmore, Andrew J Ballard, Joshua Bambrick, et al. Accurate structure prediction of biomolecular interactions with alphafold 3.Nature, 630(8016):493–500, 2024

    2024

  20. [28]

    Chai-1: Decoding the molecular interactions of life.BioRxiv, pages 2024–10, 2024

    Chai Discovery team, Jacques Boitreaud, Jack Dent, Matthew McPartlon, Joshua Meier, Vinicius Reis, Alex Rogozhonikov, and Kevin Wu. Chai-1: Decoding the molecular interactions of life.BioRxiv, pages 2024–10, 2024

    2024

  21. [29]

    Boltz-1: Democratizing biomolecular interaction modeling.bioRxiv, pages 2024–11, 2024

    Jeremy Wohlwend, Gabriele Corso, Saro Passaro, Mateo Reveiz, Ken Leidal, Wojtek Swiderski, Tally Portnoi, Itamar Chinn, Jacob Silterra, Tommi Jaakkola, et al. Boltz-1: Democratizing biomolecular interaction modeling.bioRxiv, pages 2024–11, 2024

    2024

  22. [30]

    Trifast.https://github.com/latkins/trifast, 2025

    Liam Atkinson. Trifast.https://github.com/latkins/trifast, 2025

    2025

  23. [31]

    Generative flows on discrete state-spaces: Enabling multimodal flows with applications to protein co-design.arXiv preprint arXiv:2402.04997, 2024

    Andrew Campbell, Jason Yim, Regina Barzilay, Tom Rainforth, and Tommi Jaakkola. Generative flows on discrete state-spaces: Enabling multimodal flows with applications to protein co-design.arXiv preprint arXiv:2402.04997, 2024

    2024 arXiv

  24. [32]

    Eli5: Flashattention.https://gordicaleksa.medium.com/eli5-flash-attention-5c44017022ad, 2023

    Aleksa Gordic. Eli5: Flashattention.https://gordicaleksa.medium.com/eli5-flash-attention-5c44017022ad, 2023. 10 Flash Invariant Point AttentionA Preprint A Appendix A.1 List of biomolecular design models with IPA structure modules Table 2: A list of models that are based on IP...

    2023

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.