Pith. sign in

REVIEW 4 cited by

RiNALMo: General-Purpose RNA Language Models Can Generalize Well on Structure Prediction Tasks

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2403.00043 v2 pith:7GJF775D submitted 2024-02-29 q-bio.BM cs.LG

Rafael Josip Penić , Tin Vlašić , Roland G. Huber , Yue Wan , Mile Šikić This is my paper · ORCID
classification q-bio.BMcs.LG
keywords languagerinalmostructuregeneralizehiddenmodelmodelsprediction
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

While RNA has recently been recognized as an interesting small-molecule drug target, many challenges remain to be addressed before we take full advantage of it. This emphasizes the necessity to improve our understanding of its structures and functions. Over the years, sequencing technologies have produced an enormous amount of unlabeled RNA data, which hides a huge potential. Motivated by the successes of protein language models, we introduce RiboNucleic Acid Language Model (RiNALMo) to unveil the hidden code of RNA. RiNALMo is the largest RNA language model to date, with 650M parameters pre-trained on 36M non-coding RNA sequences from several databases. It can extract hidden knowledge and capture the underlying structure information implicitly embedded within the RNA sequences. RiNALMo achieves state-of-the-art results on several downstream tasks. Notably, we show that its generalization capabilities overcome the inability of other deep learning methods for secondary structure prediction to generalize on unseen RNA families.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. In-Context Learning can distort the relationship between sequence likelihoods and biological fitness

    cs.LG 2025-04 conditional novelty 6.0 of 10

    Repeated motifs collapse pseudo-perplexity to near one in transformer protein language models because the model retrieves the masked residue from the duplicate copy, a behavior that can distort fitness rankings.

  2. NeuralPLexer3: Accurate Biomolecular Complex Structure Prediction with Flow Models

    cs.LG 2024-12 conditional novelty 6.0 of 10

    A flow-based model, NeuralPLexer3, predicts biomolecular complex structures with improved physical validity and speed over AlphaFold3 on the PoseBusters benchmark, and introduces NPBench and ConfBench for broader evaluation.

  3. Multimodal Modeling of CRISPR-Cas12 Activity Using Foundation Models and Chromatin Accessibility Data

    q-bio.GN 2025-06 conditional novelty 5.0 of 10

    RNA-FM embeddings, fed into a lightweight CNN with binarized ATAC-seq accessibility labels, outperform published Cas12 gRNA activity baselines on the Kim et al. HT1 test set.

  4. A Comparative Review of RNA Language Models

    q-bio.BM 2025-05 conditional novelty 5.0 of 10

    A head-to-head zero-shot benchmark of 13 RNA language models finds that strong secondary-structure prediction often comes at the cost of poor functional classification performance.

Pith tools