Pith. sign in

REVIEW 3 major objections 5 minor 5 references

Leveraging Large Language Models to Predict Antibody Biological Activity Against Influenza A Hemagglutinin

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A sequence-only language model fine-tuned on around 5,000 assay pairs predicts antibody binding and hemagglutination inhibition against influenza A hemagglutinin, reaching AUROC ≥ 0.91 for known antibodies and 0.90 for unseen HAs.

desk verdict Competent MAMMAL-for-HA application with careful split design, but the 0.9 AUROC for 'unseen HAs' likely leans on memorized antibody breadth; needs an antibody-only ablation. read the letter →

arxiv 2502.00694 v1 pith:I2JQ5JWA submitted 2025-02-02 cs.LG cs.AIq-bio.QM

Ella Barkan , Ibrahim Siddiqui , Kevin J. Cheng , Alex Golts , Yoel Shoshan , Jeffrey K. Weber , Yailin Campos Mota , Michal Ozery-Flato
show 1 more author
Giuseppe A. Sautto
This is my paper · ORCID
classification cs.LGcs.AIq-bio.QM
keywords antibodyactivitypredictioninfluenzaAhemagglutininMAMMALlanguagemodelfine-tuningsequence-basedhemagglutinationinhibitionELISAbindingdatasplits
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a language model fine-tuned on a few thousand antibody–hemagglutinin assay pairs can predict, from amino acid sequence alone, whether an antibody binds influenza A HA and whether it blocks receptor binding. With roughly 5,000 labeled pairs across 188 antibodies and 79 HAs, the fine-tuned model reaches AUROC ≥ 0.91 for held-out pairs of known antibodies, 0.90 for known antibodies tested against unseen HA sequences, and 0.73 for entirely new antibodies, falling to 0.63–0.66 when the new antibody is dissimilar to anything in the training set. If this holds, antibody candidates could be prioritized in silico before wet-lab testing, and existing antibody panels could be screened against emerging viral variants. The paper also reports that breadth-of-protection scores derived from the model correlate with measured breadth, especially for H3 hemagglutination inhibition.

What carries the argument

The load-bearing object is MAMMAL, a transformer-based biomedical language model pretrained on protein, antibody, and interaction data, paired with its AbAg Bind input format: the antibody heavy-chain and light-chain variable-region sequences and the HA antigen sequence are fed as a single prompt, and the model is fine-tuned with a classification head to output a binding or HAI activity score. The evaluation machinery is the four-way split design: lenient random pairs, HA-exclusive folds, antibody-exclusive folds, and antibody-cluster-exclusive folds built by clustering at 50% sequence identity. That split design is what lets the authors separate 'can the model fill in missing measurements' from 'can it extrapolate to new molecules.'

What would settle it

Re-run the same fine-tuning procedure on an independently generated antibody–HA dataset with harmonized assay protocols, and check whether AUROC on an antibody-exclusive split stays near 0.73; alternatively, retrain after shifting the positive cutoffs (for example AUC-ELISA > 5 or HAI MIC < 1 µg/mL) and see whether the ranking of model variants reverses.

Watch

Extended reading notes

Core claim

The central claim is that a sequence-only classifier, built by fine-tuning the pretrained biomedical language model MAMMAL on antibody heavy- and light-chain variable regions paired with full HA sequences, accurately separates active from inactive antibody–HA pairs in the assay-defined sense. The authors report that this transfer-learning approach outperforms random weight initialization across nearly every metric comparison, with the largest gaps for binding activity (AUROC 0.92 vs 0.62 in the lenient split) and for HAI. They interpret the performance gradient across splits—lenient, HA-exclusive, mAb-exclusive, mAb-cluster-exclusive—as evidence that the model generalizes to new viral strains but only moderately to new antibodies. They further use aggregated validation scores to rank antibodies by breadth of protection, with Pearson correlations up to 0.49.

Load-bearing premise

The entire accuracy story rests on the binary labels: binding is called positive when AUC-ELISA exceeds 1, and HAI is called positive when the minimum inhibitory concentration is below 10 µg/mL, and those hand-chosen cutoffs come mostly from the authors' own prior assays.

Editorial extensions

If this is right

  • For panels of already characterized antibodies, the model can impute untested mAb–HA pairs with AUROC ≥ 0.91, reducing the number of wet-lab assays needed to complete strain-by-antibody matrices.
  • Screening existing antibody panels against a newly emerged HA sequence is predicted to work at AUROC ≈ 0.90, giving strain-surveillance efforts a fast sequence-only triage step.
  • For de novo antibody candidates, the model's AUROC of 0.73 (0.63–0.66 for dissimilar antibodies) means it should be used as a coarse filter, not a final arbiter, in antibody discovery.
  • Aggregated model scores rank antibodies by measured breadth of protection with Pearson correlations up to 0.49, so the same model can help select broadly protective leads, not just single-pair positives.
  • Fine-tuning MAMMAL beats random initialization on nearly all metric comparisons, and the gains are largest on the easier splits, confirming that pretrained protein-language representations carry useful signal even with roughly 5,000 labeled pairs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same recipe is likely portable to other viral antigens with small assay panels, such as RSV or SARS-CoV-2; the binding constraint will be label diversity, not model capacity.
  • The monotone drop from lenient to mAb-cluster-exclusive suggests a quantifiable scaling law: expected AUROC for novel antibodies should track the number and diversity of antibody clusters in the training set, so adding one representative antibody per cluster could be tested as a deliberate data-acquisition strategy.
  • Because 21% of the antigens here are computationally optimized COBRA designs, the 0.90 AUROC on unseen HAs should be revalidated against genuinely drifted field strains before relying on it for variant surveillance.
  • A practical screening pipeline could combine this classifier with a diversity filter: candidates within existing clusters can be ranked confidently, while cluster-novel candidates need wet-lab confirmation, mirroring the 0.73 versus 0.63–0.66 gap.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper fine-tunes the MAMMAL biomedical language model on antibody–hemagglutinin (HA) sequence pairs to predict binary binding (ELISA) and hemagglutination inhibition (HAI) activity, using a dataset of 188 mAbs and 79 HAs assembled largely from the authors' prior studies. The evaluation applies 5-fold cross-validation under four split regimes (lenient, HA-exclusive, mAb-exclusive, and mAb-cluster-exclusive) and reports AUROC and AUPRC, comparing MAMMAL-initialized models against randomly initialized models. The headline results are AUROC ≥ 0.91 for lenient splits, 0.90 for HA-exclusive splits, 0.73 for mAb-exclusive splits, and 0.63–0.66 for the mAb-cluster-exclusive split, with parallel trends for binding and HAI tasks. The paper also reports a supplementary analysis of antibody breadth prediction.

Significance. If the reported performance is taken at face value, the approach could be practically useful for prioritizing antibody candidates against known and emerging HA sequences, and the study is one of a small number to evaluate sequence-only antibody–antigen activity prediction under different generalization scenarios. The evaluation design is a real strength: four explicitly defined split types, per-fold label-balance maintenance, both AUROC and AUPRC, and a random-initialization control that supports attributing some gains to pretrained MAMMAL weights. The breadth-prediction analysis is a useful additional check. However, the central claim of high performance on unseen HAs (HA-exclusive split) is not yet established, because the split design allows the model to exploit antibody identity and per-antibody breadth without genuinely using HA sequence information; the paper lacks the antibody-only ablation needed to test this. The hand-chosen assay thresholds also affect every reported number. With additional controls and benchmarks, the manuscript could support its conclusions.

major comments (3)
  1. [Data splits] The HA-exclusive split keeps all pairs with the same HA sequence in the same fold, but the same antibodies appear in both training and test folds across many other HA partners (mean 28±17 assays per mAb). The model sees the full antibody HC/LC sequences at test time and can therefore rank test pairs almost entirely by an antibody-specific base rate of broad versus narrow reactivity, without reading the HA sequence. The observed progression—AUROC 0.90 for HA-exclusive, 0.73 for mAb-exclusive, and 0.63–0.66 for mAb-cluster-exclusive—is exactly the pattern an antibody-only shortcut would produce. The random-initialization control does not rule this out, because the randomly initialized model also receives antibody identity. I request an ablation that removes or masks the HA input (e.g., an antibody-only model) or, at minimum, a simple per-antibody training-positive-rate baseline, to quantify how much of the HA-exclusive AUROC is attributable to HA-specific generalization. Without this, the abstract's claim that the model achieves 'an AUROC of 0.9 for unseen HAs' is not supported as a statement about HA-specific prediction.
  2. [Data description] All binary labels are derived from hand-chosen assay thresholds: AUC-ELISA > 1 for binding and minimum inhibitory concentration < 10 μg/mL for HAI. The paper does not report the distribution of raw assay values near these cutoffs, nor any sensitivity analysis with alternative thresholds. If assay noise or different source-paper protocols systematically straddle the thresholds, the reported AUROC values measure threshold crossing rather than a stable biological activity. Please add a sensitivity analysis (e.g., AUC-ELISA > 2 and HAI < 1 μg/mL) and report the number of pairs within, say, ±1 AUC unit or ±1 dilution of the cutoff. This is load-bearing because every headline number in the abstract and Table 4 is computed against these particular cutoffs.
  3. [Results] The paper provides no comparison against non-LLM baselines or previously published antibody–HA predictors, such as sequence-similarity k-nearest-neighbors, logistic regression on antibody or HA embeddings, or the existing HAI machine-learning models cited in Related Work. Without an external reference, the practical significance of the AUROC differences between MAMMAL fine-tuning and random initialization is unclear, and the added value of the pretrained model over simpler sequence features is not demonstrated. Please add at least one lightweight sequence-based baseline (e.g., k-NN on antibody CDR similarity, or a linear classifier on OAS/UniProt embeddings) across the same four splits.
minor comments (5)
  1. [Results] The text contains a broken cross-reference: 'As shown in Error! Reference source not found.3' should refer to Figure 3.
  2. [Results] In the breadth-prediction paragraph, the sentence 'Table 4 presents antibody statistics for each assay type' appears to refer to Table 3, which is the correct table for antibody breadth statistics; Table 4 reports model evaluation.
  3. [Methods] Methods, model training: 'ensuring input sequences are not truncates' should read 'not truncated'.
  4. [Abstract] The abstract contains a typo: 'eYicient' should be 'efficient'.
  5. [Figure 1] The Figure 1 caption mentions a spider diagram for performance comparison, but no spider diagram appears in the main text or appendix; either add it or remove the mention.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: held-out AUROC evaluations are independent of the fitted model, and self-citations function as external artifacts or empirical data rather than as circular premises.

full rationale

The paper's derivation chain is an empirical machine-learning evaluation: fixed antibody and HA sequences with binary labels (AUC-ELISA > 1 for binding; HAI < 10 ug/mL for inhibition) are used to fine-tune a pretrained model, and performance is measured by 5-fold cross-validation under four distinct data splits. The reported AUROC and AUPRC values are computed on held-out pairs, not on training pairs, so the headline metrics are not fitted constants renamed as predictions. The random-initialization control provides a direct comparison that supports the attribution of improved performance to MAMMAL pretrained weights rather than to the training procedure alone. The MAMMAL framework (ref 50) is a public pretrained model with released code and weights; its pretraining did not use the assay labels in this paper, so citing it is not circular. The antibody and HA assay data (refs 7-18, 65-67) are experimental measurements from prior studies, not outputs of the model being evaluated; using one's own experimental data is a normal and non-circular practice. The breadth-of-protection analysis also remains out-of-sample: it averages validation-fold predictions for antibodies whose pairs were excluded from training in the mAb-exclusive split, then correlates those predictions with observed positive rates, so no label used in the evaluation was seen during training for that antibody. The paper's stated limitations about dataset scarcity and the need for laboratory validation are consistent with an external, falsifiable setup. The possible concern that the HA-exclusive split may leverage antibody-identity memorization is a correctness/robustness issue requiring an antibody-only ablation, not a circularity, because the model does not receive test labels and the prediction is not defined in terms of the evaluation target. No step reduces by construction to its own input, and no load-bearing claim rests on a self-citation as a uniqueness theorem or ansatz.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim depends on four hand-chosen thresholds (ELISA, HAI, cluster identity, breadth cutoff) that define the task and the hardest evaluation, plus domain assumptions about MAMMAL transferability and the sufficiency of sequence-only inputs. No new physical or biological entities are introduced.

free parameters (4)
  • ELISA binding positive threshold = AUC-ELISA > 1
    Hand-chosen cutoff defining the binary binding label. The reported AUROC is relative to this threshold; a different cutoff would change the task and the performance metrics.
  • HAI positive threshold = minimum inhibitory concentration < 10 ug/mL
    Hand-chosen cutoff defining the binary HAI label. HAI values range from 0.005 to 20 ug/mL; the 10 ug/mL cutoff determines the 11% positive rate and strongly shapes the AUPRC.
  • Antibody cluster identity threshold = 50% sequence identity (MMseqs2)
    Chosen for the mAb-cluster-exclusive split. The reported cluster-exclusive AUROCs (0.63-0.66) depend on this clustering stringency.
  • Breadth-positive threshold = 30% positive assays
    Used to dichotomize antibodies into broad or not broad in the breadth-of-protection AUROC analysis. Changing this threshold changes the reported breadth AUROCs.
assumptions (4)
  • domain assumption MAMMAL's pretrained sequence representations encode features relevant to antibody-antigen binding and HAI activity.
    The entire approach depends on ibm/biomed.omics.bl.sm.ma-ted-458m providing useful inductive bias. Support comes from the MAMMAL paper (ref 50) and the reported improvement over random initialization, not from an independent mechanistic justification.
  • domain assumption AUC-ELISA > 1 and HAI < 10 ug/mL are valid, consistent proxies for biological activity across the dataset.
    Labels are defined from these thresholds. Assay protocols vary across the source papers and across antibody hosts and subtypes, which could add batch noise or bias.
  • domain assumption Sequence-only representations (HC/LC variable regions plus full HA) are sufficient for activity prediction.
    The model ignores structure, glycosylation, and assay context. If these are necessary for HAI activity, the mAb-exclusive results could be either optimistic or pessimistic.
  • domain assumption MMseqs2 clustering at 50% identity prevents antibody leakage between train and test.
    The cluster-exclusive split treats 50% identity as sufficient separation. If 50% identity still permits shared epitopes or germline ancestry, the cluster-exclusive AUROC could be inflated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Leveraging Large Language Models to Predict Antibody Biological Activity Against Influenza A Hemagglutinin." pith.science (2026). https://pith.science/paper/I2JQ5JWA

@misc{pith2026250200694,
  author       = {Pith},
  title        = {Pith review of: Leveraging Large Language Models to Predict Antibody Biological Activity Against Influenza A Hemagglutinin},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I2JQ5JWA}},
  note         = {Machine review of arXiv:2502.00694}
}
abstract

Monoclonal antibodies (mAbs) represent one of the most prevalent FDA-approved modalities for treating autoimmune diseases, infectious diseases, and cancers. However, discovery and development of therapeutic antibodies remains a time-consuming and expensive process. Recent advancements in machine learning (ML) and artificial intelligence (AI) have shown significant promise in revolutionizing antibody discovery and optimization. In particular, models that predict antibody biological activity enable in-silico evaluation of binding and functional properties; such models can prioritize antibodies with the highest likelihoods of success in costly and time-intensive laboratory testing procedures. We here explore an AI model for predicting the binding and receptor blocking activity of antibodies against influenza A hemagglutinin (HA) antigens. Our present model is developed with the MAMMAL framework for biologics discovery to predict antibody-antigen interactions using only sequence information. To evaluate the model's performance, we tested it under various data split conditions to mimic real-world scenarios. Our models achieved an AUROC $\geq$ 0.91 for predicting the activity of existing antibodies against seen HAs and an AUROC of 0.9 for unseen HAs. For novel antibody activity prediction, the AUROC was 0.73, which further declined to 0.63-0.66 under stringent constraints on similarity to existing antibodies. These results demonstrate the potential of AI foundation models to transform antibody design by reducing dependence on extensive laboratory testing and enabling more efficient prioritization of antibody candidates. Moreover, our findings emphasize the critical importance of diverse and comprehensive antibody datasets to improve the generalization of prediction models, particularly for novel antibody development.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

5 extracted references · 5 canonical work pages

  1. [5]

    Structural Biology of Influenza Hemagglutinin: An Amaranthine Adventure

    Wu NC, Wilson IA. Structural Biology of Influenza Hemagglutinin: An Amaranthine Adventure. Viruses 2020;12:1053. https://doi.org/10.3390/v12091053. [6] Sun X, Ma H, Wang X, Bao Z, Tang S, Yi C, et al. Broadly neutralizing antibodies to combat influenza virus infection. Antiviral Research 2024;221:105785. https://doi.org/10.1016/j.antiviral.2023.105785. [7] ...

    work page arXiv 2020

  2. [32]

    Large scale paired antibody language models

    Kenlay H, Dreyer FA, Kovaltsuk A, Miketa D, Pires D, Deane CM. Large scale paired antibody language models. PLOS Computational Biology 2024;20:e1012646. https://doi.org/10.1371/journal.pcbi.1012646. [33] Kim J, McFee M, Fang Q, Abdin O, Kim PM. Computational and artificial intelligence-based methods for antibody development. Trends Pharmacol Sci 2023;44:17...

    work page arXiv 2024

  3. [46]

    AbAgIntPre: A deep learning method for predicting antibody-antigen interactions based on sequence information

    Huang Y , Zhang Z, Zhou Y . AbAgIntPre: A deep learning method for predicting antibody-antigen interactions based on sequence information. Front Immunol 2022;13. https://doi.org/10.3389/fimmu.2022.1053617. [47] Zhang J, Du Y , Zhou P , Ding J, Xia S, Wang Q, et al. Predicting unseen antibodies’ neutralizability via adaptive graph neural networks. Nat Mach ...

    work page arXiv 2022

  4. [61]

    tug-of-war

    Yao Y , Li X, Liao B, Huang L, He P , Wang F , et al. Predicting influenza antigenicity from Hemagglutintin sequence data based on a joint random forest method. Sci Rep 2017;7:1545. https://doi.org/10.1038/s41598-017-01699-z. [62] Spatial exclusion leads to “tug-of-war” ecological dynamics between competing species within microchannels | PLOS Computational...

    work page arXiv 2017

  5. [73]

    random-initialization

    Allen JD, Ray S, Ross TM. Split inactivated COBRA vaccine elicits protective antibodies against H1N1 and H3N2 influenza viruses. PLOS ONE 2018;13:e0204284. https://doi.org/10.1371/journal.pone.0204284. [74] Allen JD, Jang H, DiNapoli J, Kleanthous H, Ross TM. Elicitation of Protective Antibodies against 20 Years of Future H3N2 Cocirculating Influenza Virus ...

    work page arXiv 2018

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.