{"id":"78e909c3-1aed-4849-b3a3-b16ab1efde28","arxiv_id":"2507.02724","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A hierarchical contrastive learning model that aligns protein sequences with Pfam family and clan labels improves intra- and cross-species protein-protein interaction prediction.","lead":"HIPPO uses hierarchical contrastive learning, where protein sequences are matched to family and clan labels, to predict protein-protein interactions. It reports improved F1 scores on human datasets and six model species, including zero-shot transfer without retraining.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Cross-species evaluation may leak target labels through the GNN input graph; without the masking protocol, the claimed zero-shot superiority is uninterpretable.","rationale":"The reader's conditional verdict is appropriate, and the reader is right that the cross-species claims lack support, but in my reading the more load-bearing gap is the undefined cross-species graph protocol rather than the Pfam/annotation coverage assumption. The architecture makes graph edges part of the model input, and the paper never specifies whether the target-species STRING graphs used for zero-shot evaluation contain the edges that are being predicted. If they do, HIPPO receives label information through the GIN message-passing step, whereas sequence-only baselines such as ESM-2 and ProteinBERT do not, so the claimed cross-species advantage could be a graph-transduction artifact. This is a correctness risk rather than a stylistic issue: the six-species experiment is the paper's main claim to broad applicability and appears in the abstract without any corresponding Methods description. The concrete test I propose, an edge-masking rerun, cleanly separates the contribution of hierarchical pre-training from graph structure; if the authors release code and splits, the test is cheap. The annotation-coverage concern is real but secondary: even a fully covered Pfam label set would not resolve the graph-label question, and if the graph protocol is flawed, no amount of annotation coverage can salvage the cross-species numbers. I therefore leave the reader's verdict at CONDITIONAL rather than REJECT: the intra-species results, ablations, and binding-site analyses are coherent and the hierarchical-pretraining idea is worthwhile, but the cross-species claim cannot be accepted on the current text.","tokens_in":11479,"tokens_out":7195,"duration_ms":90484,"concrete_test":"Run the reported cross-species benchmark in two configurations: (i) the exact graph used by the authors; (ii) the same graph with every test edge removed from the GIN message-passing input (and, stronger, all target-species STRING edges removed, forcing sequence-only inference). Compare micro-F1 of HIPPO against ESM-2 and ProteinBERT. If configuration (ii) erases HIPPO's advantage or drops it below baselines, the Sec. 2.4 claim depends on test-label information in the graph and must be restated as transductive PPI prediction, not zero-shot transfer. The minimal check is to require the authors' code/splits and inspect whether test edges are present in the GNN input; the rerun settles the magnitude.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.4 claims HIPPO 'consistently outperforms' five baselines on six cross-species datasets and the abstract promises 'zero-shot transferability... without retraining.' The decisive condition is never stated: what graph does the GIN (Sec. 3.2) see during cross-species inference? The GIN updates protein embeddings by aggregating over edges, and in the reported architecture the same graph is used for training and prediction. If each target-species STRING graph is supplied with its known interactions as edges—including the positive pairs being scored—then HIPPO receives label information through its message-passing neighborhood, while sequence-only baselines such as ESM-2, ProteinBERT, and PIPR do not. The comparison would then measure transductive graph exploitation, not zero-shot sequence transfer, and the paper's central cross-species SOTA claim would be an artifact of an unspecified evaluation graph. Section 3.1 lists only human STRING/SHS27k/SHS148k datasets and never defines the construction, edge-masking, or split rule for E. coli, yeast, mouse, worm, Arabidopsis, and fly. This is not a biological annotation-coverage issue; it is a protocol issue that must be resolved before any cross-species number can be interpreted.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HIPPO, a hierarchical contrastive learning framework for protein-protein interaction (PPI) prediction. It combines ProteinBERT-style sequence and annotation encoders with hierarchical clan/family contrastive losses adapted from HiMulConE, a sequence-annotation InfoNCE alignment term, a focal-loss matching objective, and a GIN encoder over the PPI graph. Experiments cover intra-species multi-label PPI prediction on human SHS27k and SHS148k under Random/BFS/DFS splits, zero-shot cross-species prediction on six model organisms, ablations of hierarchical and keyword supervision, and attention-based binding-site analyses. The central claims are state-of-the-art predictive performance and zero-shot transferability to other species without retraining.","tokens_in":11727,"tokens_out":7337,"duration_ms":79446,"significance":"If the empirical claims were established, HIPPO would make a useful contribution: it demonstrates a concrete way to inject Pfam clan/family hierarchy and Swiss-Prot keyword annotations into PPI representation learning, with plausible gains on hard unseen-protein pairs. The manuscript is broad in scope, evaluating five baselines across six species and providing attention-based binding-site validation. Credit is due for presenting the full loss formulation (Eqs. 1-5) and a clear architecture diagram. However, the current evaluation has unresolved protocol questions about graph construction, seeded-run reporting, and dataset statistics, so the significance of the central SOTA claim cannot be assessed until those points are fixed.","major_comments":[{"comment":"The paper never specifies whether test edges are masked when the GIN computes node embeddings. In §3.2 the dataset is defined as G=(P,X) with X_train ∪ X_test = X, and the GIN aggregates over edges in this graph; if the graph used at inference contains the edge being scored, the model receives label information through message passing. This would make the cross-species comparison in §2.4, and the claimed zero-shot advantage over sequence-only baselines, an artifact of transductive graph exploitation rather than sequence transfer. Please state the exact edge set used for message passing in each evaluation, for both intra-species and cross-species settings, and confirm that the same masking protocol was applied to GNN-PPI and all other graph-based baselines.","section":"§3.2 and §2.4"},{"comment":"Section 3.4 states that each experiment is run five times with different random seeds and that the best performance is reported. Best-of-five reporting inflates scores and makes the asterisk significance claims in Fig. 2 impossible to interpret unless the same seeds and selection rule were used for every baseline. Please report mean ± standard deviation over seeds, state how many of the five seeds favored HIPPO over each baseline, and report whether all methods used identical seed sets; the 2.9% average improvement is not interpretable without this information.","section":"§3.4"},{"comment":"The results section concedes that GNN-PPI outperforms HIPPO by 1.7% under the random split on SHS27k (§2.2), while §2.4 asserts that HIPPO consistently outperforms five baselines across all cross-species datasets. Since random splitting is the most widely used protocol in PPI benchmarks, the qualified result should be reflected in the abstract and conclusion, or the cross-species claim should be restricted to the splits and settings where the advantage actually holds.","section":"§2.2 vs. §2.4"},{"comment":"The dataset statistics are inconsistent as written: §3.1 reports that SHS27k comprises 63,408 interactions and SHS148k includes 36,902 interactions, which conflicts with the dataset names and with the original SHS27k/SHS148k definitions used by the baselines. Please verify the interaction counts, correct the text, and state whether the same data files were used for all methods; otherwise the comparison with published numbers is not reproducible.","section":"§3.1"},{"comment":"The pretraining objectives in Eqs. (1)-(4) require hierarchical and keyword annotations for every sequence in the batch. The cross-species experiments do not report the coverage of Pfam families/clans and Swiss-Prot keywords in the six target proteomes, and no ablation varies annotation coverage. For uncharacterized proteins in rare organisms, missing annotations would remove the proposed supervisory signal, and the cross-species advantage may reduce to the underlying sequence encoder alone. Please report coverage statistics and performance broken down by annotation availability, or explicitly describe how missing labels are handled.","section":"§3.3 and §2.4"}],"minor_comments":[{"comment":"The caption says five cross-species datasets but lists six species (E. coli, S. cerevisiae, C. elegans, M. musculus, A. thaliana, D. melanogaster); the text also says six. Please correct the caption.","section":"Fig. 4 caption"},{"comment":"The notation is inconsistent: the text defines annotation embeddings as zT_i, but the first numerator uses zA_i and zA_j. Please unify the symbols for sequence and annotation embeddings throughout Eq. (3).","section":"Eq. (3)"},{"comment":"Panel labels are confusing: panels labeled D appear twice, and the text refers to both Fig. 2D and Fig. 3E for the keyword ablation. Please renumber the panels and update all cross-references.","section":"Fig. 2"},{"comment":"The caption of Fig. 1 says two trainable graph isomorphism blocks, while §3.2 says three recursive GIN blocks. Please make the architecture description consistent.","section":"Fig. 1 and §3.2"},{"comment":"The term L_pair_max(l-1) in Eq. (1) is not defined in the manuscript. Please provide its definition or cite the exact equation from the HiMulConE paper it is taken from.","section":"Eq. (1)"},{"comment":"Reference [11] contains a stray question mark, and references [12] and [25] duplicate the same Contrastive Predictive Coding paper. Please clean up the reference list.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"I found no signs of misconduct, but the current manuscript's evaluation protocol is too underspecified for the strength of the claimed results, particularly the cross-species SOTA claim. The graph-masking question is the most important point to resolve; if the cross-species results survive a corrected protocol with masked edges and proper seeded statistics, the paper could be suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the hierarchical contrastive pretraining idea is sensible and the ablations show it does something: removing hierarchical supervision drops F1 by up to 15%, especially on hard pairs. Second, the cross-species zero-shot claim is currently unverifiable because the paper never says what graph the GIN sees when scoring target-species pairs. If those pairs' known interactions are used as edges, the comparison measures transductive exploitation, not sequence transfer.\n\nThe new piece is the combination of HiMulConE's hierarchical loss with ProteinBERT-style sequence-annotation alignment, applied to PPI prediction. Each ingredient is published, but the specific mix is new, and the binding-site attention analysis is a nice extra. The dataset choice (Pfam clans/families, Swiss-Prot keywords) is reasonable, and the model description is clear.\n\nNow the soft spots. The biggest is the missing cross-species protocol. Section 3.2 defines the graph for training, but Section 2.4 never states how it is built for E. coli, yeast, etc. This is not a minor omission; it controls whether the zero-shot numbers mean anything. The paper must report whether target-graph edges are masked, and ideally compare against sequence-only baselines with the same graph. Second, Sec 3.4 says 'we report the best performance' over five seeds. That is not acceptable for claiming SOTA; give means and standard deviations. Third, the dataset description in Sec 3.1 has SHS27k with 63,408 interactions and SHS148k with 36,902, which is backwards relative to the names. Minor, but it signals sloppy data reporting. Fourth, the cross-species assumption that Pfam/Swiss-Prot annotations are available for uncharacterized proteins is never stress-tested; no analysis of performance vs. annotation coverage. The 1.7% loss to GNN-PPI under random split is fine as long as the SOTA wording is softened.\n\nThe math is straightforward—the losses are standard contrastive variants, no circular fitting. The citation pattern looks fair: HiMulConE and ProteinBERT are credited.\n\nWho should read this? People working on contrastive pretraining for protein interaction prediction, and anyone building on hierarchical label losses. The intra-species results might survive scrutiny after the seed reporting is fixed; the cross-species claim will need a real protocol.\n\nMy recommendation: send to peer review, but with a clear request for the cross-species graph construction and seed-level results. Without those, the cross-species section should not be cited.","headline":"Solid intra-species results with a real hierarchical-supervision finding, but the cross-species zero-shot claim is uninterpretable until the evaluation graph is specified.","tokens_in":12272,"tokens_out":3137,"would_cite":false,"duration_ms":31718,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Hierarchical contrastive learning over Pfam clan-family labels improves protein-protein interaction prediction and transfers to new species.","keywords":["protein-protein interaction prediction","hierarchical contrastive learning","cross-species generalization","Pfam clan-family hierarchy","sequence-annotation alignment","graph neural network","zero-shot transfer","multi-label protein classification"],"falsifier":"A falsifying observation would be cross-species micro-F1 that does not increase with annotation coverage: restrict the test set to proteins without any Pfam clan assignment and compare HIPPO with an annotation-free sequence-only encoder. If the two perform the same on those proteins, the hierarchy is not what drives transfer.","tokens_in":11268,"feed_emoji":"🧬","tokens_out":8446,"duration_ms":84807,"temperature":0.7,"pith_summary":"HIPPO is a pretraining framework that aligns protein sequences with hierarchical biological annotations, then fine-tunes a graph network for protein-protein interaction (PPI) prediction. The paper's central contention is that shaping the embedding space with the Pfam clan-family hierarchy improves PPI prediction, and that this structured supervision is what enables transfer across species. On human benchmarks, HIPPO reports an average micro-F1 gain of 2.9% over the best baseline; on six non-human test species, it reports the highest F1 score in every species. If this holds, interaction screens for rare or poorly characterized organisms could be built from sequence and annotation data rather than expensive species-specific experiments.","feed_headline":"Pfam hierarchy beats sequence-only models for cross-species PPIs","feed_subtitle":"Adding Pfam clan-family supervision lifts micro-F1 for unseen proteins and rare organisms.","key_machinery":"The load-bearing mechanism is a clan-family tree of 621 clans and 6,329 families from Pfam, used as multi-label supervision in a hierarchical contrastive loss (Eq. 1). The loss, adapted from the HiMulConE constraint, enforces that confidence in a higher-level ancestor never falls below confidence in its descendants, pulling proteins with shared evolutionary ancestry together at the correct level of granularity. Around this hierarchy, a ProteinBERT-style sequence encoder and an annotation language model are aligned by a symmetric InfoNCE loss (Eq. 3) and a focal sequence-annotation matching loss (Eq. 4), then a Graph Isomorphism Network propagates PPI network context before a dot-product classifier.","core_discovery":"The central claim is that protein-protein interaction prediction improves when protein embeddings are pretrained against a hierarchy of functional labels, specifically the Pfam clan-family tree, alongside flat sequence-annotation alignment. The paper argues that proteins sharing evolutionary ancestry also share interaction determinants, so multi-label contrastive supervision makes related proteins cluster in embedding space without needing interaction labels for them. Evidence includes a 2.9% average micro-F1 improvement over the best baseline on human benchmarks, the top F1 in all six cross-species benchmarks, and an up-to-15% drop in F1 when hierarchical labels are removed, with the largest loss on pairs whose proteins were both unseen during training. The model also localizes binding sites better with hierarchy, raising attention overlap on one complex from 0.406 to 0.719.","pith_inferences":["Because the supervision comes from Swiss-Prot/Pfam annotations, the method's cross-species advantage should scale with annotation coverage; a testable extension the paper does not report is stratifying test proteins by whether they have a clan or family assignment.","The pretraining objective is not task-specific, so the same hierarchical contrastive scheme could be applied to protein function prediction or binding-site prediction directly, not just to PPI classification.","If the hard-pair gains hold on larger interactomes, hierarchical supervision would become a low-cost complement to experimental interaction assays for prioritizing candidates in under-studied species."],"forward_implications":["A model pretrained on human sequences can score PPIs in other species without retraining, which would make rare-organism interactome mapping cheaper.","The largest gains occur for pairs of completely unseen proteins, so the hierarchy compensates exactly where sequence-only encoders are weakest.","Removing hierarchical supervision costs up to 15% in micro-F1, so hierarchy is a substantive part of the learned representation, not a minor regularizer.","Hierarchically trained attention identifies interaction interfaces more precisely, which could make predicted PPIs more interpretable and easier to test."],"supporting_citations":[{"why":"Supplies the Swiss-Prot protein sequences and curated keyword annotations used in the pretraining stage.","marker":"[18]"},{"why":"Provides the Pfam family and clan definitions that form the hierarchical supervisory labels.","marker":"[19]"},{"why":"Defines the GNN-PPI baseline and the SHS27k/SHS148k datasets with DFS/BFS/random splits.","marker":"[10]"},{"why":"Introduces the Graph Isomorphism Network used to propagate PPI network context before classification.","marker":"[21]"},{"why":"Supplies the hierarchical multi-label contrastive loss (HiMulConE) that the paper adapts for protein hierarchies.","marker":"[22]"},{"why":"Provides the ProteinBERT sequence and annotation encoders adapted for multimodal sequence-annotation alignment.","marker":"[23]"},{"why":"Defines InfoNCE, the symmetric contrastive objective used to align sequence and annotation embeddings.","marker":"[25]"},{"why":"ESM-2 serves as a strong sequence-language-model baseline in intra- and cross-species comparisons.","marker":"[26]"},{"why":"INTREPPID serves as the cross-species-specific baseline that HIPPO must beat in six-organism transfer tests.","marker":"[27]"}],"fun_headline_variants":["Pfam hierarchy boosts cross-species PPI prediction","Hierarchical labels improve zero-shot PPI prediction across species","Pfam clan-family supervision lifts PPI performance in rare organisms","Hierarchy-aware contrastive learning predicts PPIs across species"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that Pfam clan/family and Swiss-Prot keyword annotations are present and predictive for the proteins being scored; if those labels are missing or uninformative, the model loses its supervision signal and the cross-species advantage may disappear.","fun_headline_variants_meta":{"raw":{"variants":["Pfam hierarchy boosts cross-species PPI prediction","Hierarchical labels improve zero-shot PPI prediction across species","Pfam clan-family supervision lifts PPI performance in rare organisms","Hierarchy-aware contrastive learning predicts PPIs across species"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000613,"raw_usage":{"total_tokens":2835,"prompt_tokens":917,"completion_tokens":1918,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":533,"completion_tokens_details":{"reasoning_tokens":1851}},"tokens_in":533,"tokens_out":1918,"duration_ms":17872,"temperature":1.0,"reasoning_tokens":1851,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:22:16.876442+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A falsifying observation would be cross-species micro-F1 that does not increase with annotation coverage: restrict the test set to proteins without any Pfam clan assignment and compare HIPPO with an annotation-free sequence-only encoder. If the two perform the same on those proteins, the hierarchy is not what drives transfer.","supporting_citations":[{"cited_title":"Nucleic acids research28(1), 45–48 (2000)","cited_arxiv_id":null,"evidence_quote":"Supplies the Swiss-Prot protein sequences and curated keyword annotations used in the pretraining stage."},{"cited_title":"Nucleic acids research49(D1), 412–419 (2021)","cited_arxiv_id":null,"evidence_quote":"Provides the Pfam family and clan definitions that form the hierarchical supervisory labels."},{"cited_title":"In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp","cited_arxiv_id":null,"evidence_quote":"Supplies the hierarchical multi-label contrastive loss (HiMulConE) that the paper adapts for protein hierarchies."},{"cited_title":"Bioinformatics38(8), 2102–2110 (2022)","cited_arxiv_id":null,"evidence_quote":"Provides the ProteinBERT sequence and annotation encoders adapted for multimodal sequence-annotation alignment."}],"review_version":1}