{"id":"dae94d87-216c-4d99-9bf7-d5b4ee646d19","arxiv_id":"2501.01462","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A teacher-student knowledge distillation framework trained on 11,247 blood transcriptomes reports high AUCs for pan-infection, four pathogens, and sepsis diagnosis.","lead":"This paper builds a large blood gene expression compendium and uses a teacher-student AI approach to predict infections and specific pathogens such as HIV, RSV, and sepsis. A reliable version of this could help doctors choose antibiotics faster and reduce unnecessary prescriptions.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Feature selection uses the full dataset before any train/validation split; all reported AUCs are optimistically biased and the central claim is unsupported.","rationale":"The reader's weakest assumption is precisely the load-bearing flaw: 35 differential gene pairs are selected with Fisher's exact test on the full pan-infection and disease-specific datasets before any train/validation split, so validation samples have influenced feature selection. Every reported metric in the paper (pan-infection AUC 0.97; student AUCs 0.99, 0.94, 0.93, 0.94, 0.99) is downstream of this step. The concern is concrete and testable with a nested or train-only feature-selection protocol. Other issues, such as the inconsistent dataset counts (88 vs. 89 datasets; the listed category sums to 10,753 not 11,247 unless the Table 1 sub-totals are intended to sum to 11,247), and the absence of code or precise data accessions, are secondary but reinforce the need for an independent check. The paper's own limitation statement (small pathogen cohorts) does not address selection bias. Because the main numbers are not trustworthy as reported, rejection is warranted unless strict train-only selection reproduces similar AUCs. I therefore agree with the reader's REJECT verdict and recommend it be kept unchanged until the proposed test is run.","tokens_in":12833,"tokens_out":4644,"duration_ms":46355,"concrete_test":"Run a nested cross-validation for the pan-infection teacher: split the 6,333 pan-infection samples into, say, 5 folds; within each fold, select 35 DGPs using Fisher's exact test on the training fold only, train the transformer on that training fold, and evaluate on the held-out fold. Compute the mean AUC across folds and compare with the reported 0.97. Repeat the same strict protocol for each disease-specific student model: select 35 DGPs per disease using only the training fold of that disease's data, then train the student (with distillation from a teacher trained only on the corresponding pan-infection training folds) and evaluate on the held-out fold. If the mean AUC drops by more than 0.03 relative to any reported value, the reported performance is not a valid estimate of generalization. Also publish the exact GEO accession list so the split can be reproduced.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The manuscript's central evaluation is compromised by feature selection performed on the full dataset before any train/validation split. In Methods ('PAGE and Data processing'), 35 DGP groups are identified using Fisher's exact test (Eq. 1) on all samples of the pan-infection dataset and, separately, on all samples of each disease-specific dataset (Table 2). The same 35 features are then used for both training and validation, so the validation samples have already influenced which gene pairs are used. Consequently, the reported AUCs (0.97 for pan-infection, and 0.99/0.94/0.93/0.94/0.99 for staphylococcus, streptococcus, HIV, RSV, sepsis) are optimistically biased and cannot be interpreted as independent estimates of generalization. The paper also provides neither code nor precise GEO accessions, so the figures are not independently checkable as described. The selection bias is the load-bearing problem: if features were chosen only on training data, the high AUCs may shrink substantially, especially for the smallest cohorts (streptococcal n=306, staphylococcal n=685).","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes TSGPS, a teacher-student knowledge-distillation framework for diagnosing infections from host blood transcriptomes. The authors curate 11,247 samples from GEO, reduce expression profiles to 35 differential gene pairs (DGPs) using their PAGE method, train a pan-infection 'teacher' model (reported AUC 0.97 for both bacterial and viral infection), and distill it into lightweight student models for staphylococcal infection (AUC 0.99), streptococcal infection (0.94), HIV (0.93), RSV (0.94), and sepsis (0.99). The paper also reports parameter compression ratios, comparisons with classical baselines and existing biomarkers, and pathway analyses of the selected gene pairs.","tokens_in":13070,"tokens_out":6517,"duration_ms":61249,"significance":"If the reported numbers held under unbiased evaluation, the paper would provide a valuable lightweight diagnostic pipeline and a large integrated transcriptome resource. The strengths are the scale of the curated compendium, the use of within-sample gene-pair features to mitigate platform effects, the explicit comparison with existing baselines, and the attempt to demonstrate extreme parameter compression. However, the current evaluation does not support the headline claims: feature selection is performed before the train/validation split, no external validation cohorts are used for the pathogen models, and no code or dataset accessions are provided. The central idea is plausible and testable, but the reported AUCs must be re-estimated under a leakage-free protocol before the diagnostic claims can be accepted.","major_comments":[{"comment":"The 35 DGPs are selected by Fisher's exact test on the full pan-infection and disease-specific datasets before any train/validation split; the text says these DGPs 'are designated as the foundation for both training and validation datasets during model development.' This is selection leakage: the validation samples have influenced which gene pairs are used, so all reported AUCs (0.97 for pan-infection and 0.99/0.94/0.93/0.94/0.99 for the pathogen and sepsis students) are optimistically biased and are not estimates of generalization. Please nest feature selection inside each training fold, or hold out independent cohorts before any feature selection, and re-report all metrics.","section":"Methods, PAGE and Data processing"},{"comment":"No external validation cohort is used for any of the four pathogen models; the 'validation' is a random split of the same curated data, and the sepsis comparison with SeptiCyte and sNIP uses only four-fold cross-validation. Given the known batch and platform effects in GEO compendia, leave-one-dataset-out or an independent cohort is needed to support the 'generalizable' claim. The manuscript should also report confidence intervals or bootstrap estimates so that the improvements over the vanilla and classical baselines can be assessed.","section":"Results, Pan-infection foundation model enhances pathogen prediction"},{"comment":"The distillation temperature, loss weights, the number of DGPs, and the network architecture are given as fixed values (T=5, weights 0.2 and 0.8, 35 DGPs, number of attention heads and layers), but the manuscript does not state whether these hyperparameters were selected using the validation set. If any model-selection step used the validation data, the reported performance is additionally optimistic; please describe the hyperparameter-selection protocol explicitly or state that no tuning was performed.","section":"Methods, Training TSGPS"},{"comment":"The data availability statement provides only a generic GEO URL and refers to 'Fig.2b' for the data used in training and testing; exact GEO accessions, sample labels, and code for PAGE, TSGPS, and the baseline models are not provided. Without these, the curation, the DGP lists, and the reported numbers cannot be reproduced or audited. Please deposit the accession-to-sample mapping and the training/feature-selection code in a public repository.","section":"Data availability"}],"minor_comments":[{"comment":"The abstract says 89 blood transcriptome datasets while the Results and Fig.1 say 88; please reconcile the dataset count.","section":"Abstract and Results"},{"comment":"Equation (1) and Table 2 are garbled: the four contingency-table cells a, b, c, and d are not defined, and the displayed formula appears to be a nonstandard version of Fisher's exact test; please define the contingency table and give the standard hypergeometric expression.","section":"Methods, PAGE and Data processing"},{"comment":"The counts in the text (1,505 bacterial, 5,113 viral, 1,809 sepsis, 2,326 healthy) sum to 10,753, not 11,247, and Table 1's pan-infection row (985 healthy, 6,333 infection, 7,318 total) does not reconcile with the stated totals; please clarify the cohort definitions and overlaps.","section":"Results, Establishment of pan-infection data"},{"comment":"The Methods state that the teacher is trained on pan-infection data 'excluding specific conditions such as sepsis, HIV, RSV, etc.', while Table 1's pan-infection virus count of 3,843 may include HIV and RSV; please clarify exactly which samples are included in the teacher's training set.","section":"Methods, Training TSGPS"},{"comment":"The terms 'original model', 'vanilla model', and 'student model' are used inconsistently (e.g., Fig.5d and the surrounding text); please define each term once and use it consistently throughout.","section":"Results, Pan-infection foundation model enhances pathogen prediction"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's heavy reliance on the authors' own PAGE method and bvnGPS baseline is not by itself disqualifying, but it makes independent verification more important; the absence of code and accession lists is a serious concern for a cs.LG submission. If the authors re-run the analysis with leakage-free feature selection and provide external validation, the work could become suitable. I recommend major revision rather than reject because the principal methodological flaw is correctable in principle, but the revision must be substantial: every reported performance number depends on the current feature-selection protocol."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's my read of 2501.01462. The core idea is reasonable: train a pan-infection teacher on a large host-transcriptome compendium, then distill into lightweight pathogen-specific students. The dataset curation is a real step forward—11k samples across many platforms is not trivial—and the parameter compression numbers are honest. The KD improvement over vanilla students is also plausible.\n\nBut the evaluation has a load-bearing problem. The 35 gene pairs are selected using Fisher's exact test on the full dataset before any split (Methods, PAGE and Data processing, Eq. 1 and Table 2). Validation samples therefore influence which features are used. That inflates AUCs, potentially substantially, especially for the smaller cohorts (streptococcal n=306, staph n=685). The reported AUCs of 0.97 and 0.99–0.93 are not independent estimates of generalization. There are also no confidence intervals, no external cohorts for the four pathogen models, and the sepsis comparison uses only four-fold CV. The paper gives no code or exact GEO accessions, so I can't check the pipeline. There are inconsistent counts too (abstract says 11,247 samples and 89 datasets; results say 88 datasets and numbers that sum to 10,753).\n\nWhat's good: the teacher-student framing for infection diagnosis is a sensible application of existing techniques, and the paper compares against vanilla models and classic ML—though those comparisons inherit the same feature-selection bias. Heavy self-citation is not a flaw by itself, since PAGE is their own published method.\n\nThe fix is straightforward: redo feature selection inside the training folds, use nested CV or a proper external validation cohort, and report CIs. If the AUCs survive that, this could be a useful applied contribution. As it stands, the central numbers are unsupported. I would not cite it as-is. For peer review, I'd send it out with a request for major revision, because the question is important and the flaw is correctable. If the authors return with train-only feature selection and external validation, it's worth a second look.\n\nBottom line: promising idea, compromised execution. The stress-test note is accurate.","headline":"Useful framework, but the evaluation leaks labels through feature selection; the AUCs as reported should not be trusted.","tokens_in":13578,"tokens_out":3687,"would_cite":false,"duration_ms":33385,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a teacher-student framework built on 35 differential gene pairs can diagnose bacterial and viral infections, specific pathogens, and sepsis from host blood transcriptomes with AUCs up to 0.99.","keywords":["pan-infection","knowledge distillation","host transcriptome","gene pair signature","blood gene expression","pathogen prediction","sepsis diagnosis","teacher-student model"],"falsifier":"Rerun the entire TSGPS pipeline with the DGP selection nested inside cross-validation folds (or on training subsets only) and compare the resulting AUCs; if the gap exceeds a few points, the published numbers are inflated by feature-selection leakage and would not generalize to new cohorts.","tokens_in":12654,"feed_emoji":"🦠","tokens_out":2274,"duration_ms":24282,"temperature":0.7,"pith_summary":"The authors aim to show that a single pan-infection 'teacher' model, trained on the largest assembled collection of host blood transcriptomes, can transfer knowledge to lightweight 'student' models that diagnose specific pathogens and sepsis. They report that this approach reaches an AUC of 0.97 for distinguishing bacterial from viral infection and AUCs of 0.99, 0.94, 0.93, 0.94, and 0.99 for staphylococcal infection, streptococcal infection, HIV, RSV, and sepsis, respectively. If these numbers hold outside the reported validation sets, the work would offer a deployable, low-parameter diagnostic that could guide antibiotic use and early sepsis care. The authors further claim that the gene-pair signatures across diseases share biological pathways, which they offer as a mechanistic reason for why knowledge transfer works.","feed_headline":"One host-blood model diagnoses six infections, with AUC up to 0.99","feed_subtitle":"A single pan-infection teacher plus distilled lightweight students distinguishes bacteria, viruses, and sepsis from 11,247 transcriptomes.","key_machinery":"The load-bearing mechanism is PAGE (a pathway-based gene-pair scoring method) combined with Fisher's exact test to select 35 differential gene pairs (DGPs) that represent each disease's host response. These DGPs, computed within biological pathways, transform raw expression values into pairwise comparisons that are more comparable across platforms. A transformer-based teacher model is trained on the pan-infection DGPs, and each student model is trained with a weighted loss combining cross-entropy with a distillation loss that matches the teacher's softened predictions, transferring shared immune-response features.","core_discovery":"The central discovery is a framework called Teacher-Student Gene Pair Signature (TSGPS) that couples a pan-infection foundation model with distilled pathogen-specific student models. Using 11,247 blood transcriptome samples from 89 datasets, the teacher model learns a general host-response representation that separates health, bacterial infection, and viral infection with an AUC of 0.97. Distilling this teacher into smaller student models yields AUCs of 0.99 for staphylococcal infection, 0.94 for streptococcal infection, 0.93 for HIV, 0.94 for RSV, and 0.99 for sepsis, consistently outperforming vanilla versions of the same student architectures and traditional classifiers. The paper argues that this demonstrates both cross-infection and cross-disease knowledge transfer, and that the compressed students (down to 95.6% parameter reduction) remain accurate enough for clinical deployment.","pith_inferences":["A natural extension would be to test whether the same teacher-student pipeline transfers to other pathogens with small cohorts, such as fungal or parasitic infections, where the pathway-sharing argument would predict measurable gains.","The reported AUCs likely overstate real-world performance because the 35 DGPs were selected using the full dataset before splitting; an honest estimate would require selecting features inside cross-validation folds.","If the pathway-based knowledge transfer is real, one could deliberately choose student diseases that share more pathways with the teacher and expect larger distillation gains, a prediction testable on the same data.","The sepsis results suggest that this framework might be adapted for other infection-triggered syndromes, such as acute respiratory distress syndrome, though the paper does not explore that direction."],"forward_implications":["If the reported AUCs generalize, a single pan-infection model could support multiple downstream diagnostic tasks without retraining a large model for each pathogen.","The 95.6% parameter compression of the extreme student model suggests that host-response diagnostics could run on resource-limited clinical devices.","The shared pathways between pan-infection and sepsis models imply that knowledge gained from broad infection data can improve disease-specific screening such as sepsis.","The 35-gene-pair signature for each disease is compact enough to be measured with targeted assays, potentially enabling rapid molecular tests.","The consistent improvement of distilled students over vanilla models indicates that knowledge distillation is a viable strategy when pathogen-specific training data are scarce."],"supporting_citations":[{"why":"PAGE method for gene-pair data integration and sepsis signature; supplies the core feature-engineering approach that produces the 35 DGPs.","marker":"[7]"},{"why":"bvnGPS is a pretrained neural-network diagnostic for bacterial and viral infection that serves as a baseline the pan-infection teacher must outperform.","marker":"[14]"},{"why":"Hinton's knowledge-distillation framework is the conceptual basis for the teacher-student transfer used throughout TSGPS.","marker":"[24]"},{"why":"SeptiCyte is a molecular host-response assay for sepsis used as a state-of-the-art comparison for the sepsis student model.","marker":"[16]"},{"why":"sNIP is another sepsis molecular biomarker baseline compared against the sepsis student model.","marker":"[31]"},{"why":"The transformer architecture (attention mechanism) is the foundation of both teacher and transformer-based student models.","marker":"[28]"},{"why":"Prior deep-learning work on discriminating infection types via pairwise host gene expression, which the study extends to multiple pathogens.","marker":"[13]"}],"fun_headline_variants":["Teacher-student model pairs host response to six infections","One blood transcriptome model, six pathogen diagnoses","Pan-infection teacher distills to sepsis and four pathogen models","Largest host-response dataset powers six-infection diagnostic"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The 35 differential gene pairs are picked using the entire dataset, including the validation samples, before any train/test split, so the reported accuracy depends on these features being just as informative when chosen only from training data.","fun_headline_variants_meta":{"raw":{"variants":["Teacher-student model pairs host response to six infections","One blood transcriptome model, six pathogen diagnoses","Pan-infection teacher distills to sepsis and four pathogen models","Largest host-response dataset powers six-infection diagnostic"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000574,"raw_usage":{"total_tokens":2725,"prompt_tokens":971,"completion_tokens":1754,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":587,"completion_tokens_details":{"reasoning_tokens":1690}},"tokens_in":587,"tokens_out":1754,"duration_ms":13765,"temperature":1.0,"reasoning_tokens":1690,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:49:48.007430+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the entire TSGPS pipeline with the DGP selection nested inside cross-validation folds (or on training subsets only) and compare the resulting AUCs; if the gap exceeds a few points, the published numbers are inflated by feature-selection leakage and would not generalize to new cohorts.","supporting_citations":[{"cited_title":"S., Wong, M","cited_arxiv_id":null,"evidence_quote":"PAGE method for gene-pair data integration and sepsis signature; supplies the core feature-engineering approach that produces the 35 DGPs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"bvnGPS is a pretrained neural-network diagnostic for bacterial and viral infection that serves as a baseline the pan-infection teacher must outperform."},{"cited_title":"T., Rapisarda, A., Sutherland, A","cited_arxiv_id":null,"evidence_quote":"SeptiCyte is a molecular host-response assay for sepsis used as a state-of-the-art comparison for the sepsis student model."},{"cited_title":"P., Wiewel, M","cited_arxiv_id":null,"evidence_quote":"sNIP is another sepsis molecular biomarker baseline compared against the sepsis student model."},{"cited_title":"& Geng, Q","cited_arxiv_id":null,"evidence_quote":"Prior deep-learning work on discriminating infection types via pairwise host gene expression, which the study extends to multiple pathogens."}],"review_version":1}