Predicting ICU In-Hospital Mortality Using Adaptive Transformer Layer Fusion

Changqi Qin; Chia Xin Liang; Chunjie Tian; Guoguang Lao; Han Wang; Hejiao Luo; Hongying Luo; Huafeng Liu; Junfeng Hao; Junhao Song

REVIEW 5 major objections 5 minor 13 references

Predicting ICU In-Hospital Mortality Using Adaptive Transformer Layer Fusion

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read ALFIA's adaptive layer fusion beats strong tabular classifiers on ICU mortality AUPRC, and its embeddings add 2–3 more points when fed to gradient-boosted trees.

desk verdict A careful empirical study with a reproducible benchmark and a small, plausible AUPRC gain over tabular baselines, but the gain is over-attributed to adaptive layer fusion because no same-backbone text-only control is run. read the letter →

arxiv 2506.04924 v2 pith:OWIGVM2F submitted 2025-06-05 cs.LG

Han Wang , Ruoyun He , Guoguang Lao , Ting Liu , Hejiao Luo , Changqi Qin , Hongying Luo , Junmin Huang

show 11 more authors

Zihan Wei Lu Chen Yongzhi Xu Ziqian Bi Junhao Song Tianyang Wang Chia Xin Liang Xinyuan Song Huafeng Liu Junfeng Hao Chunjie Tian

This is my paper

classification cs.LG

keywords ICUmortalitypredictionadaptivelayerfusionLoRABERTMIMIC-IVeAUPRCclinicaltextencoding

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a text-based transformer can beat strong tabular models at predicting ICU in-hospital mortality within the first 24 hours after admission. It introduces ALFIA, which freezes a BERT-style model, attaches LoRA adapters, and learns a small set of weights that fuse the top transformer layers into one embedding before an attention-based classifier. On the authors' cw-24 benchmark built from MIMIC-IV and eICU, ALFIA reports higher AUPRC than an AutoGluon ensemble and other tabular baselines (0.585 vs 0.577 on MIMIC-IV), and feeding ALFIA's embedding to GBDTs as ALFIA-boost adds 2–3 AUPRC points. If the pattern holds, the practical payoff is an early-warning risk score that works from routine clinical text without extra feature engineering.

What carries the argument

The load-bearing component is the Adaptive Layer Fusion (ALF) module. For each of the top $N_f$ transformer layers it forms an attention-masked average-pooled summary, computes a cross-layer attention score against a global query, and optionally passes the result through a sigmoid gate to obtain layer weights $\lambda_j$; the full hidden-state sequences are then combined as $\mathbf{h}^{\mathrm{fused}}_t = \sum_{j=1}^{N_f} \lambda_j \mathbf{h}^{(L-N_f+j)}_t$. After a post-fusion block of LayerNorm, residual, and GELU layers, local token-level attention and global masked-mean pooling are concatenated and fused into $H^{\mathrm{ALF}}_{\mathrm{out}}$, which the Attentional Classifier Head (multi-head self-attention plus a feed-forward network) turns into logits. LoRA adapters keep the BERT backbone trainable at low rank. These learned weights are what the paper credits for separating survivor and non-survivor clusters in UMAP and for the AUPRC gains.

What would settle it

Retrain ALFIA on the cw-24 MIMIC-IV split with a single layer (no fusion) and with uniform layer weights, and separately score a TF-IDF/linear baseline on the same templated text. If AUPRC stays inside ALFIA's reported 0.585 (95% CI 0.552–0.617) under either change, the adaptive fusion is not the source of the claimed improvement.

Watch

Extended reading notes

Core claim

ALFIA's central claim is that adaptive fusion of hidden layers is what lets a pre-trained language model outperform tabular classifiers on imbalanced ICU mortality data. Operationally, the paper encodes each 24-hour patient record as a sentence, runs a BioLinkBERT-large backbone with LoRA adapters, and learns layer weights over the top four hidden states through cross-layer attention and a sigmoid gate; the weighted token sequence is pooled into local and global contexts and passed through an attention head that predicts hospital mortality. On the MIMIC-IV cw-24 test split this reaches AUPRC 0.585 (95% CI 0.552–0.617) versus 0.577 for the AutoGluon ensemble and 0.566 for FT-Transformer, at AUROC 0.894; on external eICU the MIMIC-trained model leads the GBDT and FT-Transformer baselines by about 1.5 AUPRC points. The same fused embedding, combined with original features and handed to CatBoost, LightGBM, XGBoost, or a neural net (ALFIA-boost and ALFIA-nn), raises AUPRC by another 2–3 points, which the authors read as evidence that the fusion module produces a task-optimized latent representation rather than a mere classifier improvement.

Load-bearing premise

The load-bearing premise is that turning the 133-field tabular record into the fixed sentence template neither hides nor adds information, so ALFIA's AUPRC edge comes from what the fusion module learns and not from wording, field order, or other surface regularities of the template.

Editorial extensions

If this is right

On the MIMIC-IV cw-24 test split, ALFIA reports AUPRC 0.585, ahead of the AutoGluon ensemble (0.577) and FT-Transformer (0.566), which is the paper's headline comparison.
The ALFIA-boost and ALFIA-nn variants, which feed ALF embeddings plus original features to CatBoost, LightGBM, XGBoost, or a neural net, claim an additional 2–3 AUPRC points over the tabular baselines.
A model trained on MIMIC-IV and evaluated on eICU keeps an AUPRC lead of about 1.5 points over GBDTs and FT-Transformer, suggesting the learned fusion transfers across institutions.
Inference on an RTX 4090 runs at 8–9 ms per sample with 1–3 GB memory, which the authors argue is practical for clinical early-warning systems.
On 100-case cross-dataset samples, ALFIA's AUPRC and AUROC exceed both participating ICU specialists' scores, supporting the claim that the tool can aid expert risk stratification.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct test the paper leaves open: run a linear or TF-IDF model on the same templated sentences and permute the template's field order. If AUPRC stays near 0.585, the edge is in the text encoding, not in layer fusion.
The 24-hour window is the only one tested; the natural extension is shorter windows (6h, 12h). If fusion is most valuable when signal is sparse, the AUPRC gap over tabular baselines should widen as the window shrinks.
Because cw-24's text is generated from structured fields, the claim of operating on 'routine clinical text' has not been tested on genuine free-text notes; feeding ALFIA real MIMIC-IV narratives would decide that.
The learned per-layer weights $\lambda_j$ could be reported as patient-level explanations; if they track severity, they could double as an interpretability map or an alert-threshold modulator. The paper shows the weight heatmap but does not explore this use.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Punchline: this is a reasonably careful empirical study with a reproducible benchmark and a small but plausibly real AUPRC edge over strong tabular baselines on two large ICU datasets — but the authors over-attribute that edge to their adaptive layer fusion module, and the headline claim sits on overlapping confidence intervals with no significance test.

What's new and good: the cw-24 benchmark is a clean, well-specified 24-hour ICU mortality task with a fixed 75/12.5/12.5 split and strict temporal constraints; code, benchmark, and data links are provided. ALFIA's point-estimate AUPRC (0.585 vs AutoGluon ensemble's 0.577 on MIMIC-IV) is credible, and the external eICU validation is a genuinely valuable check. The ablation in Figure 6b,c — ALF embeddings beating CLS/max/mean pooling when fed to nnFastAI — is the right kind of evidence and is one of the paper's better moments. The ALFIA-boost variants showing 2–3 point gains are also worth a look.

Where it's soft: the most load-bearing problem is experimental attribution. ALFIA is BioLinkBERT + LoRA + ALF + ACH on templated text, but the baselines are AutoGluon tabular models on the raw features. There is no row for plain BERT/LoRA with standard pooling on the same templated text. So the gap could come from the text encoder itself, not from adaptive layer fusion. Figure 6b,c partially addresses this, but only in the nnFastAI setting and without reporting numbers, so the main table still conflates modality with architecture. Second, the abstract calls the input 'routine clinical text,' but it's a generated template from structured variables — the authors know this, Section 2.2 describes it plainly, and the overclaim should be fixed. Third, the expert comparison is too thin to carry the 'surpassed clinical specialists' conclusion: two doctors, 100 samples, no protocol description. Fourth, Figure 1A contradicts Table 1 on MIMIC-IV deaths (13,838 vs 7,086), which erodes confidence in the data pipeline. The CI overlap on the headline comparison is a minor issue in itself but matters because the paper leans on 'significantly higher' with no test.

Overall: the work is a decent contribution to the clinical-text-mortality niche, worth a serious referee, but it needs a significance test, a same-text BERT/LoRA control, and a corrected figure before the architecture claim is supportable.

Referee Report

5 major / 5 minor

Summary. The paper introduces ALFIA, an architecture that combines a LoRA-adapted BERT backbone with an adaptive layer-fusion (ALF) module and an attentional classifier head (ACH), and evaluates it on a new benchmark, cw-24, built from MIMIC-IV and eICU. The input to ALFIA is a templated text rendering of a 133-dimensional tabular feature matrix. The central claim is that ALFIA outperforms tabular classifiers such as AutoGluon's ensemble in AUPRC (0.585 vs 0.577 on MIMIC-IV) and that embedding-plus-GBDT variants (ALFIA-boost) yield further gains of 2 to 3 AUPRC points. External validation on eICU and a comparison with ICU specialists are also reported.

Significance. If the central claim is sustained, the adaptive layer-fusion module would be a reproducible architectural contribution for clinical mortality prediction from structured ICU data. The paper has genuine strengths: a public code and data repository, a fixed random seed for reproducibility, a genuinely held-out test split, external validation on eICU, and a broad set of baselines including an AutoGluon ensemble. The latent-space analysis (Section 3.4) is a valuable attempt to explain why the fused embeddings separate mortality from survival. However, as written, the paper does not establish that the adaptive layer-fusion module, rather than the transformer backbone or the text template, is responsible for the reported improvement, and several load-bearing claims are unsupported by the evidence presented.

major comments (5)

[Section 3.2, Table 3] The experimental design cannot attribute the reported AUPRC advantage to the ALF module. ALFIA consists of BioLinkBERT + LoRA + ALF + ACH consuming templated text, while the strongest baselines (AutoGluon ensemble, LightGBM, FT-Transformer) consume tabular features. There is no row in Table 3 for the same BioLinkBERT + LoRA backbone with standard CLS/mean pooling and a plain classification head on the identical templated text. Figure 6b,c compares ALF embeddings against CLS/max/mean pooling only through nnFastAI, not in the end-to-end training setup of Table 3, and no numeric results are reported in the text. Without this control, the headline gain (0.585 vs 0.577) may reflect the text modality or the biomedical backbone rather than adaptive layer fusion. Please add this ablation and report numeric results.
[Abstract and Section 2.2] The abstract's claim that ALFIA operates "directly on routine clinical text" is unsupported because the input is a generated template derived from 133 structured features (Section 2.2, Figure 1B), not actual clinical notes. This overstatement recurs in the Introduction and Discussion. Additionally, Section 2.4's statement that "All compared models use tabular data (encoded as text) as input" is contradicted by Section 3.4's description of the "original feature engineering matrix ... 133 dimensions" used by AutoGluon; please clarify exactly which baselines saw the text and which saw the raw tabular matrix, since this determines whether the comparison isolates the text encoder.
[Section 3.2, Table 3, Discussion Section 4] The claimed consistent superiority of ALFIA is not statistically supported. The AUPRC advantage over the AutoGluon ensemble (0.585, CI 0.552-0.617, vs 0.577, CI 0.540-0.609) is within overlapping confidence intervals, and no significance test (e.g., bootstrap or a paired test for AUPRC) is reported. Moreover, ALFIA is numerically below AutoGluon on AUROC (0.894 vs 0.899), F1 (0.552 vs 0.554), and F2 (0.635 vs 0.653) in the same table. The Discussion's "significantly higher" regarding the AUPRC advantage is therefore not justified. Report paired tests and either demonstrate significance or soften the claim.
[Section 3.3, Figure 5] The external eICU validation is presented through figures without a numeric table, and the two eICU analyses appear inconsistent: Figure 5a shows external eICU AUPRC values in the 0.28-0.36 range, while Figure 4's eICU panels show AUPRC values around 0.45-0.60 for the same method classes. The authors should clarify which protocol each figure reports (the mapping/imputation protocol described in Section 3.3 versus the direct eICU test set), and provide point estimates with confidence intervals for the external validation claim of "gains of about 1.5 percentage points."
[Section 3.3, Figures 5e,f] The expert comparison is under-specified: it is not reported what information the two ICU specialists were shown (the generated template, the raw tabular features, or full clinical charts), how they were asked to produce a mortality probability, and how their confidence intervals were computed. As described, the claim that "our model surpassed clinical specialists in previously unencountered cases" is not verifiable and should either be fully specified or removed.

minor comments (5)

[Table 3 caption] The Table 3 caption does not state that results are for the MIMIC-IV cw-24 test split; eICU results appear only in figures, so please add a numeric table for eICU or state explicitly which dataset Table 3 covers.
[Figure 3a] Figure 3a refers to simultaneous optimization of "LoRA, FLA, and ACH modules," but the acronym FLA is never defined; presumably it means the Adaptive Layer Fusion (ALF) module, so please correct the label.
[Figure 1a] The patient-selection flow diagram in Figure 1a contains internally inconsistent numbers (for example, the text reports 157,883 eICU admissions with 8.77% mortality, while the diagram shows different totals and percentages such as 156,466 and 91.2%); please redraw the diagram with consistent cohort counts.
[Section 3.4] It is unclear whether the CLS/max/mean pooling embeddings used in Figure 6b,c are computed from the LoRA-finetuned BioLinkBERT or from the frozen pretrained model; the comparison is only meaningful if all embeddings come from the same fine-tuned backbone, so please state this explicitly.
[Throughout] The paper uses "PRAUC" in the Table 3 caption and "AUPRC" elsewhere; please unify the terminology.

Circularity Check

1 steps flagged · score 4.0 of 10

ALFIA's main held-out result is independent, but the ALFIA-boost/nn 'confirmation' experiment is described as training on test-set embeddings, making its reported gains fitted rather than predicted.

fitted input called prediction [Section 3.2, paragraph following Table 3]
"To confirm our technique, we ran additional experiments that combined the embeddings generated by the ALF module from training, validation, and test sets with original features and fed them into mainstream machine learning algorithms for further training. This effectively replaces the previous attention-based classification head. This practice resulted in significant performance increases, with gains of around 2-3 percentage points over baseline procedures."

The sentence states that embeddings 'from training, validation, and test sets' were fed into algorithms 'for further training'; therefore the derivative ALFIA-boost/nn models are fitted with evaluation-set data. The reported 2-3 percentage point improvement is then not a held-out prediction but a score computed on data that participated in fitting, so the gain is forced by construction. If the protocol was intended to be train/validation-only fitting, the published text does not say so, and as written it describes the opposite.

full rationale

The centrally advertised ALFIA result (Table 3: AUPRC 0.585 vs AutoGluon 0.577 on MIMIC-IV, with external eICU evaluation) rests on a held-out test split and an independently hosted external dataset, so that claim has independent content and is not circular. The ALF module is learned from the training data and is not defined in terms of the target outcome. The one step that reduces by construction is the ALFIA-boost/nn confirmation: its text places test-set embeddings into further training, so the reported gains are fitted evaluation-set scores rather than predictions. The absence of a same-backbone BERT/LoRA-without-ALF ablation weakens attribution of the headline AUPRC gain to the fusion module, but that is an experimental-control gap rather than circularity. No load-bearing self-citation or imported uniqueness theorem appears in the derivation chain.

Assumptions & free parameters 8 free parameters · 7 assumptions · 2 invented entities

The central claim depends on the template text encoding being information-preserving, on the clinical scores' validity within a 24-hour window, and on standard transformer machinery. No physical entities are postulated; the introduced artifacts are the ALF module and CW-24 benchmark. Main free parameters are hand-set hyperparameters, several of which (learning rate, batch size, pos_weight) are unreported.

free parameters (8)

Number of fused layers (N_fuse) = 4
Top 4 transformer layers are fused; chosen by hand. Central to the ALF module's capacity (Section 2.3.2, Section 3.1).
LoRA rank r = 16
Low-rank adapter dimension; standard choice, set by hand (Section 3.1).
LoRA alpha = 16
LoRA scaling factor; set by hand (Section 3.1).
LoRA dropout = 0.05
Regularization for adapters; set by hand (Section 3.1).
ACH attention heads = 4
Multi-head self-attention heads in classifier head (Figure 2); set by hand.
BCEWithLogitsLoss pos_weight = not reported
Class-imbalance weight shown in Figure 2 but its value is never given; needed to reproduce training.
Learning rate, batch size, optimizer, warmup, max epochs = not reported
Core training hyperparameters are absent from the manuscript; reproducibility gap (Section 2.4, Section 3.1).
Early stopping patience = 5
Patience in epochs for validation AUPRC early stopping (Section 2.4).

assumptions (7)

standard math Transformer hidden states H^(l) and attention operations behave as specified in Vaswani et al. (2017) and Devlin et al. (2019).
The entire ALF/ACH pipeline is built on standard transformer forward passes (Section 2.3.1).
domain assumption Templated text generated from tabular fields preserves all predictive information and adds no systematic artifact that baselines lack.
The comparison isolates architecture performance only if the text encoding is information-preserving and artifact-neutral (Section 2.2, Figure 1B).
domain assumption AUPRC is the appropriate primary metric for this imbalanced mortality task.
Justified by class imbalance (Section 3.2); defensible but selected after observing it is the metric where ALFIA leads.
domain assumption Severity scores (APACHE III/IV, SAPS II, SOFA, GCS, LODS, OASIS) recorded in the first 24 hours are valid predictors of hospital mortality.
The CW-24 benchmark and all models rely on these scores as inputs (Section 2.1).
domain assumption Mode-imputed intersection features between eICU and MIMIC-IV remain clinically valid inputs for cross-dataset evaluation.
External validation maps MIMIC features onto eICU with mode imputation for missing fields (Section 3.3).
domain assumption Pre-trained biomedical LMs (e.g., BioLinkBERT) transfer to templated clinical descriptions.
The best backbone is selected on validation AUPRC; transfer is assumed across templates and institutions (Section 3.1).
domain assumption No future information beyond the 24-hour cutoff entered the predictive variables.
The benchmark asserts 'automated validation checks' enforce temporal integrity; no code or certificate is provided (Section 2.1).

invented entities (2)

ALF (Adaptive Layer Fusion) module
purpose: Learns attention-based layer weights lambda_j to combine the top 4 transformer layer hidden states before classification (Section 2.3.2).
Performance evidence is internal to this paper's benchmark; the released code would permit external testing, but the manuscript provides no independent falsifiable handle beyond its own experiments.
CW-24 (CriticalWindow-24) benchmark independent evidence
purpose: Standardized 24-hour ICU mortality benchmark built from MIMIC-IV and eICU (Section 2.1).
Publicly released on GitHub and Zenodo, allowing external groups to reproduce or refute the benchmark's construction.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Predicting ICU In-Hospital Mortality Using Adaptive Transformer Layer Fusion." pith.science (2026). https://pith.science/paper/OWIGVM2F

@misc{pith2026250604924,
  author       = {Pith},
  title        = {Pith review of: Predicting ICU In-Hospital Mortality Using Adaptive Transformer Layer Fusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OWIGVM2F}},
  note         = {Machine review of arXiv:2506.04924}
}

read the original abstract

Early identification of high-risk ICU patients is crucial for directing limited medical resources. We introduce ALFIA (Adaptive Layer Fusion with Intelligent Attention), a modular, attention-based architecture that jointly trains LoRA (Low-Rank Adaptation) adapters and an adaptive layer-weighting mechanism to fuse multi-layer semantic features from a BERT backbone. Trained on our rigorous cw-24 (CriticalWindow-24) benchmark, ALFIA surpasses state-of-the-art tabular classifiers in AUPRC while preserving a balanced precision-recall profile. The embeddings produced by ALFIA's fusion module, capturing both fine-grained clinical cues and high-level concepts, enable seamless pairing with GBDTs (CatBoost/LightGBM) as ALFIA-boost, and deep neuro networks as ALFIA-nn, yielding additional performance gains. Our experiments confirm ALFIA's superior early-warning performance, by operating directly on routine clinical text, it furnishes clinicians with a convenient yet robust tool for risk stratification and timely intervention in critical-care settings.

Figures

Figures reproduced from arXiv: 2506.04924 by the authors.

**Figure 3.** Training pipeline of ALFIA and performance comparison of different pre-trained [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗

**Figure 4.** Performance comparison of machine learning models on MIMIC-IV and eICU [PITH_FULL_IMAGE:figures/full_fig_p016_4.png] view at source ↗

Figures from the paper (2 more)

**Figure 5.** Figure 5: Performance evaluation of ALFIA on external eICU dataset. (a) [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]

**Figure 6.** Figure 6: ALFIA optimizes latent space distribution of samples. (a) [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

13 extracted references · 6 canonical work pages

[1]

10, 2016]), https://doi.org/10.1007/978- 3-319-43742-2_21

Lee, J, Dubin, JA & Maslove, DM, in Secondary Analysis of Electronic Health Records [Internet] (Springer, [Sept. 10, 2016]), https://doi.org/10.1007/978- 3-319-43742-2_21

doi:10.1007/978- 2016
[2]

org / 10

Wang, L, Guo, X, Shi, H, Ma, Y, Bao, H, Jiang, L, Zhao, L, Feng, Z, Zhu, T & Lu, L, CRISP: A causal relationships- guided deep learning framework for ad- vanced ICU mortality prediction, BMC Medical Informatics and Decision Making 25, 165, https : / / doi . org / 10 . 1186 / s12911-025-02981-1 ([Apr. 15, 2025])

work page 2025
[3]

Devlin, J, Chang, MW, Lee, K & Toutanova,K, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, (May 24, 2019),https:// doi.org/10.48550/arXiv.1810.04805

arXiv doi:10.48550/arxiv.1810.04805 2019
[4]

16, 2021),https:// doi.org/10.48550/arXiv.2106.09685

Hu, EJ, Shen, Y, Wallis, P, Allen-Zhu, Z, Li, Y, Wang, S, Wang, L & Chen, W, LoRA: Low-Rank Adaptation of Large Lan- guage Models,(Oct. 16, 2021),https:// doi.org/10.48550/arXiv.2106.09685

arXiv doi:10.48550/arxiv.2106.09685 2021
[5]

org / 10

Johnson, AEW, Bulgarelli, L, Shen, L, Gayles, A, Shammout, A, Horng, S, Pol- lard, TJ, Hao, S, Moody, B, Gow, B, Lehman, LwH, Celi, LA & Mark, RG, MIMIC-IV, a freely accessible electronic health record dataset, Scientific Data10, Publisher: Nature Publishing Group, 1, https : / / doi . org / 10 . 1038 / s41597 - 022-01899-x ([Jan. 3, 2023])

work page 2023
[6]

org / 10

Pollard, TJ, Johnson, AEW, Raffa, JD, Celi, LA, Mark, RG & Badawi, O, The eICU Collaborative Research Database, a freely available multi-center database for critical care research, Scientific Data References 21 5, Publisher: Nature Publishing Group, 180178, https : / / doi . org / 10 . 1038 / sdata.2018.178 ([Sept. 11, 2018])

work page 2018
[7]

2, 2023), https : / / doi

Vaswani, A, Shazeer, N, Parmar, N, Uszko- reit, J, Jones, L, Gomez, AN, Kaiser, L & Polosukhin, I,Attention Is All You Need, (Aug. 2, 2023), https : / / doi . org / 10 . 48550/arXiv.1706.03762

arXiv doi:10.48550/arxiv.1706.03762 2023
[8]

org/10.48550/arXiv.1907.11692

Liu, Y, Ott, M, Goyal, N, Du, J, Joshi, M, Chen, D, Levy, O, Lewis, M, Zettle- moyer, L & Stoyanov, V,RoBERTa: A Robustly Optimized BERT Pretraining Ap- proach, (July 26, 2019), https : / / doi . org/10.48550/arXiv.1907.11692

arXiv doi:10.48550/arxiv.1907.11692 2019

Show all 13 references

[9]

org / 10

Lee, J, Yoon, W, Kim, S, Kim, D, Kim, S, So, CH & Kang, J, BioBERT: a pre- trained biomedical language representa- tion model for biomedical text mining, Bioinformatics 36, 1234–1240, https:// doi . org / 10 . 1093 / bioinformatics / btz682 ([Feb. 15, 2020])

2020
[10]

13, 2020), https : / / doi

Erickson, N, Mueller, J, Shirkov, A, Zhang, H, Larroy, P, Li, M & Smola, A, AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data, (Mar. 13, 2020), https : / / doi . org / 10 . 48550 / arXiv.2003.06505

2020 doi arXiv
[11]

Ke, G, Meng, Q, Finley, T, Wang, T, Chen, W, Ma, W, Ye, Q & Liu, TY,LightGBM: A Highly Efficient Gradient Boosting Deci- sion Tree,in Advances in Neural Informa- tion Processing Systems30 (Curran Asso- ciates, Inc., [2017])

2017
[12]

org / 10

Hollmann, N, Müller, S, Purucker, L, Krishnakumar, A, Körfer, M, Hoo, SB, Schirrmeister, RT & Hutter, F, Accurate predictions on small data with a tabu- lar foundation model, Nature637, Pub- lisher: Nature Publishing Group, 319–326, https : / / doi . org / 10 . 1038 / s41586 -...

2025
[13]

29, 2022), https : / / doi

Yasunaga, M, Leskovec, J & Liang, P, LinkBERT: Pretraining Language Models with Document Links, (Mar. 29, 2022), https : / / doi . org / 10 . 48550 / arXiv . 2203.15827

2022 doi arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

10, 2016]), https://doi.org/10.1007/978- 3-319-43742-2_21

Lee, J, Dubin, JA & Maslove, DM, in Secondary Analysis of Electronic Health Records [Internet] (Springer, [Sept. 10, 2016]), https://doi.org/10.1007/978- 3-319-43742-2_21

doi:10.1007/978- 2016

[2] [2]

org / 10

Wang, L, Guo, X, Shi, H, Ma, Y, Bao, H, Jiang, L, Zhao, L, Feng, Z, Zhu, T & Lu, L, CRISP: A causal relationships- guided deep learning framework for ad- vanced ICU mortality prediction, BMC Medical Informatics and Decision Making 25, 165, https : / / doi . org / 10 . 1186 / s12911-025-02981-1 ([Apr. 15, 2025])

work page 2025

[3] [3]

Devlin, J, Chang, MW, Lee, K & Toutanova,K, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, (May 24, 2019),https:// doi.org/10.48550/arXiv.1810.04805

arXiv doi:10.48550/arxiv.1810.04805 2019

[4] [4]

16, 2021),https:// doi.org/10.48550/arXiv.2106.09685

Hu, EJ, Shen, Y, Wallis, P, Allen-Zhu, Z, Li, Y, Wang, S, Wang, L & Chen, W, LoRA: Low-Rank Adaptation of Large Lan- guage Models,(Oct. 16, 2021),https:// doi.org/10.48550/arXiv.2106.09685

arXiv doi:10.48550/arxiv.2106.09685 2021

[5] [5]

org / 10

Johnson, AEW, Bulgarelli, L, Shen, L, Gayles, A, Shammout, A, Horng, S, Pol- lard, TJ, Hao, S, Moody, B, Gow, B, Lehman, LwH, Celi, LA & Mark, RG, MIMIC-IV, a freely accessible electronic health record dataset, Scientific Data10, Publisher: Nature Publishing Group, 1, https : / / doi . org / 10 . 1038 / s41597 - 022-01899-x ([Jan. 3, 2023])

work page 2023

[6] [6]

org / 10

Pollard, TJ, Johnson, AEW, Raffa, JD, Celi, LA, Mark, RG & Badawi, O, The eICU Collaborative Research Database, a freely available multi-center database for critical care research, Scientific Data References 21 5, Publisher: Nature Publishing Group, 180178, https : / / doi . org / 10 . 1038 / sdata.2018.178 ([Sept. 11, 2018])

work page 2018

[7] [7]

2, 2023), https : / / doi

Vaswani, A, Shazeer, N, Parmar, N, Uszko- reit, J, Jones, L, Gomez, AN, Kaiser, L & Polosukhin, I,Attention Is All You Need, (Aug. 2, 2023), https : / / doi . org / 10 . 48550/arXiv.1706.03762

arXiv doi:10.48550/arxiv.1706.03762 2023

[8] [8]

org/10.48550/arXiv.1907.11692

Liu, Y, Ott, M, Goyal, N, Du, J, Joshi, M, Chen, D, Levy, O, Lewis, M, Zettle- moyer, L & Stoyanov, V,RoBERTa: A Robustly Optimized BERT Pretraining Ap- proach, (July 26, 2019), https : / / doi . org/10.48550/arXiv.1907.11692

arXiv doi:10.48550/arxiv.1907.11692 2019