Pith. sign in

REVIEW 4 major objections 6 minor 4 references

TT-XAI: Trustworthy Clinical Text Explanations via Keyword Distillation and LLM Reasoning

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Keyword distillation of clinical notes simultaneously improves BERT classification and the faithfulness of LIME and LLM explanations.

desk verdict Keyword distillation is a plausible cheap win for clinical NLP, but the headline F1 gain is confounded by raw text being truncated while keywords are extracted from full notes. read the letter →

arxiv 2508.08273 v1 pith:NCAM3VY4 submitted 2025-07-30 cs.CL cs.LG

Kristian Miok , Blaz Škrlj , Daniela Zaharie , Marko Robnik Šikonja This is my paper · ORCID
classification cs.CLcs.LG
keywords trustworthyAIclinicalNLPelectronichealthrecordskeywordextractionexplainabilityLIMEchain-of-thoughtreasoninglengthofstayprediction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that converting long, noisy discharge summaries into a compact keyword representation, built by the RaKUn graph-based extractor and Med7 clinical entities, improves both prediction and interpretability at once. On a 467-note MIMIC-IV kidney-stone cohort, distilled inputs raise ModernBERT's macro-F1 from 0.665 to 0.767 at a 512-token budget, with gains at every tested context length. The same focus set sharpens LIME into a 'focused LIME' whose deletion-curve AUC drops from 0.742 to 0.668, and steers LLaMA-3 chain-of-thought reasoning that three blinded specialists rate higher (3.15 vs 2.58 on a 1–5 scale). If correct, the result points to a lightweight preprocessing recipe that makes clinical classifiers both more accurate and more auditable.

What carries the argument

The central mechanism is keyword distillation: the RaKUn algorithm builds a word co-occurrence graph per note, ranks terms by load centrality, and forms bigram and trigram keyphrases, which are then unioned with Med7 clinical named entities to form a focused 'focus set' of up to 512 phrases. This focus set plays a triple role: it is the input to the fine-tuned ModernBERT classifier, the restricted perturbation space for the focused LIME variant, and the guiding prefix in the keyword-augmented chain-of-thought prompts.

What would settle it

Inspect the distilled inputs for a random sample of the 467 notes and check whether date-like tokens or explicit stay-length phrases survive; if stripping those tokens makes the keyword advantage over raw text disappear, the classification gain is an artifact of label leakage rather than of distillation.

Watch

Extended reading notes

Core claim

The central claim is that raw discharge notes are information-dilute for both prediction and explanation: most of the signal for a prolonged-stay label is concentrated in a small set of clinically salient terms. By distilling each note into its top keyword phrases (RaKUn) plus medical entities (Med7), the paper reports that ModernBERT's macro-F1 rises from 0.665 to 0.767 at 512 tokens, that a LIME variant perturbing only this focused set yields a lower deletion-curve AUC (0.668 vs 0.742), and that LLM explanations generated from keyword-augmented prompts are rated higher by three blinded specialists (3.15 vs 2.58). The finding is that distillation, a simple unsupervised preprocessing step, improves machine prediction and human-perceived explainability simultaneously, rather than trading one off against the other.

Load-bearing premise

The preprocessing genuinely strips direct length-of-stay statements (admission and discharge dates and any explicit stay-length phrasing) from the notes, so the classifier must learn clinical indicators rather than read the answer off the page.

Editorial extensions

If this is right

  • At a 512-token budget, keyword-distilled inputs raise macro-F1 to 0.767 compared with 0.665 for raw text, implying that the same model can be run on far fewer tokens with better accuracy.
  • Focused LIME's lower deletion AUC (0.668 vs 0.742) means explanations rank decision-driving tokens more accurately, which should make post-hoc auditing of clinical classifiers more reliable.
  • Keyword-guided chain-of-thought prompts produce explanations that blinded specialists rate as clearer and more clinically useful, supporting their use in interactive clinical decision support.
  • Because the distillation is an unsupervised preprocessing step, it can be bolted onto existing clinical classifiers and explanation pipelines without retraining the underlying language model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same distillation-and-focus-set recipe should transfer to other MIMIC-IV prediction tasks (readmission, mortality, sepsis) and to any long-document classifier where the label depends on a few explicit facts.
  • A direct ablation that explicitly strips date tokens and other stay-length phrasing from both raw and distilled inputs would isolate the distillation effect from any label leakage and should be run before clinical deployment.
  • The deletion-curve fidelity measure used for LIME could be adapted to grade the keyword-augmented LLM explanations themselves, unifying the two evaluation branches into a single metric.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes TT-XAI, a framework that distills long MIMIC-IV discharge notes for kidney-stone patients into keyword representations using the RaKUn algorithm and Med7 named-entity recognition, then uses these distilled inputs for two purposes: (1) fine-tuning a ModernBERT classifier to predict prolonged length of stay, and (2) guiding focused LIME explanations and LLM-generated chain-of-thought reasoning. The authors report that keyword inputs outperform raw truncated notes in macro-F1 across context lengths (e.g., 0.767 vs. 0.665 at 512 tokens), that focused LIME achieves lower deletion-curve AUC (0.668 vs. 0.742), and that both LLM-based scoring and a blinded three-specialist study favor keyword-augmented explanations (average 3.15 vs. 2.58). The paper concludes that lightweight keyword distillation simultaneously improves predictive performance, token-level explanation fidelity, and human-perceived explanation quality.

Significance. If the central claim holds, the paper offers a lightweight, domain-aware preprocessing step that could improve both accuracy and interpretability for clinical text models, which would be practically valuable given the length and noise of EHR notes. The manuscript has real strengths: the code is promised to be publicly available, the evaluation includes a blinded human specialist study in addition to automatic metrics, and the pipeline is simple enough to be reproducible. However, the significance is substantially tempered by evaluation-design issues. The classification comparison does not control for information access, because the raw-text baseline is truncated from the beginning of the note while the keyword representation is extracted from the full note; the explanation-fidelity comparison compares different classifiers under different perturbation spaces; and the LLM-based evaluation uses a judge from the same model family as the generator. These issues are load-bearing for the paper's headline claims and require additional experiments or explicit controls before the conclusions can be accepted.

major comments (4)
  1. [Section 4.1, Table 1] The central classification claim is confounded by unequal information access. The 'Original' baseline is truncated to the first 50-1024 tokens of the raw discharge note, while the 'Keywords' representation is built by RaKUn from the entire note, which the authors state can exceed 6,000 tokens (Section 3.1), and is then truncated. Consequently, the keyword model can draw on document-level evidence, whereas the raw model only sees a prefix. Clinically relevant content such as the hospital course or discharge instructions may appear late in MIMIC-IV notes, so the reported F1 gains may reflect access to more of the document rather than the value of domain-aware distillation per se. A fair control should feed the raw model either a random 512-token segment sampled from the full note or the complete note through a long-context model; without such a control, the claim that 'distilled keyword inputs consistently outperform raw text' (Section 4.1) is not established.
  2. [Section 3.1] The paper does not explicitly state that admission and discharge dates were removed from the discharge summaries during preprocessing. MIMIC-IV discharge notes typically contain 'Admission Date' and 'Discharge Date' fields, from which the binary LOS_long label can be computed directly. Section 3.1 mentions only 'basic filtering to remove administrative boilerplate and empty sections' and never rules out this direct label leak. The low raw-text accuracy in Table 1 (e.g., 0.638 at 512 tokens) suggests that dates were probably not present, but this must be stated explicitly and verified in the preprocessing description; otherwise the classification improvements in Section 4.1 and the downstream explanation findings inherit the uncertainty.
  3. [Section 4.2, Section 3.4] The explanation-fidelity comparison is not apples-to-apples. Classical LIME is applied to the raw-text model, while focused LIME is applied to the keyword model, so the deletion curves in Figure 3 compare different classifiers operating on different input spaces. A lower AUC for focused LIME could reflect the fact that the keyword model's decision is more concentrated on a small salient set, rather than that focused LIME produces more faithful explanations. In addition, the focus set F is built from the same RaKun keywords whose benefit is being demonstrated, making the deletion test partially self-referential; and the evaluation is restricted to 20 preselected correctly classified long-stay examples, which limits generality. A cleaner test would compare classical and focused LIME on the same model (e.g., both on the keyword model, or both on a model that takes the full note with the same perturbation budget), and would report how the 20 examples were selected.
  4. [Section 4.3, Section 3.5] The LLM-based evaluation is self-referential and the reporting is inconsistent. Section 3.5 states that 'LLM LLaMA 3 70B then scored each explanation,' yet Section 4.3 and Table 2 describe these as 'expert scores.' More importantly, the scoring model belongs to the same model family (LLaMA-3) as the generator (DeepSeek-distilled LLaMA-3 14B), so apparent preferences for keyword-augmented prompts may reflect a within-family bias rather than a general quality difference. The human specialist study (Section 4.4, Table 3) is an independent check, but it uses only 10 explanation pairs and three raters, and one of the three specialists actually scored Method B lower (3.33 vs. 3.45), so the 'consensus' phrasing in Section 4.4 overstates the evidence. A larger, more diverse rater pool and a non-LLM or cross-family judge would be needed to support the claim that keyword augmentation consistently improves explanation clarity and clinical usefulness.
minor comments (6)
  1. [Section 4.3, Table 2] The text says 'Experts rated each' but the experimental protocol in Section 3.5 specifies LLaMA-3 70B scoring; please replace 'expert' with 'LLM judge' or clarify that these are LLM-provided scores.
  2. [Section 3.4] The deletion-test description says 'top-k influential tokens' but never specifies the value of k or the step size used to construct the deletion curve; please provide these details for reproducibility.
  3. [Section 4.2 vs. Section 3.5] It is unclear whether the 20 correctly classified long-stay examples used for deletion curves are the same 20 true-positive cases used for LLM reasoning; please state the overlap explicitly.
  4. [Table 1] No statistical significance tests are reported for the accuracy/F1 differences across folds; given the small cohort (467 admissions) and the overlapping standard deviations at several context lengths, a paired test or confidence interval would strengthen the 'consistently outperform' claim.
  5. [Figure 3 caption] The caption's phrase 'explanations aligned with tokens in the explanations' is circular and should be replaced with a precise description of what the deletion curve measures.
  6. [Section 4.4, Table 3] Specialist 3 gave Method A a higher score than Method B (3.45 vs. 3.33); the text acknowledges this but should soften the phrase 'consensus clearly indicates' in Section 4.4 accordingly.

Circularity Check

2 steps flagged · score 3.0 of 10

Partially self-referential evaluations: focused LIME and LLM self-assessment, but classification and human study are independent.

  1. other [Section 3.4 (Focused LIME) and Section 4.2 (Explanation Faithfulness).]
    "This set is constructed by taking the union of: – The top 512 keyphrases extracted by the Rakun algorithm for that note, and – Named clinical entities identified by Med7 [Kormilitzin et al., 2021], excluding low-informative categories such as dosage and frequency terms."

    The focus set for focused LIME is defined as the top Rakun keyphrases and Med7 entities, i.e., the same keyword distillation whose benefit Section 4.2 claims to demonstrate. The fidelity test then measures how much the model's prediction changes when these same keyword-derived tokens are deleted. This makes the evaluation space self-referential: the method is tested on its own output. A control with a different focus set would be needed to establish that keyword distillation per se, rather than the restricted perturbation space, improves explanation fidelity.

  2. other [Section 3.5 (LLM reasoning setup) and Section 4.3 (LLM scoring).]
    "We evaluate explanation quality using deletion-based fidelity metrics, self-assessment via LLaMA-3 scoring, and a blinded human study with domain experts. ... LLM LLaMA 3 70B then scored each explanation on a 1–5 scale for clarity and clinical relevance."

    The quality signal for the LLM reasoning branch is generated by a LLaMA-3 model (70B) scoring outputs produced by a DeepSeek-distilled LLaMA-3 14B model. The paper itself labels the procedure as self-assessment. This is self-referential because the model family serves as both generator and judge. However, the blinded human specialist study in Section 4.4 provides independent external evidence, so this step is not load-bearing for the entire paper.

full rationale

The classification comparison is not circular: the keyword inputs are produced by an unsupervised extractor from the full note, and the raw-text baseline is an independent benchmark, even though the raw-text truncation is a confounding control issue. The focused LIME fidelity comparison in Section 4.2 is partially self-referential because the perturbation set is the output of the same Rakun/Med7 distillation being tested. The LLM scoring in Section 4.3 is also a same-family self-assessment. Neither step forces the reported result by construction: the deletion AUC and average ratings are contingent, and the human expert study is an independent external check. The paper's self-citation to Rakun is not load-bearing. This yields a score of 3: some evaluative self-referentiality, but the central claim still has independent content.

Assumptions & free parameters 8 free parameters · 6 assumptions · 0 invented entities

The central claim rests on the keyword extraction configuration (Rakun thresholds, candidate and retained budgets, Med7 categories), the unstated removal of date fields that would leak the label, the validity of deletion-curve AUC and LLM-as-judge metrics, and the representativeness of a 467-note cohort. No new physical or theoretical entities are introduced.

free parameters (8)
  • Rakun merge threshold = 1.1
    Controls meta-vertex merging in keyword extraction; chosen without sensitivity analysis, and changes which terms become keywords.
  • Rakun alpha = 0.3
    Weight for balancing frequency and informativeness in ranking; fixed without ablation.
  • Rakun minimum token length = 3
    Filters out short tokens before graph construction; arbitrary and can alter the keyword list.
  • Keyword candidate budget = 1024
    Maximum number of keyword candidates extracted per note; not justified by analysis.
  • Retained keyphrase budget = 512
    Only the top 512 keyphrases are kept as the distilled input; this is the core representation whose benefit is claimed.
  • LLM sampling temperature = 0.5
    Fixed for all reasoning generation; no sensitivity analysis or justification.
  • Fine-tuning learning rate = 2e-6
    Used for ModernBERT in both pipelines; standard but not justified for this task.
  • Fine-tuning epochs = 3
    Training duration for each fold; fixed without early stopping or tuning.
assumptions (6)
  • ad hoc to paper Preprocessing removes all direct length-of-stay signals, including admission and discharge dates, from the discharge summaries.
    MIMIC-IV discharge notes contain 'Admission Date' and 'Discharge Date' fields; the paper never explicitly confirms their removal, yet the classification comparison depends on the model not being able to read the label directly (Section 3.1).
  • domain assumption Rakun load centrality and Med7 NER terms capture the clinically salient content for LOS prediction.
    The keyword representation is assumed to preserve predictive signal while removing noise; no random-keyword baseline or ablation is provided (Sections 3.2 and 4.1).
  • domain assumption Deletion-curve AUC is a valid measure of explanation faithfulness.
    Section 3.4 defines lower AUC after removing top-ranked tokens as better fidelity; this equates output sensitivity with faithfulness, which is an interpretive assumption.
  • domain assumption LLaMA-3 70B ratings of clarity and clinical usefulness are a valid proxy for human judgments.
    Section 4.3 relies on the LLM judge without validating it against human preferences or accounting for known LLM-judge biases.
  • domain assumption The cohort median LOS is a clinically meaningful threshold for 'prolonged stay.'
    Section 3.1 sets labels by thresholding at the median, producing a dataset-relative definition rather than an external clinical standard.
  • domain assumption The 467-note kidney-stone cohort is representative enough for 5-fold CV conclusions.
    Single diagnosis, single institution, and no external validation; cross-validation assumes the sample represents the target population.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TT-XAI: Trustworthy Clinical Text Explanations via Keyword Distillation and LLM Reasoning." pith.science (2026). https://pith.science/paper/NCAM3VY4

@misc{pith2026250808273,
  author       = {Pith},
  title        = {Pith review of: TT-XAI: Trustworthy Clinical Text Explanations via Keyword Distillation and LLM Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NCAM3VY4}},
  note         = {Machine review of arXiv:2508.08273}
}
read the original abstract

Clinical language models often struggle to provide trustworthy predictions and explanations when applied to lengthy, unstructured electronic health records (EHRs). This work introduces TT-XAI, a lightweight and effective framework that improves both classification performance and interpretability through domain-aware keyword distillation and reasoning with large language models (LLMs). First, we demonstrate that distilling raw discharge notes into concise keyword representations significantly enhances BERT classifier performance and improves local explanation fidelity via a focused variant of LIME. Second, we generate chain-of-thought clinical explanations using keyword-guided prompts to steer LLMs, producing more concise and clinically relevant reasoning. We evaluate explanation quality using deletion-based fidelity metrics, self-assessment via LLaMA-3 scoring, and a blinded human study with domain experts. All evaluation modalities consistently favor the keyword-augmented method, confirming that distillation enhances both machine and human interpretability. TT-XAI offers a scalable pathway toward trustworthy, auditable AI in clinical decision support.

Figures

Figures reproduced from arXiv: 2508.08273 by the authors.

Figure 1
Figure 1. System overview of TT-XAI. Raw clinical notes (blue) are dis [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Prompt variants used to elicit clinical reasoning from LLaMA-3. The [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Average deletion curve across 20 long-stay examples comparing classical [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

4 extracted references · 1 canonical work pages

  1. [323]

    Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hall- ström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, et al

    Springer, 2019. Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hall- ström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, et al. Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference.arXiv preprint arXiv:2412.13663, 2024. J...

    arXiv 2019

  2. [657]

    Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marcel Sturm, and Noemie Elhadad

    ACM, 2020. Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marcel Sturm, and Noemie Elhadad. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. InProceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1721–1730. ACM, 2015. Shruthi Chari, Oshani Seneviratne, M...

    work page 2020

  3. [2022]

    Xuansheng Wu, Haiyan Zhao, Yaochen Zhu, Yucheng Shi, Fan Yang, Tianming Liu, Xiaoming Zhai, Wenlin Yao, Jundong Li, Mengnan Du, et al

    URLhttps://arxiv.org/abs/2201.11903. Xuansheng Wu, Haiyan Zhao, Yaochen Zhu, Yucheng Shi, Fan Yang, Tianming Liu, Xiaoming Zhai, Wenlin Yao, Jundong Li, Mengnan Du, et al. Usable xai: 10 strategies towards exploiting explainability in the llm era.arXiv preprint arXiv:2403.08946, 2024. Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J. Liu. PEGASUS: Pr...

    arXiv 2024

  4. [4205]

    why should i trust you?

    Association for Computational Linguistics, 2020. Sarthak Jain and Byron C. Wallace. Attention is not explanation. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3543–3556. Association for Computational Linguistics, 2019. 16 K. Miok et al. Alistair E. W...

    arXiv 2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.