REVIEW 4 major objections 5 minor 43 references
Structured Information Matters: Explainable ICD Coding with Patient-Level Knowledge Graphs
T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read The paper argues that automated ICD coding improves when the coder also sees a structured, patient-level knowledge graph extracted from the clinical note, not just the raw text. On its own benchmarks, this graph branch raises Macro-F1 from
desk verdict Solid extension of patient-level KG coding, but the headline F1 gains are not yet attributable to graph structure — needs a capacity-matched control and a proper validation split. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the patient-level knowledge graph: a document-level graph assembled from 14 entity types—problems, treatments, tests, drugs, dosages, frequencies, routes, forms, body parts, symptoms, and related categories—connected by five relation families: clinical, temporal, posology, bodypart-direction, and bodypart-problem. Entities and relations are extracted by off-the-shelf clinical named-entity recognition and relation extraction models, then serialized as triples. A deep graph convolutional network (DGCNN) embeds the graph, with node features concatenated across layers, and these graph embeddings are fused with text segment embeddings from a biomedical pre-trained trans
What would settle it
Train the same model with a control graph built from randomly paired entities and random relation labels, keeping graph size and model capacity fixed. If Macro-F1 stays near 11.05 on the full label set—or the gap to the text-only baseline stays near 1.36—then the specific structure of the graph is not what drives the gain. The paper's own entity and relation ablations provide a direct partial check: removing the problem entity drops Macro-F1 to 9.77, so a random-graph control would fill in what remains unexplained.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that a patient-level knowledge graph is a useful patient representation for ICD-9 coding: it retains most of the note's information in a small fraction of its tokens, and injecting that graph into a transformer-based coder improves Macro-F1 over the text-only baseline by 1.36 points on the full label set and 3.20 points on the top-50 subset. The gains are not uniform across graph components. Ablations show that the 'clinical relationship' relation and the 'problem' entity type carry the most coding signal, while body-part-direction relations contribute least. The authors interpret these patterns as evidence that structured, entity-and-relati
Load-bearing premise
The observed F1 gains are caused by the structured content of the knowledge graph, not by the added DGCNN encoder and extra attention head; if a text-only model with equal added capacity matched the graph model's F1, the structured-information explanation would be unsupported.
Editorial extensions
If this is right
- ICD coding accuracy improves on both the full and top-50 code sets, with the largest gains on frequent codes and on codes the text-only model misses entirely, such as anemia and personal history of tobacco use.
- Training efficiency improves: the graph branch reaches competitive Macro-F1 within the first few epochs, which is practically useful when computational resources are constrained.
- The ablation results give a prioritization signal for graph construction: clinical relationships and problem entities matter most, so future extraction pipelines can focus annotation or model effort on those components.
- The graph offers a compact structured artifact—about a quarter of the original note—that can serve simultaneously as an input feature and as a source of attention-based evidence for predictions.
- The paper's gains are demonstrated on top of its chosen baseline; the authors note that integrating structured input into more recent, stronger coding architectures could yield further improvements.
Reading between the lines
- My inference: the same graph-construction and fusion recipe would likely transfer to other clinical coding settings, including ICD-10 and long clinical documents beyond discharge summaries, because the entity and relation types are largely vocabulary-agnostic; this transfer is not tested in the paper.
- My inference: a matched-capacity text-only control or a control with randomly wired graphs would isolate whether the benchmark improvements come from the graph's structure or simply from the extra DGCNN parameters; without such a control, the causal attribution to structure remains open.
- My inference: the entropy-based compression measure could be used as a routing signal—documents whose graphs lose more information than usual could be flagged for additional full-text processing, a practical design the paper does not explore.
- My inference: the attention highlights shown in the case studies are visually more focused than the text-only baseline, but turning this into a trustworthy clinician-facing explanation would require expert evalution rather than visual inspection alone.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes constructing patient-level knowledge graphs from MIMIC-III discharge summaries using Healthcare NLP named-entity recognition and relation-extraction models, yielding 14 entity types and 5 relationship types. These graphs are integrated into the PLM-ICD architecture by adding a DGCNN encoder whose node representations are initialized with RoBERTa-base, followed by a separate multi-head label-wise attention layer; text and graph representations are concatenated for ICD-9 code prediction. The authors report Macro-F1 improvements over PLM-ICD of +1.36 on MIMIC-III Full (9.69 to 11.05) and +3.20 on Top-50 (64.61 to 67.81), faster early-epoch training, an entropy-based claim that the graph retains about 90% of the information in 23% of the original text, and qualitative evidence of improved attention-based explanations. Ablations remove individual relationship and entity types to analyze their contribution.
Significance. The idea of using patient-level knowledge graphs to represent the input document, rather than only to enhance code representations, is a genuinely underexplored direction in automated ICD coding. If the reported gains are robust, the approach could provide a practical way to inject structured clinical information into PLM-based coders and to improve the conciseness of attention-based explanations. However, the significance is currently moderated by the absence of a capacity-matched text-only control and a structure-destroying graph control, by apparent model selection on the test set, and by the absence of variance/error reporting. These issues prevent the paper from establishing its central causal claim that the structure of the knowledge graph, rather than added model capacity or surface entity cues, drives the improvement.
major comments (4)
- [§3, Figure 2, Eqs. (5)–(8); Table 4]
- [Appendix A.4, Tables 10–11]
- [§4.2, Table 3]
- [§3, Eqs. (10)–(13); Tables 2 and 7]
minor comments (5)
- [Appendix A.1, Table 8 text]
- [Figure 2]
- [§6 Limitations]
- [Eq. (9)]
- [Figure 6]
Circularity Check
No significant circularity: F1 gains are genuine held-out predictions; the only self-citation (Nguyen et al. 2023) is non-load-bearing, and the capacity-matched-control concern is a confound, not a definitional reduction.
full rationale
The paper's central claim is that integrating patient-level knowledge graphs into PLM-ICD improves ICD coding, evaluated on MIMIC-III Full and Top-50 held-out test sets with external ICD-9 labels. The Macro-F1 improvements are measured predictions from a trained model, not quantities fitted to the test labels; no equation defines the output in terms of the KG construction itself. Equations (1)-(9) describe a standard text-plus-graph multi-label classifier with binary cross-entropy loss on held-out labels, so the coding result is not equivalent to the input by construction. The entropy-based 'information retained' metric (Table 2, Eqs. 10-13) is an internal compression statistic comparing serialized graph to original text; it is not presented as the coding result, so it does not create circularity in the coding claim. The ablations (Table 4) are empirical sensitivity analyses that retain the same model architecture, and although they do not isolate graph structure from added capacity, that is a confound in experimental attribution, not a circular derivation. The only self-citation is Nguyen et al. (2023), co-authored by Viktor Schlegel, used as a baseline (2Stage) and referenced in Limitations as a more advanced model; it is not load-bearing for the paper's central claim. The paper itself acknowledges not testing other baseline architectures. No specific circular step can be exhibited via quotation and reduction, so the score reflects only the minor non-load-bearing self-citation and otherwise the evaluation is self-contained against external benchmarks.
Assumptions & free parameters
free parameters (2)
- DGCNN architecture (layers and node embedding size) =
Full: 1 layer, 768; Top-50: 2 layers, 384-384
- RE model selection (5 of 14) =
CR, TE, PR, BD, BP
assumptions (4)
- domain assumption Named entities and relationships extracted by the off-the-shelf Healthcare NLP models are sufficiently accurate and relevant for ICD coding.
- ad hoc to paper Token-level Shannon entropy of a serialized graph measures how much clinical information is retained.
- domain assumption Label-wise attention weights in the graph branch represent feature importance for explainability.
- domain assumption The graph and text representations provide complementary, non-redundant information.
Cite this review
Pith. "Pith review of Structured Information Matters: Explainable ICD Coding with Patient-Level Knowledge Graphs." pith.science (2026). https://pith.science/paper/XNEBNCAM
@misc{pith2026250909699,
author = {Pith},
title = {Pith review of: Structured Information Matters: Explainable ICD Coding with Patient-Level Knowledge Graphs},
year = {2026},
howpublished = {\url{https://pith.science/paper/XNEBNCAM}},
note = {Machine review of arXiv:2509.09699}
}
read the original abstract
Mapping clinical documents to standardised clinical vocabularies is an important task, as it provides structured data for information retrieval and analysis, which is essential to clinical research, hospital administration and improving patient care. However, manual coding is both difficult and time-consuming, making it impractical at scale. Automated coding can potentially alleviate this burden, improving the availability and accuracy of structured clinical data. The task is difficult to automate, as it requires mapping to high-dimensional and long-tailed target spaces, such as the International Classification of Diseases (ICD). While external knowledge sources have been readily utilised to enhance output code representation, the use of external resources for representing the input documents has been underexplored. In this work, we compute a structured representation of the input documents, making use of document-level knowledge graphs (KGs) that provide a comprehensive structured view of a patient's condition. The resulting knowledge graph efficiently represents the patient-centred input documents with 23\% of the original text while retaining 90\% of the information. We assess the effectiveness of this graph for automated ICD-9 coding by integrating it into the state-of-the-art ICD coding architecture PLM-ICD. Our experiments yield improved Macro-F1 scores by up to 3.20\% on popular benchmarks, while improving training efficiency. We attribute this improvement to different types of entities and relationships in the KG, and demonstrate the improved explainability potential of the approach over the text-only baseline.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
James Blundell. 2023. Health information and the importance of clinical coding. Anaesthesia & Intensive Care Medicine
work page 2023
-
[2]
Pengfei Cao, Yubo Chen, Kang Liu, Jun Zhao, Shengping Liu, and Weifeng Chong. 2020. Hypercore: Hyperbolic and co-graph representation for automatic icd coding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3105--3114
work page 2020
-
[3]
Finneas Catling, Georgios P Spithourakis, and Sebastian Riedel. 2018. Towards automated clinical coding. International journal of medical informatics, 120:50--61
work page 2018
-
[4]
Ilias Chalkidis, Manos Fergadiotis, Sotiris Kotitsas, Prodromos Malakasiotis, Nikolaos Aletras, and Ion Androutsopoulos. 2020. An empirical study on large-scale multi-label text classification including few and zero-shot labels. arXiv preprint arXiv:2010.01653
work page Pith review arXiv 2020
-
[5]
Koby Crammer, Mark Dredze, Kuzman Ganchev, Partha Talukdar, and Steven Carroll. 2007. Automatic code assignment to medical text. In Biological, translational, and clinical language processing, pages 129--136
work page 2007
-
[6]
Hang Dong, V \' ctor Su \'a rez-Paniagua, William Whiteley, and Honghan Wu. 2021. Explainable automated coding of clinical notes using hierarchical label-wise attention networks and label embedding initialisation. Journal of biomedical informatics, 116:103728
work page 2021
-
[7]
Joakim Edin, Alexander Junge, Jakob D Havtorn, Lasse Borgholt, Maria Maistro, Tuukka Ruotsalo, and Lars Maal e. 2023. Automated medical coding on mimic-iii and mimic-iv: a critical review and replicability study. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2572--2582
work page 2023
-
[8]
Mat \'u s Falis, Hang Dong, Alexandra Birch, and Beatrice Alex. 2022. Horses to zebras: ontology-guided data augmentation and synthesis for icd-9 coding. In Proceedings of the 21st Workshop on Biomedical Language Processing. Association for Computational Linguistics
work page 2022
Show all 43 references
-
[9]
Mat \'u s Falis, Maciej Pajak, Aneta Lisowska, Patrick Schrempf, Lucas Deckers, Shadia Mikhael, Sotirios Tsaftaris, and Alison O’Neil. 2019. Ontological attention ensembles for capturing semantic concepts in icd code prediction from clinical text. In Proceedings of the Tenth I...
2019
-
[10]
Malte Feucht, Zhiliang Wu, Sophia Althammer, and Volker Tresp. 2021. Description-based label attention classifier for explainable icd-9 classification. arXiv preprint arXiv:2109.12026
2021 arXiv
-
[11]
Chao-Wei Huang, Shang-Chi Tsai, and Yun-Nung Chen. 2022. Plm-icd: automatic icd coding with pretrained language models. arXiv preprint arXiv:2207.05289
2022 arXiv
-
[12]
Shaoxiong Ji, Matti H \"o ltt \"a , and Pekka Marttinen. 2021. Does the magic of bert apply to medical code assignment? a quantitative study. Computers in biology and medicine, 139:104998
2021
-
[13]
Shaoxiong Ji, Xiaobo Li, Wei Sun, Hang Dong, Ara Taalas, Yijia Zhang, Honghan Wu, Esa Pitk \"a nen, and Pekka Marttinen. 2022. A unified review of deep learning for automated medical coding. ACM Computing Surveys
2022
-
[14]
John Snow Labs . 2024. Healthcare NLP . https://www.johnsnowlabs.com/healthcare-nlp/
2024
-
[15]
Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. 2016. Mimic-iii, a freely accessible critical care database. Scientific data, 3(1):1--9
2016
-
[16]
Patrick Lewis, Myle Ott, Jingfei Du, and Veselin Stoyanov. 2020. Pretrained language models for biomedical and clinical tasks: understanding and extending the state-of-the-art. In Proceedings of the 3rd clinical natural language processing workshop, pages 146--157
2020
-
[17]
Fei Li and Hong Yu. 2020. Icd coding from clinical text using multi-filter residual convolutional neural network. In proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8180--8187
2020
-
[18]
Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard Zemel. 2015. Gated graph sequence neural networks. arXiv preprint arXiv:1511.05493
2015 arXiv
-
[19]
Yang Liu, Hua Cheng, Russell Klopfer, Matthew R Gormley, and Thomas Schaaf. 2021. Effective convolutional attention network for multi-label clinical document classification. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5941--5953
2021
-
[20]
Jueqing Lu, Lan Du, Ming Liu, and Joanna Dipnall. 2020. Multi-label few/zero-shot learning with knowledge aggregated from multiple label graphs. arXiv preprint arXiv:2010.07459
2020 arXiv
-
[21]
George Michalopoulos, Michal Malyska, Nicola Sahar, Alexander Wong, and Helen Chen. 2022. Icdbigbird: a contextual embedding model for icd code classification. arXiv preprint arXiv:2204.10408
2022 arXiv
-
[22]
James Mullenbach, Sarah Wiegreffe, Jon Duke, Jimeng Sun, and Jacob Eisenstein. 2018. Explainable prediction of medical codes from clinical text. arXiv preprint arXiv:1802.05695
2018 arXiv
-
[23]
Anthony N Nguyen, Donna Truran, Madonna Kemp, Bevan Koopman, David Conlan, John O’Dwyer, Ming Zhang, Sarvnaz Karimi, Hamed Hassanzadeh, Michael J Lawley, et al. 2018. Computer-assisted diagnostic coding: effectiveness of an nlp-based approach using snomed ct to icd-10 mappings...
2018
-
[24]
Thanh-Tung Nguyen, Viktor Schlegel, Abhinav Kashyap, and Stefan Winkler. 2023. A two-stage decoder for efficient icd coding. arXiv preprint arXiv:2306.00005
2023 arXiv
-
[25]
World Health Organization et al. 1978. International classification of diseases:[9th] ninth revision, basic tabulation list with alphabetic index. World Health Organization
1978
-
[26]
Suzanne Pereira, Aur \'e lie N \'e v \'e ol, Philippe Massari, Michel Joubert, and Stefan Darmoni. 2006. Construction of a semi-automated icd-10 coding help system to optimize medical and economic coding. In MIE, pages 845--850. Citeseer
2006
-
[27]
Anthony Rios and Ramakanth Kavuluru. 2018. Few-shot and zero-shot multi-label learning for structured label spaces. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing, volume 2018...
2018
-
[28]
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In The semantic web: 15th international conference, ESWC 2018, Heraklion, Crete, Greece, June 3--7, 2018, proc...
2018
-
[29]
Congzheng Song, Shanghang Zhang, Najmeh Sadoughi, Pengtao Xie, and Eric Xing. 2021. Generalized zero-shot text classification for icd coding. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pages 4018--4024
2021
-
[30]
Wei Sun, Shaoxiong Ji, Erik Cambria, and Pekka Marttinen. 2021. Multitask recalibrated aggregation network for medical code prediction. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 367--383. Springer
2021
-
[31]
Fei Teng, Yiming Liu, Tianrui Li, Yi Zhang, Shuangqing Li, and Yue Zhao. 2022. A review on deep neural networks for icd coding. IEEE Transactions on Knowledge and Data Engineering, 35(5):4357--4375
2022
-
[32]
Betty Van Aken, Jens-Michalis Papaioannou, Marcel G Naik, Georgios Eleftheriadis, Wolfgang Nejdl, Felix A Gers, and Alexander L \"o ser. 2022. This patient looks like that patient: Prototypical networks for interpretable diagnosis prediction from clinical text. arXiv preprint ...
2022 arXiv
-
[33]
Petar Veli c kovi \'c , Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903
2017 arXiv
-
[34]
Thanh Vu, Dat Quoc Nguyen, and Anthony Nguyen. 2020. A label attention model for icd coding from clinical text. arXiv preprint arXiv:2007.06351
2020 arXiv
-
[35]
Tao Wang, Linhai Zhang, Chenchen Ye, Junxi Liu, and Deyu Zhou. 2022. A novel framework based on medical concept driven attention for explainable medical code prediction via external knowledge. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1407--1416
2022
-
[36]
Xiancheng Xie, Yun Xiong, Philip S Yu, and Yangyong Zhu. 2019. Ehr coding with multi-scale feature attention and structured knowledge graph propagation. In Proceedings of the 28th ACM international conference on information and knowledge management, pages 649--658
2019
-
[37]
Zhichao Yang, Shufan Wang, Bhanu Pratap Singh Rawat, Avijit Mitra, and Hong Yu. 2022. Knowledge injected prompt based fine-tuning for multi-label few-shot icd coding. In Proceedings of the conference on empirical methods in natural language processing. Conference on empirical ...
2022
-
[38]
Quan Yuan, Jun Chen, Chao Lu, and Haifeng Huang. 2021. The graph-based mutual attentive network for automatic diagnosis. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pages 3393--3399
2021
-
[39]
Zheng Yuan, Chuanqi Tan, and Songfang Huang. 2022. Code synonyms do matter: Multiple synonyms matching network for automatic icd coding. arXiv preprint arXiv:2203.01515
2022 arXiv
-
[40]
Muhan Zhang, Zhicheng Cui, Marion Neumann, and Yixin Chen. 2018. An end-to-end deep learning architecture for graph classification. In AAAI
2018
-
[41]
Zachariah Zhang, Jingshu Liu, and Narges Razavian. 2020. Bert-xml: Large scale automated icd coding using bert pretraining. arXiv preprint arXiv:2006.03685
2020 arXiv
-
[42]
online" 'onlinestring :=
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...
-
[43]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.