{"id":"2f8f6ddc-1b7f-4d77-9b1f-5cf5b23e8eb1","arxiv_id":"2509.09699","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Integrating patient-level knowledge graphs into the PLM-ICD model improves ICD-9 coding Macro-F1 by up to 3.2% on MIMIC-III while adding explainability.","lead":"This paper builds a knowledge graph of a patient's medical condition from a discharge summary, then feeds both the graph and the raw text into an ICD coding model, improving Macro-F1 by up to 3.2 percent. A generalist should read it because automated clinical coding errors cost hospitals and patients, and this is a clean test of whether structured patient summaries help.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing capacity-matched/random-graph control: reported F1 gains are compatible with added DGCNN+attention parameters and entity features, not necessarily the KG's structure.","rationale":"I reviewed the claims and evidence. The strongest experimental result (Table 3) is plausible, and the architecture description is sufficiently detailed to reproduce; the paper also candidly states limitations (no comparison with other graph types, no relation semantics modeled). However, the headline causal attribution--that structured information, not extra capacity, drives the gains--is not established by the current experimental design. This is exactly the weakest assumption the reader identified. The ablations in Table 4 keep the added DGCNN branch, so they cannot separate structure from capacity. The entropy-based graph-quality analysis and t-SNE visualizations are suggestive but secondary; the explainability examples are selected on outcome and on a label with large improvement. None of this constitutes fraud or internal inconsistency, and the concern is addressable with a random-edge or matched-capacity control plus validation-based hyperparameter selection. Therefore the reader's CONDITIONAL verdict remains appropriate; I would not change it.","tokens_in":21480,"tokens_out":4380,"duration_ms":45375,"concrete_test":"Run the Full and Top-50 experiments with the same PLM-ICD+DGCNN model, node embeddings, and hyperparameters (Table 9), but rewire each patient KG's edges randomly while preserving the degree sequence; run at least 3 seeds and report mean +/- std Macro-F1. If random-rewired graphs keep performance near 10.98-11.05 (Full) and 67.5-67.8 (Top-50), the specific relational structure is not the cause and the central claim fails. If performance drops toward the text-only baseline (9.69/64.61), the structure matters and the claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim--that patient-level knowledge graph structure improves ICD coding--is supported only by comparing PLM-ICD (text-only) against PLM-ICD plus a new graph branch. In Section 3 (Figure 2), adding the graph means adding a DGCNN encoder whose node embeddings are initialized with RoBERTa-base, plus a second multi-head label-wise attention layer (Eqs. 5-7) and a concatenated prediction head. Thus the architecture grows in capacity by a substantial, unmeasured amount. The ablations in Table 4 do not fix this: they remove relationship/entity types from the graph but keep the added DGCNN+attention machinery. Even with Clinical Relationship triples removed, Macro-F1 is 10.07 vs PLM-ICD's 9.69; with 'problem' entities removed it is 9.77. These residual gains can be explained by the extra parameters and by simple bag-of-entity surface cues, without any contribution from graph topology or relational semantics. The statistical 'information retained' result (Table 2) is about serialized-graph compression, not about causal attribution of the coding gains. Without a capacity-matched text-only baseline or a structure-destroying control, the observed 1.36%/3.20% Macro-F1 improvements do not establish that structured information, rather than added model capacity, is what matters.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes constructing patient-level knowledge graphs from MIMIC-III discharge summaries using Healthcare NLP named-entity recognition and relation-extraction models, yielding 14 entity types and 5 relationship types. These graphs are integrated into the PLM-ICD architecture by adding a DGCNN encoder whose node representations are initialized with RoBERTa-base, followed by a separate multi-head label-wise attention layer; text and graph representations are concatenated for ICD-9 code prediction. The authors report Macro-F1 improvements over PLM-ICD of +1.36 on MIMIC-III Full (9.69 to 11.05) and +3.20 on Top-50 (64.61 to 67.81), faster early-epoch training, an entropy-based claim that the graph retains about 90% of the information in 23% of the original text, and qualitative evidence of improved attention-based explanations. Ablations remove individual relationship and entity types to analyze their contribution.","tokens_in":21774,"tokens_out":4186,"duration_ms":42750,"significance":"The idea of using patient-level knowledge graphs to represent the input document, rather than only to enhance code representations, is a genuinely underexplored direction in automated ICD coding. If the reported gains are robust, the approach could provide a practical way to inject structured clinical information into PLM-based coders and to improve the conciseness of attention-based explanations. However, the significance is currently moderated by the absence of a capacity-matched text-only control and a structure-destroying graph control, by apparent model selection on the test set, and by the absence of variance/error reporting. These issues prevent the paper from establishing its central causal claim that the structure of the knowledge graph, rather than added model capacity or surface entity cues, drives the improvement.","major_comments":[{"comment":"","section":"§3, Figure 2, Eqs. (5)–(8); Table 4"},{"comment":"","section":"Appendix A.4, Tables 10–11"},{"comment":"","section":"§4.2, Table 3"},{"comment":"","section":"§3, Eqs. (10)–(13); Tables 2 and 7"}],"minor_comments":[{"comment":"","section":"Appendix A.1, Table 8 text"},{"comment":"","section":"Figure 2"},{"comment":"","section":"§6 Limitations"},{"comment":"","section":"Eq. (9)"},{"comment":"","section":"Figure 6"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is promising and the experiments are on standard benchmarks, but the central attribution claim needs substantially stronger experimental support. In addition to the capacity/control issues, the apparent selection of DGCNN hyperparameters on the test set is a review-blocking concern that should be fixed before the paper is reconsidered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look if you work on ICD coding or structured input representations. The paper builds patient-level knowledge graphs with 14 entity types and 5 relations via commercial NLP, plugs them into PLM-ICD with a DGCNN branch, and reports Macro-F1 gains of 1.36 (Full) and 3.20 (Top-50) on MIMIC-III. The numbers are genuinely new and the ablation over relationship/entity types is a sensible first step. What is actually new is the breadth: Yuan et al. only used symptoms/signs/diseases and a causal graph; this one adds drugs, dosages, temporality, body parts.\n\nThe soft spots are real but not fatal. The main one is causal attribution. Adding the graph branch adds a DGCNN encoder with RoBERTa-initialized nodes plus a second label-wise attention head, so the parameter count grows substantially. The paper never runs a matched-capacity text-only control or a random-graph/edge-shuffle control. The ablation removes relation/entity types but keeps the extra machinery, so residual gains over PLM-ICD (even with CR removed, 10.07 vs 9.69) can be explained by capacity or surface entity cues. So the title claim 'structured information matters' is not established at the reported magnitude. It's not refuted either — the graph may well help — but the evidence currently supports 'adding a second encoder helps.'\n\nSecond, Appendix A.4 reports a sweep over DGCNN layer configurations (Tables 10-11) and the best on each dataset is then used for the main result. That is test-set model selection; with no validation split and no repeated seeds, the headline numbers are optimistic. The paper should state this explicitly or move selection to a validation split.\n\nThe information-retention entropy measure is interesting but separate: it shows the serialized graph is a compact encoding of token statistics, not that the graph causes the F1 gain.\n\nOn the positive side, the writing is honest about limitations — they acknowledge no comparison to other patient KGs and no R-GCN/GAT, and the failure to adapt other baselines. The graphs themselves could be a useful artifact if released, but no release is mentioned.\n\nWho's this for? People building structured input representations for clinical NLP, and anyone who wants a case study in why ablation-only attribution can mislead. It deserves a serious referee — the core idea is reasonable and the benchmark is standard — but a referee should ask for the control, the split, and seeds before accepting the central claim.","headline":"Solid extension of patient-level KG coding, but the headline F1 gains are not yet attributable to graph structure — needs a capacity-matched control and a proper validation split.","tokens_in":22297,"tokens_out":1763,"would_cite":true,"duration_ms":18018,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that automated ICD coding improves when the coder also sees a structured, patient-level knowledge graph extracted from the clinical note, not just the raw text. On its own benchmarks, this graph branch raises Macro-F1 from","keywords":["clinical coding","ICD-9","patient-level knowledge graph","DGCNN","MIMIC-III","multi-label classification","clinical NLP","explainability"],"falsifier":"Train the same model with a control graph built from randomly paired entities and random relation labels, keeping graph size and model capacity fixed. If Macro-F1 stays near 11.05 on the full label set—or the gap to the text-only baseline stays near 1.36—then the specific structure of the graph is not what drives the gain. The paper's own entity and relation ablations provide a direct partial check: removing the problem entity drops Macro-F1 to 9.77, so a random-graph control would fill in what remains unexplained.","tokens_in":1742,"feed_emoji":"🏥","tokens_out":3502,"duration_ms":70889,"temperature":0.7,"pith_summary":"The paper argues that automated ICD coding improves when the coder also sees a structured, patient-level knowledge graph extracted from the clinical note, not just the raw text. On the paper's own benchmarks, this graph branch raises Macro-F1 from 9.69 to 11.05 on the full label set and from 64.61 to 67.81 on the top-50 subset, while reaching strong performance in fewer training epochs. The graph—built from 14 entity types and five relation families covering diagnoses, timing, drug regimens, and anatomy—compresses each note to 23% of its original length while preserving about 90% of its information as measured by entropy. The paper reads these gains as evidence that structured input representation is a real and underused lever in this long-tailed classification task, complementary to the usual strategy of enriching code representations only. A fair reader should care because the result points to a concrete, reusable intermediate representation for clinical text: smaller than the note, informative, and able to produce more focused attention-based explanations.","feed_headline":"Patient-level graphs lift ICD coding F1 by 3.2 points","feed_subtitle":"Structured triples hold 90% of the note's information in 23% of the words and sharpen code prediction","key_machinery":"The load-bearing object is the patient-level knowledge graph: a document-level graph assembled from 14 entity types—problems, treatments, tests, drugs, dosages, frequencies, routes, forms, body parts, symptoms, and related categories—connected by five relation families: clinical, temporal, posology, bodypart-direction, and bodypart-problem. Entities and relations are extracted by off-the-shelf clinical named-entity recognition and relation extraction models, then serialized as triples. A deep graph convolutional network (DGCNN) embeds the graph, with node features concatenated across layers, and these graph embeddings are fused with text segment embeddings from a biomedical pre-trained trans","core_discovery":"On the paper's own terms, the central discovery is that a patient-level knowledge graph is a useful patient representation for ICD-9 coding: it retains most of the note's information in a small fraction of its tokens, and injecting that graph into a transformer-based coder improves Macro-F1 over the text-only baseline by 1.36 points on the full label set and 3.20 points on the top-50 subset. The gains are not uniform across graph components. Ablations show that the 'clinical relationship' relation and the 'problem' entity type carry the most coding signal, while body-part-direction relations contribute least. The authors interpret these patterns as evidence that structured, entity-and-relati","pith_inferences":["My inference: the same graph-construction and fusion recipe would likely transfer to other clinical coding settings, including ICD-10 and long clinical documents beyond discharge summaries, because the entity and relation types are largely vocabulary-agnostic; this transfer is not tested in the paper.","My inference: a matched-capacity text-only control or a control with randomly wired graphs would isolate whether the benchmark improvements come from the graph's structure or simply from the extra DGCNN parameters; without such a control, the causal attribution to structure remains open.","My inference: the entropy-based compression measure could be used as a routing signal—documents whose graphs lose more information than usual could be flagged for additional full-text processing, a practical design the paper does not explore.","My inference: the attention highlights shown in the case studies are visually more focused than the text-only baseline, but turning this into a trustworthy clinician-facing explanation would require expert evalution rather than visual inspection alone."],"forward_implications":["ICD coding accuracy improves on both the full and top-50 code sets, with the largest gains on frequent codes and on codes the text-only model misses entirely, such as anemia and personal history of tobacco use.","Training efficiency improves: the graph branch reaches competitive Macro-F1 within the first few epochs, which is practically useful when computational resources are constrained.","The ablation results give a prioritization signal for graph construction: clinical relationships and problem entities matter most, so future extraction pipelines can focus annotation or model effort on those components.","The graph offers a compact structured artifact—about a quarter of the original note—that can serve simultaneously as an input feature and as a source of attention-based evidence for predictions.","The paper's gains are demonstrated on top of its chosen baseline; the authors note that integrating structured input into more recent, stronger coding architectures could yield further improvements."],"supporting_citations":[{"why":"Provides the PLM-ICD architecture and text-only baseline onto which the graph branch is added; the paper's main gains are measured relative to this model.","marker":"Huang et al., 2022"},{"why":"Supplies the MIMIC-III discharge summaries and ICD-9 labels used in all experiments and ablations.","marker":"Johnson et al., 2016"},{"why":"Supplies the Healthcare NLP library whose named-entity recognition and relation extraction models produce the triples that make up the patient-level knowledge graph.","marker":"John Snow Labs, 2024"},{"why":"Supplies the DGCNN graph convolutional architecture used to embed the patient graph before fusion with text.","marker":"Zhang et al., 2018"},{"why":"Supplies RoBERTa-PM, the biomedical/clinical pre-trained language model used for text segment encoding and graph node initialization.","marker":"Lewis et al., 2020"},{"why":"Supplies the label-wise attention mechanism that both the text and graph branches use to produce label-specific patient representations.","marker":"Vu et al., 2020"},{"why":"Supplies the only prior patient-level medical graph for ICD coding that the paper compares against, motivating the wider entity and relation coverage.","marker":"Yuan et al., 2021"}],"fun_headline_variants":["Graph-based patient notes boost ICD coding F1 by 3.2","90% info kept in 23% text: patient KGs improve ICD coding","Patient graphs sharpen ICD coding, up to 3.2 F1","Explainable ICD coding: patient KGs lift F1 up to 3.2","Patient-level graphs retain 90% info in 23% text, aid ICD coding"],"cache_read_input_tokens":23936,"weakest_assumption_plain":"The observed F1 gains are caused by the structured content of the knowledge graph, not by the added DGCNN encoder and extra attention head; if a text-only model with equal added capacity matched the graph model's F1, the structured-information explanation would be unsupported.","fun_headline_variants_meta":{"raw":{"variants":["Graph-based patient notes boost ICD coding F1 by 3.2","90% info kept in 23% text: patient KGs improve ICD coding","Patient graphs sharpen ICD coding, up to 3.2 F1","Explainable ICD coding: patient KGs lift F1 up to 3.2","Patient-level graphs retain 90% info in 23% text, aid ICD coding"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00092,"raw_usage":{"total_tokens":3810,"prompt_tokens":796,"completion_tokens":3014,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":540,"completion_tokens_details":{"reasoning_tokens":2910}},"tokens_in":540,"tokens_out":3014,"duration_ms":20616,"temperature":1.0,"reasoning_tokens":2910,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T10:20:57.419366+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same model with a control graph built from randomly paired entities and random relation labels, keeping graph size and model capacity fixed. If Macro-F1 stays near 11.05 on the full label set—or the gap to the text-only baseline stays near 1.36—then the specific structure of the graph is not what drives the gain. The paper's own entity and relation ablations provide a direct partial check: removing the problem entity drops Macro-F1 to 9.77, so a random-graph control would fill in what remains unexplained.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the MIMIC-III discharge summaries and ICD-9 labels used in all experiments and ablations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Healthcare NLP library whose named-entity recognition and relation extraction models produce the triples that make up the patient-level knowledge graph."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies RoBERTa-PM, the biomedical/clinical pre-trained language model used for text segment encoding and graph node initialization."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the only prior patient-level medical graph for ICD coding that the paper compares against, motivating the wider entity and relation coverage."}],"review_version":1}