pith. sign in

arxiv: 2606.07530 · v1 · pith:RPZNNTUAnew · submitted 2026-04-21 · 💻 cs.CL

Finding New Connections between Concepts from Medline Database Incorporating Domain Knowledge

Yang Weikang , Chowdhury S.M. Mazharul Hoque , Jin Wei This is my paper

Pith reviewed 2026-07-05 09:52 UTC · model glm-5.2

classification 💻 cs.CL
keywords Literature-Based DiscoveryABC modelSwansonMetaMapMEDLINETF-IDFconcept linkingbiomedical text mining
0
0 comments X

The pith

Extended ABC model recovers 20 of 21 known medical linking terms

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper extends Swanson's ABC model for Literature-Based Discovery by applying it to the MEDLINE database using MetaMap for concept extraction and TF-IDF for concept scoring. The model takes two medical topics (A and C) as input, processes roughly 22 million article references through a multi-threaded pipeline, and identifies intermediate connecting concepts (B terms) that appear in the literature of both topics. The authors evaluate the system on three classic Swanson test pairs (fish oil/Raynaud's disease, migraine/magnesium, schizophrenia/phospholipase A2) and compare results against one prior model. The system recovered 4 of 5, 8 of 8, and 8 of 8 known linking terms respectively, while also surfacing additional candidate terms not found by the comparison model, such as Hemodynamic, Atherosclerosis, Insulin, and PGE2.

Core claim

The central contribution is a pipeline that combines MetaMap-based concept mapping with TF-IDF weighting to find intermediate linking concepts between two disjoint medical topics. The pipeline uses a three-level data structure (semantic type, concept, occurrence) that is simplified to a two-level structure for efficiency, and a multi-threaded architecture that communicates with the MetaMap server. Concepts that co-occur in the literature of both input topics A and C are identified as potential B terms, with their weights normalized within their semantic type. Tested on three established benchmark pairs, the model recovered 20 out of 21 known linking terms and produced additional novel terms.

What carries the argument

The pipeline uses MetaMap to map article titles and abstracts to UMLS concepts, builds a Semantic-Type-to-Concept (S2C) index, then scores concepts using TF-IDF where term frequency counts concept presence per sentence and inverse document frequency down-weights concepts appearing across many sentences. The final S2CW (Semantic-Type-to-Concept-to-Weight) output merges concept lists from topics A and C, and connecting B terms are those concepts present in both.

If this is right

  • The model could be applied to novel topic pairs where no known linking exists, generating hypotheses for biomedical researchers to investigate.
  • The multi-threaded architecture demonstrates that large-scale literature processing (22 million references) is feasible on commodity hardware, suggesting broader deployment is practical.
  • The additional terms discovered beyond the comparison model (e.g., PGE2 for schizophrenia/phospholipase A2) could serve as starting points for new hypothesis-driven research if validated by domain experts.
  • Extending to multi-hop chains (chain length greater than 1) would allow the model to find connections through intermediate concepts that are themselves linked by further intermediaries, potentially reaching more distant topic pairs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The three test pairs used are the most well-known benchmarks in LBD, meaning the system is evaluated on cases where answers are already known; performance on truly novel discoveries remains untested.
  • The absence of precision/recall metrics, statistical significance tests, and expert verification of newly discovered terms makes it difficult to assess whether the model's novel findings are genuinely meaningful connections or noise.
  • The comparison against a single prior model limits the ability to judge whether the extended pipeline represents a genuine improvement over existing approaches or simply recovers the same well-documented associations through a different route.
  • The model's reliance on title and abstract only (excluding full text) may miss connecting concepts that appear exclusively in article bodies, potentially limiting recall on topics where key relationships are discussed outside of abstracts.

Load-bearing premise

The paper assumes that recovering known linking terms from three decades-old, heavily studied Swanson benchmark pairs is sufficient to demonstrate the model's discovery capability, without providing precision/recall metrics, statistical significance tests, expert verification of novel findings, or comparison against more than one baseline model.

What would settle it

If the model were tested on topic pairs with no previously known linking terms and the novel connecting concepts it produced were independently judged by domain experts to be spurious or trivial, the claim of meaningful discovery would be undermined.

Figures

Figures reproduced from arXiv: 2606.07530 by Chowdhury S.M. Mazharul Hoque, Jin Wei, Yang Weikang.

Figure 1
Figure 1. Figure 1: Data flow diagram [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Sample XML file containing publication date, article ID, title and abstract [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Code snippet for connecting MetaMap server and index creation [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Sample MetaMapped document [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Sample S2C document [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: A simplified Medline document [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: GUI for the model (showing input boxes for topics A and B, and their concepts, including the outcome C) [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Page to selecting S2C documents for processing [PITH_FULL_IMAGE:figures/full_fig_p009_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: GUI showing all the relevant sentences [PITH_FULL_IMAGE:figures/full_fig_p009_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: GUI presents sentences with intermediate linking concepts [PITH_FULL_IMAGE:figures/full_fig_p010_10.png] view at source ↗
read the original abstract

In this digital world, data is everything and significantly impacts our everyday lives. Interestingly, in this small world, everything is part of an ecosystem, where everything is connected, directly or indirectly. The same thing happens to data as well. In most cases, it may seem like a particular topic does not have any connection with another one, but in reality, they are connected through a mutually related topic. Therefore, in this research, we will discuss an adaptive model modified from the ABC model by Don R. Swanson, a Literature-Based Discovery (LBD) Model, to find the hidden connections between Concepts of Interest. The model demonstrates that two topics, A and C are different and have no relationship. But they have a common topic, B that can be used to connect topics A and C This famous model will be used in this discussion to connect Medical Concepts.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 7 minor

Summary. The manuscript presents an extended version of Swanson's ABC model for Literature-Based Discovery (LBD) applied to the MEDLINE database. The system uses MetaMap for concept extraction and TF-IDF weighting to identify intermediate linking terms (B concepts) between two input topics (A and C). The architecture follows a pipe-and-filter design with multithreaded parallel processing. Evaluation is conducted on three classic Swanson LBD benchmark pairs (fish oil/Raynaud's, migraine/magnesium, schizophrenia/phospholipase A2), comparing against one prior model [16].

Significance. The work tackles a well-established problem in biomedical LBD and the engineering effort of processing ~22M MEDLINE citations with MetaMap is non-trivial. The multithreading performance analysis (Table 1) provides useful practical data. However, the evaluation methodology is the primary weakness: the system is tested only on the three most well-known LBD benchmark pairs, where the ground-truth linking terms were already established by the ABC model the paper extends. No precision@k, recall, MRR, or statistical significance is reported. The system returns all co-occurring concepts with TF-IDF weights and no retrieval cutoff, making the 'recovery' metric uninterpretable—rank 56 (Table 4, Phenothiazines) is counted the same as rank 1 (Arachidonic acid). Without knowing the total number of concepts returned per semantic type per query, these ranks could be consistent with a system returning thousands of mostly irrelevant terms. This is the load-bearing gap for the central claim of finding 'meaningful connecting concepts.'

major comments (3)
  1. §4.4, Tables 2–4: The evaluation counts a known linking term as 'found' regardless of its rank within its semantic type. In Table 4, 'Phenothiazines' at rank 56 is scored as a success equivalent to 'Arachidonic acid' at rank 1. The paper does not report the total number of concepts returned per semantic type per query, so these ranks are uninterpretable. Without a retrieval cutoff or precision@k metric, the recovery rates (4/5, 8/8, 8/8) could be consistent with a system that returns all co-occurring terms. This is load-bearing for the central claim.
  2. §4.4: The evaluation uses only three classic Swanson benchmark pairs, all of which are the most well-known LBD examples with known answers established by the very ABC model the paper extends. This creates a partial circularity: the system is tested on cases where the linking terms were already discovered by the approach it builds upon. No evaluation on novel discovery pairs with expert verification is included, and no statistical significance or baseline comparison beyond one model [16] is reported. This does not demonstrate discovery capability beyond reproducing known results.
  3. §4.4, Tables 2–4: The comparison with [16] is only qualitative—listing whether known terms appear and their ranks. No precision, recall, F-measure, or other quantitative metric is computed for either system. The claim of 'notable improvements' (§1) over existing models is not supported by any metric.
minor comments (7)
  1. §4.2, Eq. (1): The IDF formula is rendered poorly and the base of the logarithm is not specified.
  2. §4.2, Eq. (3): The normalization formula divides by the maximum weight in the semantic type, but it is unclear whether this is per-query or global.
  3. Table 2: 'Blood viscosity' is marked 'False, but found something about blood pressure' with weight 0 and rank 0. The meaning of rank 0 is ambiguous.
  4. §4.4.1: The paper states the model found 'Hemodynamic' and 'Atherosclerosis' as new linking terms, but provides no evidence or expert verification that these are meaningful connections. If this is a claimed novel discovery, support should be cited.
  5. Figure 7 caption refers to 'topics A and B' but should likely be 'A and C' per the model description.
  6. Reference [2] spells 'Swason' instead of 'Swanson'; this typo should be corrected.
  7. §4.1: The FTP URL for MEDLINE data appears to contain a typo ('nlm' vs 'nlm').

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive review. The referee correctly identifies that our evaluation methodology is the primary weakness of the manuscript. We agree with the substance of all three major comments: (1) the recovery metric is uninterpretable without a retrieval cutoff or precision@k, (2) the three classic benchmark pairs create a partial circularity and do not demonstrate novel discovery, and (3) the comparison with [16] is only qualitative and the claim of 'notable improvements' is unsupported by quantitative metrics. We will revise the manuscript accordingly, adding precision@k metrics, reporting total concept counts per semantic type, tempering the 'notable improvements' claim, and acknowledging the evaluation limitations as threats to validity. We cannot, within the current revision cycle, add expert-validated novel discovery pairs, but we will add this as an explicit limitation and future work item.

read point-by-point responses

  1. Referee: §4.4, Tables 2–4: The evaluation counts a known linking term as 'found' regardless of its rank within its semantic type. In Table 4, 'Phenothiazines' at rank 56 is scored as a success equivalent to 'Arachidonic acid' at rank 1. The paper does not report the total number of concepts returned per semantic type per query, so these ranks are uninterpretable. Without a retrieval cutoff or precision@k metric, the recovery rates (4/5, 8/8, 8/8) could be consistent with a system that returns all co-occurring terms. This is load-bearing for the central claim.

    Authors: The referee is correct on all counts. Our current evaluation treats any rank as equivalent to rank 1, which is a genuine methodological flaw. A concept at rank 56 out of an unknown denominator is not meaningfully 'found' in the same sense as a concept at rank 1. We will revise the manuscript to address this in three ways. First, we will report the total number of concepts returned per semantic type per query in Tables 2–4, making the ranks interpretable. Second, we will add precision@k metrics (k = 5, 10, 20) for each test set, so that the reader can assess whether known linking terms appear near the top of the ranked list rather than at arbitrary depths. Third, we will add a discussion of what constitutes a meaningful retrieval cutoff for LBD systems and acknowledge that without such a cutoff, the raw recovery rates overstate the system's discriminative power. We agree this is load-bearing for the central claim and the revision is necessary. revision: yes

  2. Referee: §4.4: The evaluation uses only three classic Swanson benchmark pairs, all of which are the most well-known LBD examples with known answers established by the very ABC model the paper extends. This creates a partial circularity: the system is tested on cases where the linking terms were already discovered by the approach it builds upon. No evaluation on novel discovery pairs with expert verification is included, and no statistical significance or baseline comparison beyond one model [16] is reported. This does not demonstrate discovery capability beyond reproducing known results.

    Authors: We agree with this assessment. The three benchmark pairs are the most well-known LBD examples, and testing on them does create a partial circularity since the ground-truth linking terms were established by the very ABC model we extend. We acknowledge that reproducing known results is necessary but not sufficient to demonstrate discovery capability. We will revise the manuscript to explicitly state this limitation in the evaluation section and frame the current results as reproduction validation rather than novel discovery. We will also temper the language in the abstract and introduction to accurately reflect that the system recovers known linking terms rather than claiming it finds new connections. Regarding statistical significance and additional baselines: with only three test pairs, statistical significance testing is not feasible, and we will state this honestly. Adding expert-validated novel discovery pairs is beyond the scope of the current revision cycle, but we will add this as a primary future work direction and discuss it as a threat to validity. The new linking terms our system found that were not in [16] (e.g., Hemodynamic, Atherosclerosis, Insulin, PGE2) will be recharacterized as candidate connections requiring expert verification rather than validated discoveries. revision: partial

  3. Referee: §4.4, Tables 2–4: The comparison with [16] is only qualitative—listing whether known terms appear and their ranks. No precision, recall, F-measure, or other quantitative metric is computed for either system. The claim of 'notable improvements' (§1) is not supported by any metric.

    Authors: The referee is correct. Our comparison with [16] is purely qualitative—we list whether known terms appear and their ranks, but compute no quantitative metric for either system. The claim of 'notable improvements' in §1 is not supported by any quantitative evidence. We will address this in two ways. First, we will remove or substantially temper the 'notable improvements' claim in the introduction, replacing it with an accurate description of what was observed: that our system recovered the same known linking terms as [16] using title and abstract text rather than MeSH terms, and additionally surfaced some candidate terms not found by [16]. Second, we will add quantitative metrics (precision@k, mean reciprocal rank) for both systems where comparable data is available. We note that a full F-measure comparison requires knowing the complete set of relevant linking terms for each pair, which is not well-defined beyond the terms originally reported by Swanson and [16]. We will be transparent about this limitation. revision: yes

Circularity Check

0 steps flagged

Minor self-citation in evaluation reference; no derivation-level circularity

full rationale

The paper's derivation chain is: (1) take input topics A and C, (2) process ~22M MEDLINE citations through MetaMap to extract concepts, (3) compute TF-IDF weights for concepts co-occurring with both A and C, (4) return ranked connecting concepts. This chain is not defined in terms of its evaluation targets. The ground-truth linking terms come from Gopalakrishnan et al. [16], which is a different model (graph-based on MeSH terms) from the present model (MetaMap + TF-IDF on titles/abstracts). Reference [16] includes Jin Wei as a co-author, who is also an author of the present paper, creating a self-citation. However, this self-citation is not load-bearing for the derivation: [16] provides evaluation targets (known linking terms) and a comparison baseline, not a premise or input to the present model's computation. The present model does not incorporate [16]'s outputs into its own algorithm. The evaluation weakness (no precision@k cutoff, testing on well-known Swanson examples, counting rank-56 hits the same as rank-1) is a legitimate methodological concern, but it is a correctness/evaluation rigor issue, not circularity. The model's definition of 'connecting concept' (co-occurrence with both A and C, weighted by TF-IDF) is not equivalent to 'terms found by [16]' by construction. No self-definitional, fitted-input-renamed-as-prediction, or uniqueness-theorem circularity is present.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

The paper introduces no new entities, particles, forces, or theoretical constructs. It uses existing tools (MetaMap, UMLS semantic types) and existing frameworks (Swanson ABC, TF-IDF). The free parameters are engineering choices, not theoretical constants. The axioms are standard domain assumptions from the LBD literature.

free parameters (2)
  • Number of threads (1, 2, or 3) = 3 (best performance)
    Chosen empirically based on hardware limitations; not a derived optimum.
  • Citation split size (10,000 per thread) = 10000
    Each Medline document's 30,000 citations divided into 3 parts of 10,000; chosen by hand.
axioms (4)
  • domain assumption Swanson's ABC hypothesis: if A relates to B and B relates to C, then A may relate to C
    Invoked in §1 and §2 as the foundational assumption; the entire model depends on this transitivity holding for biomedical concepts.
  • domain assumption MetaMap correctly maps biomedical text to UMLS concepts
    Invoked in §4.2; the system relies entirely on MetaMap's concept extraction being accurate.
  • domain assumption TF-IDF is an adequate scoring metric for ranking intermediate concepts
    Invoked in §4.2 (Eqs. 1-3); no justification is given for why TF-IDF is appropriate over other ranking methods.
  • domain assumption Title and abstract contain sufficient information for concept discovery
    Stated in §4.2: 'only the title and abstract are playing the main role here'; full text is excluded.

pith-pipeline@v1.1.0-glm · 12747 in / 2706 out tokens · 120142 ms · 2026-07-05T09:52:14.646468+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

16 extracted references · 16 canonical work pages

  1. [1]

    Mechanisms of migraine as a chronic evolutive condition

    Andreou AP , Edvinsson L. Mechanisms of migraine as a chronic evolutive condition. The Journal of Headache and Pain. 2019; 20:117 . DOI: 10.1186/s10194-019-1066-0

    work page doi:10.1186/s10194-019-1066-0 2019

  2. [2]

    Complementary structures in disjoint science literatures

    Swason DR. Complementary structures in disjoint science literatures. In: Proceedings of the 14th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. Chicago, IL: ACM Press; 1991. pp. 280-289

    work page 1991

  3. [3]

    Fish oil, Raynaud’ s syndrome, and undiscovered public knowledge

    Swanson DR. Fish oil, Raynaud’ s syndrome, and undiscovered public knowledge. Perspectives in Biology and Medicine. 1986; 30(1):7-18. DOI: 10.1353/ pbm.1986.0087

    work page arXiv 1986

  4. [4]

    Rediscovering Don Swanson: The past, present and future of literature-based discovery

    Smalheiser NR. Rediscovering Don Swanson: The past, present and future of literature-based discovery. Journal of Data and Information Science (Warsaw , Poland). 2017; 2(4):43. DOI: 10.1515/ jdis-2017-0019

    work page 2017

  5. [5]

    A graph-based recovery and decomposition of Swanson’ s hypothesis using semantic predications

    Cameron D, Bodenreider O, Hima Yalamanchili T , Danh SV , Thirunarayan K, Sheth AP , et al. A graph-based recovery and decomposition of Swanson’ s hypothesis using semantic predications. Journal of Biomedical Informatics. 2013; 46(2):238-251, ISSN 1532-0464. DOI: 10.1016/j. jbi.2012.09.004

    work page doi:10.1016/j 2013

  6. [6]

    Exploiting semantic relations for literature-based discovery

    Hristovski D, Friedman C, Rindflesch TC, Peterlin B. Exploiting semantic relations for literature-based discovery. In: AMIA Annual Symposium Proceedings. AMIA Symposium. National Library of Medicine; 2006. p. 349-53

    work page 2006

  7. [7]

    Literature-based knowledge discovery using natural language processing

    Hristovski D, Friedman C, Rindflesch TC, Peterlin B. Literature-based knowledge discovery using natural language processing. In: Bruza P , Weeber M, editors. Literature- Based Discovery, Information Science and Knowledge Management. V ol. 15. Berlin, Heidelberg: Springer; 2008. pp. 133-152

    work page 2008

  8. [8]

    The interaction of domain knowledge and linguistic structure in natural language processing: Interpreting hypernymic propositions in biomedical text

    Rindflesch TC, Fiszman M. The interaction of domain knowledge and linguistic structure in natural language processing: Interpreting hypernymic propositions in biomedical text. Journal of Biomedical Informatics. 2003; 36(6):462-477

    work page 2003

  9. [9]

    Phenogo: Assigning phenotypic context to gene ontology annotations with natural language processing

    Lussier Y , Borlawsky T , Rappaport D, Liu Y , Friedman C. Phenogo: Assigning phenotypic context to gene ontology annotations with natural language processing. In: Pacific Symposium on Biocomputing. National Library of Medicine; 2006. p. 64-75

    work page 2006

  10. [10]

    Literature-based discovery beyond the ABC paradigm: A contrastive approach

    Moreau E, Hardiman O, Heverin M, O’Sullivan D. Literature-based discovery beyond the ABC paradigm: A contrastive approach. BioRxiv . 2021; 09(22):461375. DOI: 10.1101/2021.09.22.461375

    work page doi:10.1101/2021.09.22.461375 2021

  11. [11]

    Literature based discovery: Models, methods, and trends

    Henry S, McInnes BT . Literature based discovery: Models, methods, and trends. Journal of Biomedical Informatics. 2017; 74:20-32, ISSN 1532-0464,. DOI: 10.1016/j.jbi.2017 .08.011

    work page doi:10.1016/j.jbi.2017 2017

  12. [12]

    Discovering context-specific relationships from biological literature by using multi-level context terms

    Lee S, Choi J, Park K, et al. Discovering context-specific relationships from biological literature by using multi-level context terms. BMC Medical Informatics and Decision Making. 2012; 12(Suppl 1):S1. DOI: 10.1186/1472-6947-12-S1-S1

    work page doi:10.1186/1472-6947-12-s1-s1 2012

  13. [13]

    A context-based ABC model for Research Advances in Data Mining Techniques and Applications 16 literature-based discovery

    Kim YH, Song M. A context-based ABC model for Research Advances in Data Mining Techniques and Applications 16 literature-based discovery. PLoS One. 2019; 14(4):e0215313. DOI: 10.1371/ journal.pone.0215313

    work page 2019

  14. [14]

    Effective mapping of biomedical text to the UMLS Metathesaurus: The MetaMap program

    Aronson AR. Effective mapping of biomedical text to the UMLS Metathesaurus: The MetaMap program. In: AMIA Annual Symposium Proceedings. AMIA Symposium. National Library of Medicine; 2001. p. 17-21. DOI: D010001275

    work page 2001

  15. [15]

    National Library of Medicine

    U.S. National Library of Medicine. MetaMap - A Tool for Recognizing UMLS Concepts in Text. 2017 . Available from: https://metamap.nlm.nih.gov/

    work page 2017

  16. [16]

    Generating hypothesis: Using global and local features in graph to discover new knowledge from medical literature

    Gopalakrishnan V , Jha K, Zhang A, Jin W . Generating hypothesis: Using global and local features in graph to discover new knowledge from medical literature. In: Proceedings of the 8th International Conference on Bioinformatics and Computational Biology, BICOB. USA; 2016. pp. 23-30

    work page 2016