Pith. sign in

REVIEW 2 major objections 4 minor 2 cited by

Leveraging Large Language Models for Generating Research Topic Ontologies: A Multi-Disciplinary Study

T0 review · 2 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuned large language models can classify research-topic relationships at 93.5% F1, the paper reports.

desk verdict Useful benchmark and solid fine-tuning study, but the 'other' category is likely contaminated and the lack of baselines/variance keeps me from trusting the headline F1. read the letter →

arxiv 2508.20693 v2 pith:QSBVPNBJ submitted 2025-08-28 cs.DL cs.CL

Tanay Aggarwal , Angelo Salatino , Francesco Osborne , Enrico Motta This is my paper
classification cs.DLcs.CL
keywords researchtopicontologiessemanticrelationclassificationlargelanguagemodelsfine-tuningPEM-Rel-8Kknowledgeorganizationsystemscross-domaintransferbiomedicinephysicsengineering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that fine-tuning open-weight large language models on a few thousand existing research-topic relationships teaches them to classify new topic pairs as broader, narrower, same-as, or unrelated, and that this skill transfers across scientific disciplines. To test this, it introduces PEM-Rel-8K, a benchmark of 8,075 labeled topic pairs sampled from the IEEE, PhySH, and MeSH taxonomies, and runs 288 experiments across 12 models with zero-shot, chain-of-thought, and fine-tuning settings. Fine-tuning decisively beats the prompting baselines; the best model, gemma-2-27b-it, reaches 93.5% F1 on the combined test set. A model fine-tuned on all three disciplines together lands only 1.2% F1 below models fine-tuned separately per discipline, and cross-domain transfer outperforms the best zero-shot strategy by roughly 16 points. If this holds, LLM classifiers could automate or accelerate the construction, expansion, and interlinking of research-field ontologies, which are currently costly to maintain by hand.

What carries the argument

PEM-Rel-8K: a modular benchmark of 8,075 topic pairs drawn from IEEE (engineering), PhySH (physics), and MeSH (biomedicine), each labeled broader, narrower, same-as, or other. It carries the argument by providing both the training signal and the evaluation yardstick for every experimental setting, and its modular split into three single-discipline subsets is what makes the cross-domain transfer and multidisciplinary comparisons possible.

What would settle it

Have independent annotators re-examine a random sample of the 'other' pairs in PEM-Rel-8K, judging whether any broader, narrower, or same-as relation holds via indirect, multi-hop, or lexical evidence. If more than a small fraction of those pairs are judged related, the benchmark's negative class is contaminated and the 93.5% F1 overstates the models' true discrimination. Alternatively, test a fine-tuned model on 'other' pairs sampled from recent literature in the three disciplines rather than from random ontology pairings and compare the F1 drop.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the semantic relation between two research-topic names—one is broader, one is narrower, they are synonyms, or they are unrelated—can be learned by fine-tuning instruction-tuned LLMs on relations sampled from existing ontologies, and that this learning generalizes across disciplines. The authors construct PEM-Rel-8K from 8,075 topic pairs labeled with four classes, fine-tune twelve open-weight models using low-rank adaptation, and report that the best model reaches 93.5% F1 on the combined test set. They also find that the best multidisciplinary model loses only 1.2% F1 relative to discipline-specific models, and that models fine-tuned o

Load-bearing premise

The load-bearing premise is that pairs of topics with no direct link in the source ontology are genuinely unrelated; if many random pairs in the 'other' class are actually indirectly related, the negative labels are wrong and the measured F1 is inflated.

Editorial extensions

If this is right

  • Fine-tuned classifiers could propose candidate broader, narrower, or same-as links for extending existing taxonomies, leaving only a small set for human review.
  • A single model fine-tuned on all available disciplines can serve multiple fields at once, reducing the need for per-discipline annotation effort.
  • Because cross-domain transfer works, data-rich fields such as biomedicine could be used to bootstrap ontologies in data-poor fields.
  • The consistent same-as versus hierarchy confusion identifies a concrete target for improvement, such as providing definitions or context for topic names.
  • The released benchmark and code make the 24 train/test configurations reproducible and directly reusable by other teams.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 'other' class is built from random pairs with no direct link in the source ontology; this likely under-represents hard negatives such as terms sharing a word or connected through a multi-hop path, so real-world deployment on literature indexing may show lower accuracy than the reported 93.5%.
  • The cross-domain transfer result hints that models learn a general notion of subtopic rather than memorizing a specific taxonomy; a natural test is whether such models can propose links between topics from different ontologies, a capability the paper does not directly evaluate.
  • Supplying each topic's definition or thesaurus context during fine-tuning could reduce the lexical-overlap errors behind the same-as confusion; this is a testable extension the paper leaves open.
  • Because the benchmark is modular, adding a fourth discipline would directly test whether the claimed generalization extends beyond closely related STEM fields.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. This paper introduces PEM-Rel-8K, a dataset of 8,075 topic-pair instances sampled from IEEE, PhySH, and MeSH, labelled as broader, narrower, same-as, or other. The authors evaluate 12 open-weight LLMs under zero-shot prompting, bidirectional chain-of-thought prompting, and LoRA fine-tuning across 24 train/test configurations. The central empirical claims are that fine-tuning vastly outperforms prompting, that a model fine-tuned on the combined benchmark loses only about 1.2% average F1 relative to domain-specific fine-tuned models, and that cross-domain transfer is effective. The best result is 93.5% F1 for gemma-27b on the full benchmark.

Significance. If the benchmark labels are valid, PEM-Rel-8K is a timely and potentially reusable resource for ontology construction, and the paper's systematic sweep over models, prompting strategies, and cross-domain settings is a genuine strength. The authors release the dataset and code, which supports reproducibility. The evaluation is not circular: labels come from external ontologies and fine-tuning results are independent of the authors' prior work. However, the central quantitative claim is load-bearing on the validity of the 'other' class and of the same-as annotations. The stress-test concern about negative examples lands: the construction in Section 3.2 does not rule out transitive or lexical semantic relations among 'other' pairs, and the paper itself provides examples where such pairs are semantically related. The benchmark therefore needs re-construction or a contamination audit before the headline F1 can be accepted.

major comments (2)
  1. [Section 3.2, cf. Section 3.1 and Table 4] The task definition in Section 3.1 treats broader/narrower as subsumption, which is transitive, but the 'other' pairs for IEEE and PhySH are generated by requiring only the absence of a direct ontology link, and the MeSH 'unrelated' criterion is unspecified. No transitive-closure, altLabel, or lexical-overlap filtering is reported. A random ancestor-descendant pair at distance >=2 can be labelled 'other' even though it satisfies the paper's own definition of broader/narrower, and examples such as 'Nanotechnology' vs 'Nanoscale technology' (Section 5.4) are plausibly related pairs that may fall into either class. Since 'other' is the best-performing class (average F1 94.3%, Table 4) and contributes to the headline 93.5% F1, mislabeled negatives would inflate the main claim. Please rebuild the negative set with transitive-closure and lexical checks, or provide a post-hoc contamination anal
  2. [Section 3.2, cf. Section 5.4] The same-as labels are produced by experts inspecting skos:altLabel for IEEE and PhySH and mesh:relatedConcept for MeSH. The paper's own example of '4G mobile communication' as an altLabel of '5G mobile communication' illustrates that altLabel can encode relatedness rather than equivalence, and mesh:relatedConcept is a broad related-concept property rather than a synonymy property. Allowing 'near-synonyms' as same-as also conflicts with the task definition's interchangeability across IR tasks. Without inter-annotator agreement statistics and a tighter restriction of same-as to strict synonymy or clearly defined lexical variants, the same-as class (average F1 88.5%, Table 4) is difficult to interpret, and the error analysis in Section 5.4 may partly reflect label noise. Please report annotation quality and either re-annotate or explicitly restrict the label definition.
minor comments (4)
  1. [Table 1] The row label 'PhySH-Rel-87' appears to be a typo for 'PhySH-Rel-875'.
  2. [Section 4.3] Beyond LoRA r and alpha, no training hyperparameters are reported (epochs, learning rate, batch size, max sequence length), and inference parameters such as temperature and max tokens are also omitted. The code release helps, but the main text should state these for reproducibility.
  3. [Table 4] The 'AVG' rows should explicitly state whether the average is macro or micro. Given the class imbalance in PhySH-Rel-875, this distinction matters for interpreting the per-relation averages.
  4. [Section 5.2.2] Cross-domain claims would be stronger with significance testing or variance estimates. Several reported differences, e.g., 94.7 vs. 94.5 on IEEE, are within plausible run-to-run noise; adding repeated-seed results or confidence intervals would help.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the empirical evaluation is self-contained, and self-citations are background or method references that do not carry the central claim.

full rationale

The paper's central claim is empirical: fine-tuning LLMs on the new PEM-Rel-8K benchmark yields strong F1 scores. The benchmark labels are sampled from external, expert-maintained ontologies (MeSH, PhySH, IEEE), not generated by the models or derived from any fitted parameter. Training and test splits are disjoint (7:1:2), and the evaluation is standard supervised classification, so test predictions are not equivalent to training inputs by construction. Self-citations appear in two places: [3] for the bidirectional CoT prompting method and [45] for a survey of knowledge organization systems. Both are used as background or as a comparison baseline, not as premises that force the headline result. The bCoT method is an externally published, falsifiable prompting strategy; the survey provides context about the ontology landscape. Neither is load-bearing for the fine-tuning performance claim. The construction of 'other' negatives by sampling pairs without a direct ontology edge is a possible data-quality concern (transitive or lexical relations could be mislabeled), but this is not circularity: the labels are not defined by the model's predictions, and the reported F1 does not reduce to the negative-sampling procedure by definition. The paper is self-contained against external ontology resources, and no derivation chain collapses into its own inputs.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The central claim is empirical and rests on the quality of the benchmark labels and the chosen experimental setup. The main unverified premise is that the constructed negative examples are truly unrelated, and that the expert annotations and source ontologies are accurate. The LoRA hyperparameters and sampling choices are hand-set but not the result of a formal derivation.

free parameters (4)
  • LoRA rank r = 16 (Mistral, Zephyr) or 256 (Llama, Gemma, Phi)
    Chosen by the authors for fine-tuning; not fitted to target data but controls adapter capacity and can influence results.
  • LoRA alpha = 16 (Mistral, Zephyr) or 128 (Llama, Gemma, Phi)
    Scaling factor for LoRA updates; a hand-set hyperparameter, not fitted.
  • Train/validation/test split ratio = 7:1:2
    Applied to all datasets; standard but arbitrary and affects evaluation variance.
  • Per-class sample sizes = IEEE 800/800/800/800; PhySH 250/250/125/250; MeSH 1000/1000/1000/1000 (broader/narrower/same-as/other)
    Sampling choices made by the authors; they determine class balance and task difficulty.
assumptions (4)
  • domain assumption MeSH, PhySH, and IEEE Thesaurus relations are correct ground truth for broader, narrower, and same-as.
    The benchmark labels come directly from these ontologies (Section 3.2); any errors in the source ontologies propagate to the benchmark.
  • ad hoc to paper Randomly sampled topic pairs with no direct link in the source ontology are semantically unrelated (other).
    Section 3.2: 'randomly generated 800 topic pairs that do not share any semantic link in the original ontology.' Indirect or multi-hop relations are not checked.
  • domain assumption Expert manual annotations of same-as relations are accurate and consistent.
    For IEEE and PhySH, three experts manually validated same-as pairs from skos:altLabel (Section 3.2); no inter-annotator agreement is reported.
  • domain assumption 4-bit quantization and LoRA fine-tuning preserve enough model capability for the task.
    All models were quantised to 4-bit and fine-tuned with LoRA (Sections 3.3 and 4.2); the paper takes this as given rather than testing it.
invented entities (1)
  • PEM-Rel-8K benchmark (with subsets IEEE-Rel-3K, PhySH-Rel-875, MeSH-Rel-4K) independent evidence
    purpose: Training and evaluation dataset for classifying semantic relations between research topics across engineering, physics, and biomedicine.
    The dataset is publicly released on GitHub, giving independent researchers a concrete artifact to use and falsify; no physical entity is postulated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Leveraging Large Language Models for Generating Research Topic Ontologies: A Multi-Disciplinary Study." pith.science (2026). https://pith.science/paper/QSBVPNBJ

@misc{pith2026250820693,
  author       = {Pith},
  title        = {Pith review of: Leveraging Large Language Models for Generating Research Topic Ontologies: A Multi-Disciplinary Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QSBVPNBJ}},
  note         = {Machine review of arXiv:2508.20693}
}
read the original abstract

Ontologies and taxonomies of research fields are critical for managing and organising scientific knowledge, as they facilitate efficient classification, dissemination and retrieval of information. However, the creation and maintenance of such ontologies are expensive and time-consuming tasks, usually requiring the coordinated effort of multiple domain experts. Consequently, ontologies in this space often exhibit uneven coverage across different disciplines, limited inter-discipline connectivity, and infrequent updating cycles. In this study, we investigate the capability of several large language models to identify semantic relationships among research topics within three academic disciplines: biomedicine, physics, and engineering. The models were evaluated under three distinct conditions: zero-shot prompting, chain-of-thought prompting, and fine-tuning on existing ontologies. Additionally, we assessed the cross-discipline transferability of fine-tuned models by measuring their performance when trained in one discipline and subsequently applied to a different one. To support this analysis, we introduce PEM-Rel-8K, a novel dataset consisting of over 8,000 relationships extracted from the most widely adopted taxonomies in the three disciplines considered in this study: MeSH, PhySH, and IEEE. Our experiments demonstrate that fine-tuning LLMs on PEM-Rel-8K yields excellent performance across all disciplines.

Figures

Figures reproduced from arXiv: 2508.20693 by the authors.

Figure 1
Figure 1. Confusion matrices for best-performing LLMs on [PITH_FULL_IMAGE:figures/full_fig_p013_1.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. OntoExtend: A Framework for Requirement-driven and Scalable Ontology Extension with LLMs

    cs.AI 2026-07 conditional novelty 6.0 of 10

    A retrieval-augmented LLM framework extends existing ontologies from competency questions, generating fragments that pass structural and functional checks and are judged usable as drafting aids by ontology engineers.

  2. Research Entity Extraction and Topic Detection from UKRI Grant Proposals

    cs.DL 2026-06 unverdicted novelty 3.0 of 10

    Mistral LLM pipeline extracts entities from 42 grant abstracts with 90.5% topic classification accuracy, outperforming a bespoke DSIT-Taxonomies system at 71.4%.

Reference graph

Works this paper leans on

61 extracted references · 44 canonical work pages · cited by 2 Pith papers

  1. [1]

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al . 2024. Phi-3 technical report: A highly capable language model locally on your phone, 2024. URL https://arxiv. org/abs/2404.14219 (2024)

    arXiv 2024

  2. [2]

    Hewett, Mojan Javaheripi, Piero Kauffmann, James R

    Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Xin Wang, Rachel Ward, Yue Wu, Dingli Yu,...

    arXiv 2024

  3. [3]

    Tanay Aggarwal, Angelo Salatino, Francesco Osborne, and Enrico Motta. 2026. Large language models for scholarly ontology generation: An extensive analysis in the engineering field. Information Processing & Management 63, 1 (2026), 104262

    work page 2026

  4. [4]

    AI@Meta. 2024. Llama 3 Model Card. (2024). https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md

    work page 2024

  5. [5]

    Simone Angioni, Angelo Salatino, Francesco Osborne, Diego Reforgiato Recupero, and Enrico Motta. 2021. AIDA: A knowledge graph about research dynamics in academia and industry. Quantitative Science Studies 2, 4 (2021), LLMs for Generating Research Topic Ontologies 1356–1398

    work page 2021

  6. [6]

    Sören Auer, Dante AC Barone, Cassiano Bartz, Eduardo G Cortes, Mohamad Yaser Jaradeh, Oliver Karras, Manolis Koubarakis, Dmitry Mouromtsev, Dmitrii Pliukhin, Daniil Radyush, et al. 2023. The sciqa scientific question answering benchmark for scholarly knowledge. Scientific Reports 13, 1 (2023), 7240

    work page 2023

  7. [7]

    Hamed Babaei Giglou, Jennifer D’Souza, and Sören Auer. 2023. LLMs4OL: Large language models for ontology learning. In International Semantic Web Conference. Springer, 408–427

    work page 2023

  8. [8]

    Joeran Beel, Bela Gipp, Stefan Langer, and Corinna Breitinger. 2016. Paper recommender systems: a literature survey. International Journal on Digital Libraries 17 (2016), 305–338

    work page 2016

Show all 61 references
  1. [9]

    Francisco Bolanos, Angelo Salatino, Francesco Osborne, and Enrico Motta. 2024. Artificial intelligence for literature reviews: Opportunities and challenges. Artificial Intelligence Review 57, 10 (2024), 259

    2024

  2. [10]

    Davide Buscaldi, Danilo Dessí, Enrico Motta, Marco Murgia, Francesco Osborne, and Diego Reforgiato Recupero. 2024. Citation prediction by leveraging transformers and natural language processing heuristics. Information Processing & Management 61, 1 (2024), 103583

    2024

  3. [11]

    Catherine Chen, Kevin Lin, and Dan Klein. 2020. Constructing taxonomies from pretrained language models. arXiv preprint arXiv:2010.12813 (2020)

    2020 arXiv

  4. [12]

    Sabrina Chow, Lilian Guo, Jonathan Chow, Chelsea Chia, Sarah Li, and Dong-Yan Huang. 2024. Semantic Search Using LLM-Aided Topic Generation on Knowledge Graphs for Paper Discovery. In2024 IEEE 14th International Symposium on Chinese Spoken Language Processing (ISCSLP) . IEEE, 353–357

    2024

  5. [13]

    Philipp Cimiano and Johanna Völker. 2005. Text2onto: A framework for ontology learning and data-driven change discovery. In International conference on application of natural language to information systems . Springer, 227–238

    2005

  6. [14]

    Michael Han Daniel Han and Unsloth team. 2023. Unsloth. http://github.com/unslothai/unsloth

    2023

  7. [15]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

    2018 arXiv

  8. [16]

    Edward Dunne and Klaus Hulek. 2020. Mathematics Subject Classification 2020. EMS Newsletter 2020–3, 115 (March 2020), 5–6. doi:10.4171/news/115/2

    2020 doi

  9. [17]

    Nadeen Fathallah, Arunav Das, Stefano De Giorgis, Andrea Poltronieri, Peter Haase, and Liubov Kovriguina. 2024. Neon-GPT: a large language model-powered pipeline for ontology learning. In European Semantic Web Conference. Springer, 36–50

    2024

  10. [18]

    Nadeen Fathallah, Steffen Staab, and Alsayed Algergawy. 2024. LLMs4Life: Large Language Models for Ontology Learning in Life Sciences. arXiv:2412.02035 [cs.AI] https://arxiv.org/abs/2412.02035

    2024 arXiv

  11. [19]

    Maarten Grootendorst. 2022. BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv preprint arXiv:2203.05794 (2022)

    2022 arXiv

  12. [20]

    Michael Gusenbauer and Neal R Haddaway. 2020. Which academic search systems are suitable for systematic reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources. Research synthesis methods 11, 2 (2020), 181–217

    2020

  13. [21]

    Daniel Han. 2024. Unsloth/zephyr-sft · hugging face. https://huggingface.co/unsloth/zephyr-sft

    2024

  14. [22]

    Kanyao Han, Pingjing Yang, Shubhanshu Mishra, and Jana Diesner. 2020. WikiCSSH: extracting computer science subject headings from Wikipedia. In ADBIS, TPDL and EDA 2020 Common Workshops and Doctoral Consortium: International Workshops: DOING, MADEISD, SKG, BBIGAP, SIMPDA, AIMi...

    2020

  15. [23]

    Qianyue Hao, Jingyang Fan, Fengli Xu, Jian Yuan, and Yong Li. 2024. Hlm-cite: Hybrid language model workflow for text-based scientific citation prediction. Advances in Neural Information Processing Systems 37 (2024), 48189–48223

    2024

  16. [24]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

  17. [25]

    Gard B Jenset, Peter J Bevan, and Akarsh Jain. 2025. A large-scale, granular topic classification system for scientific documents. PREPRINT (Version 1) available at Research Square (2025). doi:10.21203/rs.3.rs-6529718/v1

    2025 doi

  18. [26]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7B. arXiv preprint arXiv:2310.06825 (2023)

    2023 arXiv

  19. [27]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2023. Large Language Models are Zero-Shot Reasoners. arXiv:2205.11916 [cs.CL] https://arxiv.org/abs/2205.11916

    2023 arXiv

  20. [28]

    Konstantinos I Kotis, George A Vouros, and Dimitris Spiliotopoulos. 2020. Ontology engineering methodologies for the evolution of living and reused ontologies: status, trends, findings and recommendations. The Knowledge Engineering Review 35 (2020), e4

    2020

  21. [29]

    Anna Sofia Lippolis, Mohammad Javad Saeedizade, Robin Keskisärkkä, Sara Zuppiroli, Miguel Ceriani, Aldo Gangemi, Eva Blomqvist, and Andrea Giovanni Nuzzolese. 2025. Ontology generation using large language models. arXiv preprint arXiv:2503.05388 (2025). Aggarwal et al

    2025 arXiv

  22. [30]

    Carolyn E Lipscomb. 2000. Medical subject headings (MeSH). Bulletin of the Medical Library Association 88, 3 (2000), 265

    2000

  23. [31]

    Alexander Maedche and Steffen Staab. 2001. Learning Ontologies for the Semantic Web.. In SemWeb

    2001

  24. [32]

    Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, Sayak Paul, and Benjamin Bossan. 2022. PEFT: State-of-the-art Parameter-Efficient Fine-Tuning methods. https://github.com/huggingface/peft

    2022

  25. [33]

    Antonello Meloni, Simone Angioni, Angelo Salatino, Francesco Osborne, Diego Reforgiato Recupero, and Enrico Motta

  26. [34]

    Antonello Meloni, Diego Reforgiato Recupero, Francesco Osborne, angelo salatino, Enrico Motta, Sahar Vahadati, and Jens Lehmann. 2025. Exploring Large Language Models for Scientific Question Answering via Natural Language to SPARQL Translation. ACM Transactions on Intelligent ...

    2025

  27. [35]

    Roberto Navigli, Paola Velardi, Alessandro Cucchiarelli, and Francesca Neri. 2004. Quantitative and Qualitative Evaluation of the OntoLearn Ontology Learning System. In COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics. COLING, Geneva, ...

    2004

  28. [36]

    OpenAlex. 2024. OpenAlex: End-to-End Process for Topic Classification

    2024

  29. [37]

    Francesco Osborne and Enrico Motta. 2012. Mining semantic relations between research areas. In The Semantic Web–ISWC 2012: 11th International Semantic Web Conference, Boston, MA, USA, November 11-15, 2012, Proceedings, Part I 11. Springer, 410–426

    2012

  30. [38]

    Francesco Osborne and Enrico Motta. 2015. Klink-2: integrating multiple web sources to generate semantic topic networks. In The Semantic Web-ISWC 2015: 14th International Semantic Web Conference, Bethlehem, PA, USA, October 11-15, 2015, Proceedings, Part I 14 . Springer, 408–424

    2015

  31. [39]

    Francesco Osborne and Enrico Motta. 2018. Pragmatic ontology evolution: reconciling user requirements and applica- tion performance. In International Semantic Web Conference. Springer, 495–512

    2018

  32. [40]

    Francesco Osborne, Enrico Motta, and Paul Mulholland. 2013. Exploring Scholarly Data with Rexplore. InThe Semantic Web – ISWC 2013, Harith Alani, Lalana Kagal, Achille Fokoue, Paul Groth, Chris Biemann, Josiane Xavier Parreira, Lora Aroyo, Natasha Noy, Chris Welty, and Krzyszt...

    2013

  33. [41]

    A Pisu, L Pompianu, A Salatino, F Osborne, D Riboni, E Motta, D Reforgiato Recupero, et al. 2024. Leveraging Language Models for Generating Ontologies of Research Topics. In CEUR WORKSHOP PROCEEDINGS, Vol. 3747. CEUR-WS, 11

    2024

  34. [42]

    Behnam Rahdari, Peter Brusilovsky, and Alireza Javadian Sabet. 2021. Connecting students with research advisors through user-controlled recommendation. InProceedings of the 15th ACM Conference on Recommender Systems. 745–748

    2021

  35. [43]

    Bernard Rous. 2012. Major update to ACM’s computing classification system. Commun. ACM 55, 11 (2012), 12–12

    2012

  36. [44]

    Mohammad Javad Saeedizade and Eva Blomqvist. 2024. Navigating ontology development with large language models. In European Semantic Web Conference. Springer, 143–161

    2024

  37. [45]

    Angelo Salatino, Tanay Aggarwal, Andrea Mannocci, Francesco Osborne, and Enrico Motta. 2025. A survey of knowledge organization systems of research fields: Resources and challenges. Quantitative Science Studies (04 2025), 1–44. arXiv:https://direct.mit.edu/qss/article-pdf/doi/...

    2025 doi

  38. [46]

    Salatino, Thiviyan Thanapalasingam, Andrea Mannocci, Aliaksandr Birukou, Francesco Osborne, and Enrico Motta

    Angelo A. Salatino, Thiviyan Thanapalasingam, Andrea Mannocci, Aliaksandr Birukou, Francesco Osborne, and Enrico Motta. 2020. The Computer Science Ontology: A Comprehensive Automatically-Generated Tax- onomy of Research Areas. Data Intelligence 2, 3 (07 2020), 379–416. arXiv:h...

    2020 doi

  39. [47]

    Angelo A Salatino, Thiviyan Thanapalasingam, Andrea Mannocci, Francesco Osborne, and Enrico Motta. 2018. The computer science ontology: a large-scale taxonomy of research areas. InThe Semantic Web–ISWC 2018: 17th International Semantic Web Conference, Monterey, CA, USA, Octobe...

    2018

  40. [48]

    Mark Sanderson and Bruce Croft. 1999. Deriving concept hierarchies from text. In Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval . 206–213

    1999

  41. [49]

    Dmitry Scherbakov, Nina Hubig, Vinita Jansari, Alexander Bakumenko, and Leslie A Lenert. 2024. The emergence of Large Language Models (LLM) as a tool in literature reviews: an LLM automated systematic review. arXiv preprint arXiv:2409.04600 (2024)

    2024 arXiv

  42. [50]

    Zhihong Shen, Hao Ma, and Kuansan Wang. 2018. A web-scale system for scientific knowledge exploration. arXiv preprint arXiv:1805.12216 (2018)

    2018 arXiv

  43. [51]

    Arthur Smith. 2020. Physics subject headings (physh). KO KNOWLEDGE ORGANIZATION 47, 3 (2020), 257–266

    2020

  44. [52]

    Yushi Sun, Hao Xin, Kai Sun, Yifan Ethan Xu, Xiao Yang, Xin Luna Dong, Nan Tang, and Lei Chen. 2024. Are Large Language Models a Good Replacement of Taxonomies? arXiv preprint arXiv:2406.11131 (2024)

    2024 arXiv

  45. [53]

    Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et al. 2024. Gemma: Open models based on gemini research and LLMs for Generating Research Topic Ontologies technology...

    2024 arXiv

  46. [54]

    Stefani Tsaneva, Danilo Dessì, Francesco Osborne, and Marta Sabou. 2025. Knowledge graph validation by integrating LLMs and human-in-the-loop. Information Processing & Management 62, 5 (2025), 104145

    2025

  47. [55]

    Kuansan Wang, Zhihong Shen, Chiyuan Huang, Chieh-Han Wu, Yuxiao Dong, and Anshul Kanakia. 2020. Mi- crosoft Academic Graph: When experts are not enough. Quantitative Science Studies 1, 1 (02 2020), 396–413. arXiv:https://direct.mit.edu/qss/article-pdf/1/1/396/1760880/qss_a_000...

    2020 doi

  48. [56]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv:2201.11903 [cs.CL] https://arxiv.org/abs/2201.11903

    2023 arXiv

  49. [57]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020

  50. [58]

    Marcia Lei Zeng. 2008. Knowledge organization systems (KOS). KO Knowledge Organization 35, 2-3 (2008), 160–182

    2008

  51. [59]

    Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu, Hengshu Zhu, Hui Xiong, and Qing He. 2020. A Comprehensive Survey on Transfer Learning. arXiv:1911.02685 [cs.LG] https://arxiv.org/abs/1911.02685

    2020 arXiv

  52. [2021]

    arXiv 2021

    Lora: Low-rank adaptation of large language models. arXiv 2021. arXiv preprint arXiv:2106.09685 (2021)

    2021 arXiv

  53. [2023]

    Ieee Access 11 (2023), 22468–22489

    Integrating conversational agents and knowledge graphs within the scholarly domain. Ieee Access 11 (2023), 22468–22489

    2023

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.