Pith. sign in

REVIEW 4 major objections 6 minor 73 references

OntoTune: Ontology-Driven Self-training for Aligning Large Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that aligning a language model to a hierarchical ontology by self-training on the model's own ontology-conditioned responses is a data-efficient way to reorganize domain knowledge, beating large-corpus baselines on…

desk verdict A clever selection signal for ontology self-training that needs an ablation and a quality gate on its own targets before the mechanism claim holds. read the letter →

arxiv 2502.05478 v1 pith:ZMSDXR6A submitted 2025-02-08 cs.CL

Zhiqiang Liu , Chengtao Gan , Junjie Wang , Yichi Zhang , Zhongpu Bo , Mengshu Sun , Huajun Chen , Wen Zhang This is my paper · ORCID
classification cs.CL
keywords LargeLanguageModelontologyalignmentself-trainingin-contextlearningdomainadaptationhypernymdiscoverymedicalquestionansweringSNOMEDCT
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

OntoTune asks whether a language model can internalize the hierarchical structure of a domain ontology by training on its own responses. For each medical concept, the seed model generates a knowledge card twice: once with only the instruction, and once with the concept's ontology information (definition, hypernyms, synonyms) supplied in the prompt. The paper treats a large gap between these two responses as evidence that the model lacks that concept's ontology knowledge, and fine-tunes the model to reproduce the ontology-guided response without seeing the ontology. On medical hypernym discovery and six medical QA benchmarks, OntoTune's supervised variant outperforms the LLaMA-3 seed and medical LLMs trained on large corpora, while losing less general knowledge than direct ontology injection. If right, this makes existing curated ontologies plus the model itself a cheap, reusable substitute for collecting large domain corpora.

What carries the argument

The load-bearing mechanism is inconsistency-based self-training. For instruction $x$ and concept $t$, the seed model $f_\theta$ produces $y_t = f_\theta(x, t)$ and $y^o_t = f_\theta(x, t, o_t)$, where $o_t$ is the concept's definition, hypernyms, and synonyms. A hybrid similarity score, combining embedding cosine similarity, ROUGE-L, and BLEU-4, ranks each concept; the lowest-$k$ entries become the training set, and the model is fine-tuned (via SFT and/or DPO) to output $y^o_t$ given only $x$. The implicit claim is that divergence between the two pathways marks exactly the places where ontology knowledge is missing, and that reproducing the in-context-learned answer internalizes the knowledge.

What would settle it

Train OntoTune with the same pipeline but replace the ontology-conditioned target $y^o$ with a response produced by adding a random, unrelated ontology fragment (or a generic in-context exemplar) to the prompt. If the medical QA and hypernym gains persist unchanged, the ontology content is not what drives the improvement. Alternatively, check whether the low-similarity concepts selected by the filter are exactly the concepts the seed model answers incorrectly when directly queried for their hypernyms; if the model answers most of them correctly, the inconsistency proxy is not detecting missing ontology knowledge.

Watch

Extended reading notes

Core claim

The central discovery is that the objective $f_{\theta'}(x) = f_{\theta}(x, o)$ is learnable from self-generated data: the seed model's ontology-conditioned responses, produced by in-context learning with the ontology fragment, serve as training targets, and an inconsistency filter selects only the concepts the model has not yet aligned with the ontology. The paper reports that this reorganized knowledge transfers beyond the ontology itself: OntoTune_sft reaches 65.53 MRR on the medical hypernym subset of SemEval-2018 Task 9, versus 54.86 for the LLaMA-3 baseline, and an average of 62.5 on six medical QA sets, versus 60.2 for the seed, while the direct-injection baseline TaxoLLaMA drops on general tasks. The trained model also satisfies its objective: after fine-tuning, its unaided responses resemble its ontology-conditioned responses far more than the seed model's do.

Load-bearing premise

The paper assumes that when a model's answer changes after the ontology is added to the prompt, the change means the model lacked that concept's ontology knowledge and the ontology-guided answer is the better one to imitate; if that mapping fails, the selected training data is just noise and the gains come from ordinary self-training on diverse instructions.

Editorial extensions

If this is right

  • OntoTune_sft achieves the best MRR among compared methods on the medical hypernym subset of SemEval-2018 Task 9 (65.53 versus 54.86 for the LLaMA-3 baseline).
  • It reaches the highest average medical QA score among LLaMA-3-8B-based methods (62.5 versus 60.2 for the seed model) across MedQA, MedMCQA, PubMedQA, and USMLE steps 1-3.
  • OntoTune preserves more general knowledge and safety than direct ontology injection and large-corpus medical LLMs, with an average drop of only about 0.49% on MMLU, ARC, and TriviaQA for the SFT variant.
  • Data efficiency is central: gains appear with as few as 9,000 training samples, and the best trade-off is reached around 75,000 samples, far below the token budgets of corpus-trained domain LLMs.
  • The mechanism transfers across seed models (LLaMA-3-8B and Qwen2-7B) and across languages (Italian and Spanish) that never appear in the training set.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the inconsistency proxy tracks genuine knowledge gaps, the same filter could serve as a model-internal audit: the set of concepts where the unaided and ontology-conditioned responses diverge is a map of what the model has not organized.
  • The recipe generalizes beyond ontologies to any structured knowledge source that can be injected as prompt context, including knowledge graphs, regulations, or product catalogs, with divergence between unaided and context-conditioned answers selecting the training cases.
  • A direct test of the mechanism: the selected low-similarity concepts should be enriched for long-tail or under-represented terms relative to the ontology as a whole; if the filter is not doing this, the gains may come from generic instruction diversity instead.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes OntoTune, a self-training framework that aligns an LLM with a domain ontology. For each concept, the seed model generates a response y without ontology and y_o with ontology information provided in-context. The authors select the k entries with the lowest similarity between y and y_o and fine-tune the seed model on y_o via SFT and/or DPO with y_o preferred over y. Experiments in the medical domain use SNOMED CT and evaluate on SemEval-2018 Task 9 hypernym discovery and six medical QA datasets, plus MMLU/ARC/TriviaQA/Advbench for general capabilities. The paper reports state-of-the-art results among LLaMA-3-8B-based methods on the medical hypernym subset and the average medical QA score, with better preservation of general knowledge than TaxoLLaMA.

Significance. The core idea of using a small, existing ontology to reorganize domain knowledge instead of collecting large-scale corpora is timely and potentially impactful for domain adaptation. The paper's strengths include the comparison against several LLaMA-3-8B baselines, evaluation on external benchmarks that are not fitted to the training set, a robustness check with Qwen2-7B, and an analysis of self-training versus distillation from stronger models. The framework is data-efficient and the authors state that code and data will be released. If the ontology contribution is confirmed by additional ablations and the target-quality issue is addressed, this would be a useful step toward low-cost domain alignment.

major comments (4)
  1. [Section 3.1, Appendix D] The selection rule assumes y_o is better than y when the two are inconsistent. Appendix D's own examples show y_o can be factually inconsistent (Seriphus politus is defined as a snail in Clausiliidae while its listed hypernym is Family Sciaenidae-drum, which the model itself correctly identifies as fish) or confabulated (Coital sexual orgasm produces a detailed fabricated research summary in response to a prompt the model otherwise refuses). No quality gate on y_o is reported. The authors should either filter y_o for factual consistency or provide evidence that on the selected low-similarity entries y_o is indeed superior to y; without this, the measured gains in Tables 1 and 2 could come from learning longer, more structured but less correct text.
  2. [Section 3.2.2, Tables 1-2] There is no ablation that isolates the ontology signal. The comparisons against TaxoLLaMA and medical LLMs show method-level improvements but do not establish that the gains come from ontology structure rather than from self-training on systematically prompted outputs. A control condition that replaces ontology information with generic structured text (e.g., 'organize your answer systematically' or random knowledge snippets), while keeping the pipeline and selection mechanism identical, is needed to support the 'ontology-driven' claim. Additionally, ablating the inconsistency-based selection (e.g., random selection of the same number of examples) would test the contribution of the selection step.
  3. [Section 4.1, Eqs. (5) and (7)] Several free parameters are not reported and their sensitivity is not explored: the hybrid similarity weights in Eq. (5), the DPO beta in Eq. (7), and the exact value of k per corpus beyond '100000 inconsistent samples' (the text later mentions 75,000 samples in Figure 5, which is ambiguous). Since these parameters directly determine which examples are selected and how strongly preference optimization is applied, the results are not fully reproducible without reporting their values and a sensitivity analysis.
  4. [Tables 1, 2, and 3] All results are reported from single runs without variance or significance tests. Several headline gaps are small (e.g., 58.4 vs 56.4 on MedQA SFT in Table 2, and the 0.49% average decline on general knowledge in Section 4.4.1). The paper should report multiple seeds or at least bootstrap confidence intervals to establish that the reported improvements and preservation scores are not within noise.
minor comments (6)
  1. [Throughout] There are several typos and naming inconsistencies: 'exsiting' in the contributions list, 'Defintion' in the Section 3 heading, 'obverse' for 'observe' in Section 4.2.4, 'firections' in Figure 16, and 'MdeMCQA' in Figure 11's caption.
  2. [Table 1] The baseline name 'Hybird' should be 'Hybrid', and the notation for the seed model alternates between 'LLaMA3 8B*' and 'LLaMA3*' across tables and text.
  3. [Section 1, footnote 2] The footnote link to the OpenAI 'learning to reason with LLMs' page does not support the statement about in-context learning; it appears to be an incorrect reference.
  4. [Section 4.5.4] The parameter-shift metric is computed as the mean squared change in parameters, but it is unclear whether this is measured on the LoRA adapters only or the full model parameters after merging; please clarify the implementation.
  5. [Figure 5 and Section 4.5.1] The text says the model trained on 75,000 samples achieves the best performance, while the main setup uses 300,000 samples; please clarify whether the 75,000 figure is per corpus or total, and reconcile it with the earlier description of k=100000.
  6. [Section 4.5.3] The distillation source is called 'deepseek-v2.5-chat' in the text but 'deepseek-chat' in Table 4; please use a consistent name.

Circularity Check

1 steps flagged · score 2.0 of 10

Only the Appendix B verification is circular by construction; the main external benchmark claims remain independent of the training objective.

  1. self definitional [Appendix B (Training Objective Analysis) with Section 3.1 Eqs. (1)-(2) and Section 3.2.3 Eq. (6)]
    "We use the LLM trained with OntoTune_sft to generate response y and reference response y_o again to directly verify whether our training objective is achieved. ... As shown in Figure 9, we observe that under three similarity metrics, the LLM trained with OntoTune aligns well with the objective curve, showing significant improvement compared to the seed model before training. This directly indicates that the seed model fine-tuned with OntoTune generates responses that are more guided by the ontology."

    The claimed verification is the training objective itself. Section 3.1 defines the optimization target as f_theta'(x) = f_theta(x,o) (Eq. 2), and Section 3.2.3 trains with SFT by maximizing log P_theta(y_o|x) over D_sft = {x_n, y_o_n} (Eq. 6) and with DPO by preferring y_o over y. Thus a post-training increase in similarity between the model's response y and the reference y_o is a direct, expected consequence of fitting the model to y_o; it does not independently establish that the model has mastered ontology structure. The central external-benchmark claims (SemEval, MedQA, MedMCQA, etc.) are not fitted to y_o, so this circularity is confined to the Appendix B verification and does not by itself undermine the main comparative results.

full rationale

The main derivation chain is not circular: the paper generates y_o by conditioning the seed model on externally supplied SNOMED CT definitions, hypernyms, and synonyms, selects low-similarity (y, y_o) pairs, and then fine-tunes on y_o. Evaluation is performed on external benchmarks not used to construct y_o, so the headline hypernym-discovery and medical-QA improvements are independent evidence. The self-citations in the paper ([9], [50], [62], [63]) are background related work and are not load-bearing; no uniqueness theorem is imported from the authors' prior work. One genuinely self-referential step exists: Appendix B presents convergence of the trained model's y to the reference y_o as direct evidence of ontology alignment, but that convergence is definitional because SFT/DPO were explicitly optimized to make y approach y_o. This is a minor circularity and does not affect the main external evaluations. The Appendix D examples showing internally inconsistent or confabulated y_o are real data-quality risks, but they are not circularity: concerns about target correctness belong to correctness risk, not to whether the derivation reduces to its inputs. Overall score: 2.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a proxy: inconsistency between conditioned and unconditioned outputs identifies missing ontology knowledge. It also relies on the correctness of SNOMED CT and on the seed model's self-generated definitions. No new physical or conceptual entities are introduced.

free parameters (3)
  • k, number of selected inconsistent samples per corpus = 100,000
    Set to match training scale of existing domain LLMs; no principled selection criterion. It controls training data size and is evaluated only via a coarse sample-size curve.
  • hybrid similarity weights = 1.0 for each of cosine, ROUGE-L, BLEU-4
    Eq. (5) sums three metrics with equal weight; no sensitivity analysis or calibration against a validation set.
  • DPO beta = not reported
    The DPO objective in Eq. (7) includes beta controlling deviation from the reference policy, but its value is not given in the experimental setup.
assumptions (4)
  • domain assumption If the seed model has mastered ontology knowledge, then y equals y_o; otherwise y_o is better.
    Section 3.1; this is the selection signal that determines the training set. If it fails, selected examples are not missing ontology knowledge.
  • domain assumption The hybrid similarity score captures meaningful inconsistency between y and y_o.
    Section 3.2.2, Eq. (5); the lowest-k threshold is applied without validating that low similarity corresponds to ontology ignorance.
  • domain assumption SNOMED CT's is-a relations, hypernyms, and synonyms are correct and sufficient to reorganize medical knowledge.
    Section 4.1; the source ontology is taken as ground truth for training targets.
  • domain assumption The seed model's self-completed definitions are accurate enough to serve as training targets.
    Section 3.2; only 8,275 of 367,978 SNOMED concepts have definitions, so the model fills the rest.

how reviews work

0 comments
Cite this review

Pith. "Pith review of OntoTune: Ontology-Driven Self-training for Aligning Large Language Models." pith.science (2026). https://pith.science/paper/ZMSDXR6A

@misc{pith2026250205478,
  author       = {Pith},
  title        = {Pith review of: OntoTune: Ontology-Driven Self-training for Aligning Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZMSDXR6A}},
  note         = {Machine review of arXiv:2502.05478}
}
read the original abstract

Existing domain-specific Large Language Models (LLMs) are typically developed by fine-tuning general-purposed LLMs with large-scale domain-specific corpora. However, training on large-scale corpora often fails to effectively organize domain knowledge of LLMs, leading to fragmented understanding. Inspired by how humans connect concepts and organize knowledge through mind maps, we aim to emulate this approach by using ontology with hierarchical conceptual knowledge to reorganize LLM's domain knowledge. From this perspective, we propose an ontology-driven self-training framework called OntoTune, which aims to align LLMs with ontology through in-context learning, enabling the generation of responses guided by the ontology. We leverage in-context learning to identify whether the LLM has acquired the specific concept's ontology knowledge, and select the entries not yet mastered by LLM as the training set to further align the LLM with ontology. Compared to existing domain LLMs based on newly collected large-scale domain-specific corpora, our OntoTune, which relies on the existing, long-term developed ontology and LLM itself, significantly reduces data maintenance costs and offers improved generalization ability. We conduct our study in the medical domain to evaluate the effectiveness of OntoTune, utilizing a standardized medical ontology, SNOMED CT as our ontology source. Experimental results demonstrate that OntoTune achieves state-of-the-art performance in both in-ontology task hypernym discovery and out-of-ontology task medical domain QA. Moreover, compared to the latest direct ontology injection method TaxoLLaMA, our OntoTune better preserves original knowledge of LLM. The code and data are available at https://github.com/zjukg/OntoTune.

Figures

Figures reproduced from arXiv: 2502.05478 by the authors.

Figure 1
Figure 1. A simple example illustrates how hierarchical struc [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of OntoTune which aligns LLMs with ontology through in-context learning. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Ontology-aware corpus generation templates. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: The templates of TaxoLLaMA*’s instruction-tuning [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 6
Figure 6. Figure 6: Domain performances and general performances [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 8
Figure 8. Figure 8: (a) Comparison of OntoTune variants and TaxoL [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: The distribution of consistency scores for response [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Examples of prompts for the evaluation of MedQA. [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 12
Figure 12. Figure 12: Examples of prompts for the evaluation of PubMedQA. [PITH_FULL_IMAGE:figures/full_fig_p012_12.png]
Figure 13
Figure 13. Figure 13: Examples of prompts for the evaluation of USMLE-step 1-3. [PITH_FULL_IMAGE:figures/full_fig_p012_13.png]
Figure 14
Figure 14. Figure 14: An Example of inconsistent diverse corpus. [PITH_FULL_IMAGE:figures/full_fig_p014_14.png]
Figure 15
Figure 15. Figure 15: An Example of inconsistent conceptual corpus. [PITH_FULL_IMAGE:figures/full_fig_p014_15.png]
Figure 16
Figure 16. Figure 16: An Example of inconsistent professional corpus. [PITH_FULL_IMAGE:figures/full_fig_p015_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

73 extracted references · 40 canonical work pages

  1. [1]

    Emre Can Acikgoz, Osman Batur Ince, Rayene Bench, Arda Anil Boz, Ilker Kesen, Aykut Erdem, and Erkut Erdem. 2024. Hippocrates: An Open-Source Framework for Advancing Large Language Models in Healthcare.CoRR abs/2404.16621 (2024). doi:10.48550/ARXIV.2404.16621 arXiv:2404.16621

    work page Pith review arXiv doi:10.48550/arxiv.2404.16621 2024

  2. [2]

    Guilherme F. C. F. Almeida, José Luiz Nunes, Neele Engelmann, Alex Wiegmann, and Marcelo de Araújo. 2024. Exploring the psychology of LLMs’ moral and legal reasoning. Artif. Intell. 333 (2024), 104145. doi:10.1016/J.ARTINT.2024.104145

    work page arXiv 2024

  3. [3]

    Massih-Reza Amini, Vasilii Feofanov, Loïc Pauletto, Emilie Devijver, and Yury Maximov. 2022. Self-Training: A Survey. CoRR abs/2202.12040 (2022). arXiv:2202.12040 https://arxiv.org/abs/2202.12040

    arXiv 2022

  4. [4]

    Yuhang Bai, Richong Zhang, Fanshuang Kong, Junfan Chen, and Yongyi Mao

    work page

  5. [5]

    Gábor Berend, Márton Makrai, and Peter Földiák. 2018. 300-sparsans at SemEval- 2018 Task 9: Hypernymy as interaction of sparse attributes. In Proceedings of The 12th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2018, New Orleans, Louisiana, USA, June 5-6, 2018 , Marianna Apidianaki, Saif M. Mohammad, Jonathan May, Ekaterina Shutova, St...

    work page 2018

  6. [6]

    Gabriel Bernier-Colborne and Caroline Barrière. 2018. CRIM at SemEval-2018 Task 9: A Hybrid Approach to Hypernym Discovery. In Proceedings of The 12th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2018, New Orleans, Louisiana, USA, June 5-6, 2018, Marianna Apidianaki, Saif M. Mohammad, Jonathan May, Ekaterina Shutova, Steven Bethard, an...

    work page doi:10.18653/v1/s18-1116 2018

  7. [7]

    Gagan Bhatia, El Moatez Billah Nagoudi, Hasan Cavusoglu, and Muhammad Abdul-Mageed. 2024. FinTral: A Family of GPT-4 Level Multimodal Financial Large Language Models. In Findings of the Association for Computational Linguis- tics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Ass...

    work page 2024

  8. [8]

    José Camacho-Collados, Claudio Delli Bovi, Luis Espinosa Anke, Sergio Oramas, Tommaso Pasini, Enrico Santus, Vered Shwartz, Roberto Navigli, and Horacio Saggion. 2018. SemEval-2018 Task 9: Hypernym Discovery. In Proceedings of The 12th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2018, New Orleans, Louisiana, USA, June 5-6, 2018 , Mari...

    work page 2018

Show all 73 references
  1. [9]

    Zhuo Chen, Wen Zhang, Yufeng Huang, Mingyang Chen, Yuxia Geng, Hongtao Yu, Zhen Bi, Yichi Zhang, Zhen Yao, Wenting Song, Xinliang Wu, Yi Yang, Mingyi Chen, Zhaoyang Lian, Yingying Li, Lei Cheng, and Huajun Chen. 2023. Tele- Knowledge Pre-training for Fault Analysis. In 39th IE...

    2023

  2. [10]

    Daixuan Cheng, Shaohan Huang, and Furu Wei. 2024. Adapting Large Language Models via Reading Comprehension. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenRe- view.net. https://openreview.net/forum?id=y886UXPEZ0

    2024

  3. [11]

    Kanithi, Tathagata Raha, Shadab Khan, and Marco AF Pimentel

    Clément Christophe, Praveen K. Kanithi, Tathagata Raha, Shadab Khan, and Marco AF Pimentel. 2024. Med42-v2: A Suite of Clinical LLMs. CoRR abs/2408.06142 (2024). doi:10.48550/ARXIV.2408.06142 arXiv:2408.06142

    2024 doi arXiv

  4. [12]

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have Solved Question An- swering? Try ARC, the AI2 Reasoning Challenge. CoRR abs/1803.05457 (2018). arXiv:1803.05457 http://arxiv.org/abs/1803.05457

    2018 arXiv

  5. [13]

    OpenCompass Contributors. 2023. OpenCompass: A Universal Evaluation Plat- form for Foundation Models. https://github.com/open-compass/opencompass

    2023

  6. [14]

    Felix J Dorfner, Amin Dada, Felix Busch, Marcus R Makowski, Tianyu Han, Daniel Truhn, Jens Kleesiek, Madhumita Sushil, Jacqueline Lammert, Lisa C Adams, et al

  7. [15]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024)

    2024 arXiv

  8. [16]

    Zorik Gekhman, Gal Yona, Roee Aharoni, Matan Eyal, Amir Feder, Roi Reichart, and Jonathan Herzig. 2024. Does Fine-Tuning LLMs on New Knowledge Encour- age Hallucinations? CoRR abs/2405.05904 (2024). doi:10.48550/ARXIV.2405.05904 arXiv:2405.05904

    2024 doi arXiv

  9. [17]

    Ashwin Kumar Gururajan, Enrique Lopez-Cuena, Jordi Bayarri-Planas, Adrián Tormos, Daniel Hinjos, Pablo Bernabeu-Perez, Anna Arias-Duart, Pablo Agustin Martin-Torres, Lucia Urcelay-Ganzabal, Marta Gonzalez-Mallo, Sergio Álvarez- Napagao, Eduard Ayguadé Parra, and Ulises Cortés ...

    2024 doi arXiv

  10. [18]

    Adams, Jens-Michalis Papaioannou, Paul Grundmann, Tom Oberhauser, Alexander Löser, Daniel Truhn, and Keno K

    Tianyu Han, Lisa C. Adams, Jens-Michalis Papaioannou, Paul Grundmann, Tom Oberhauser, Alexander Löser, Daniel Truhn, and Keno K. Bressem. 2023. MedAl- paca - An Open-Source Collection of Medical Conversational AI Models and Training Data. CoRR abs/2304.08247 (2023). doi:10.485...

    2023 doi arXiv

  11. [19]

    Junxian He, Jiatao Gu, Jiajun Shen, and Marc’Aurelio Ranzato. 2020. Revisiting Self-Training for Neural Sequence Generation. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020 . OpenReview.net. https://openreview.ne...

    2020

  12. [20]

    William Held and Nizar Habash. 2019. The Effectiveness of Simple Hybrid Systems for Hypernym Discovery. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers , Anna Korh...

    2019

  13. [21]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring Massive Multitask Lan- guage Understanding. In 9th International Conference on Learning Representa- tions, ICLR 2021, Virtual Event, Austria, May 3-7, 2021 . ...

    2021

  14. [22]

    Courville, Alessan- dro Sordoni, and Rishabh Agarwal

    Arian Hosseini, Xingdi Yuan, Nikolay Malkin, Aaron C. Courville, Alessan- dro Sordoni, and Rishabh Agarwal. 2024. V-STaR: Training Verifiers for Self- Taught Reasoners. CoRR abs/2402.06457 (2024). doi:10.48550/ARXIV.2402.06457 arXiv:2402.06457

    2024 doi arXiv

  15. [23]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 20...

    2022

  16. [24]

    Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2024. Large Language Models Cannot Self-Correct Reasoning Yet. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2...

    2024

  17. [26]

    Zijie Huang, Daheng Wang, Binxuan Huang, Chenwei Zhang, Jingbo Shang, Yan Liang, Zhengyang Wang, Xian Li, Christos Faloutsos, Yizhou Sun, and Wei Wang. 2023. Concept2Box: Joint Geometric Embeddings for Learning Two-View Knowledge Graphs. In Findings of the Association for Comp...

    2023 doi

  18. [27]

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, De- vendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thoma...

    2023 doi arXiv

  19. [28]

    Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. 2020. What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams. CoRR abs/2009.13081 (2020). arXiv:2009.13081 https://arxiv.org/abs/2009.13081

    2020 arXiv

  20. [29]

    Cohen, and Xinghua Lu

    Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William W. Cohen, and Xinghua Lu

  21. [30]

    Weld, and Luke Zettlemoyer

    Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. 2017. TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehen- sion. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, C...

    2017 doi

  22. [31]

    Yanis Labrak, Adrien Bazoge, Emmanuel Morin, Pierre-Antoine Gourraud, Mick- ael Rouvier, and Richard Dufour. 2024. BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domains. In Findings of the Association for Computational Linguistics, ACL 20...

    2024

  23. [32]

    Jiawei Li, Yizhe Yang, Yu Bai, Xiaofeng Zhou, Yinghao Li, Huashan Sun, Yuhang Liu, Xingpeng Si, Yuhao Ye, Yixiao Wu, Yiguan Lin, Bin Xu, Ren Bowen, Chong Feng, Yang Gao, and Heyan Huang. 2024. Fundamental Capabilities of Large Language Models and their Applications in Domain S...

    2024

  24. [33]

    Hashimoto

    Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Car- los Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. AlpacaEval: An Automatic Evaluator of Instruction-following Models. https://github.com/tatsu- lab/alpaca_eval

    2023

  25. [34]

    Zhenghao Lin, Zhibin Gou, Yeyun Gong, Xiao Liu, Yelong Shen, Ruochen Xu, Chen Lin, Yujiu Yang, Jian Jiao, Nan Duan, and Weizhu Chen. 2024. Rho-1: Not All Tokens Are What You Need.CoRR abs/2404.07965 (2024). doi:10.48550/ARXIV. 2404.07965 arXiv:2404.07965

    2024 doi arXiv

  26. [35]

    Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon, and Tie-Yan Liu. 2022. BioGPT: generative pre-trained transformer for biomedical text generation and mining. Briefings Bioinform. 23, 6 (2022). doi:10.1093/BIB/ BBAC409

    2022 doi

  27. [36]

    Abdelzaher, and Jiawei Han

    Yu Meng, Martin Michalski, Jiaxin Huang, Yu Zhang, Tarek F. Abdelzaher, and Jiawei Han. 2023. Tuning Language Models as Training Data Generators for Augmentation-Enhanced Few-Shot Learning. In International Conference on Ma- chine Learning, ICML 2023, 23-29 July 2023, Honolulu...

    2023

  28. [37]

    George A. Miller. 1994. WORDNET: A Lexical Database for English. In Human Language Technology, Proceedings of a Workshop held at Plainsboro, New Jerey, USA, March 8-11, 1994 . Morgan Kaufmann. https://aclanthology.org/H94-1111/

    1994

  29. [38]

    Viktor Moskvoretskii, Ekaterina Neminova, Alina Lobanova, Alexander Panchenko, and Irina Nikishina. 2024. TaxoLLaMA: WordNet-based Model for Solving Multiple Lexical Semantic Tasks. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Vol...

    2024

  30. [39]

    Viktor Moskvoretskii, Alexander Panchenko, and Irina Nikishina. 2024. Are Large Language Models Good at Lexical Semantics? A Case of Taxonomy Learning. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LR...

    2024

  31. [40]

    Irina Nikishina, Polina Chernomorchenko, Anastasiia Demidova, Alexander Panchenko, and Chris Biemann. 2023. Predicting Terms in IS-A Relations with Pre- trained Transformers. In Findings of the Association for Computational Linguistics: IJCNLP-AACL 2023 - Findings, Nusa Dua, B...

    2023

  32. [41]

    OpenAI. 2023. GPT-4 Technical Report. CoRR abs/2303.08774 (2023). doi:10. 48550/ARXIV.2303.08774 arXiv:2303.08774

    2023 doi arXiv

  33. [42]

    Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. 2022. MedMCQA: A Large-scale Multi-Subject Multi-Choice Dataset for Medical do- main Question Answering. In Conference on Health, Inference, and Learning, CHIL 2022, 7-8 April 2022, Virtual Event (Proceedings of M...

    2022

  34. [43]

    Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2024. Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!. InThe Twelfth International Con- ference on Learning Representations, ICLR 2024, Vien...

    2024

  35. [44]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.J. Mach. Learn. Res. 21 (2020), 140:1–140:67. http://jmlr.org/p...

    2020

  36. [45]

    Mengjie Ren, Boxi Cao, Hongyu Lin, Cao Liu, Xianpei Han, Ke Zeng, Guan- glu Wan, Xunliang Cai, and Le Sun. 2024. Learning or Self-aligning? Re- thinking Instruction Fine-tuning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume ...

    2024 doi

  37. [46]

    Stefan Schulz and Gunnar O. Klein. 2008. SNOMED CT - advances in concept mapping, retrieval, and ontological foundations. Selected contributions to the Semantic Mining Conference on SNOMED CT (SMCS 2006). BMC Medical Informatics Decis. Mak. 8, S-1 (2008), S1. doi:10.1186/1472-...

    2008 doi

  38. [47]

    Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, Xavier Garcia, Peter J

    Avi Singh, John D. Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, Xavier Garcia, Peter J. Liu, James Harrison, Jaehoon Lee, Kelvin Xu, Aaron T. Parisi, Abhishek Kumar, Alexander A. Alemi, Alex Rizkowsky, Azade Nova, Ben Adlam, Bernd Bohnet, Gamaleldin Fathy Elsayed, Ha...

    2024

  39. [48]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucu- rull, David Esiobu, Jude Fernandes, Jeremy...

    2023 doi arXiv

  40. [49]

    Gladys Tyen, Hassan Mansoor, Victor Carbune, Peter Chen, and Tony Mak. 2024. LLMs cannot find reasoning errors, but can correct them given the error location. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August ...

    2024

  41. [50]

    Pan, Wen Zhang, and Huajun Chen

    Junjie Wang, Mingyang Chen, Binbin Hu, Dan Yang, Ziqi Liu, Yue Shen, Peng Wei, Zhiqiang Zhang, Jinjie Gu, Jun Zhou, Jeff Z. Pan, Wen Zhang, and Huajun Chen

  42. [51]

    Mengru Wang, Yunzhi Yao, Ziwen Xu, Shuofei Qiao, Shumin Deng, Peng Wang, Xiang Chen, Jia-Chen Gu, Yong Jiang, Pengjun Xie, Fei Huang, Huajun Chen, and Ningyu Zhang. 2024. Knowledge Mechanisms in Large Language Models: A Survey and Perspective. CoRR abs/2407.15017 (2024). doi:1...

    2024 doi arXiv

  43. [52]

    Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, and Furu Wei. 2021. MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers. In Findings of the Association for Computational Lin- guistics: ACL/IJCNLP 2021, Online Event, August 1-6, 20...

    2021 doi

  44. [53]

    Smith, Daniel Khashabi, and Hannaneh Hajishirzi

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. Self-Instruct: Aligning Language Models with Self-Generated Instructions. InProceedings of the 61st Annual Meeting of the Association for Computational Lingui...

    2023 doi

  45. [54]

    Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. 2023. PMC- LLaMA: Further Finetuning LLaMA on Medical Papers. CoRR abs/2304.14454 (2023). doi:10.48550/ARXIV.2304.14454 arXiv:2304.14454

    2023 doi arXiv

  46. [55]

    CoRR abs/2406.14282 (2024)

    Learning to Plan for Retrieval-Augmented Large Language Models from Knowledge Graphs. CoRR abs/2406.14282 (2024). doi:10.48550/ARXIV.2406.14282 arXiv:2406.14282

    2024 doi arXiv

  47. [56]

    Guohui Xiao, Diego Calvanese, Roman Kontchakov, Domenico Lembo, Antonella Poggi, Riccardo Rosati, and Michael Zakharyaschev. 2018. Ontology-Based Data Access: A Survey. In Proceedings of the Twenty-Seventh International Joint Con- ference on Artificial Intelligence, IJCAI 2018...

    2018 doi

  48. [57]

    Hovy, and Quoc V

    Qizhe Xie, Minh-Thang Luong, Eduard H. Hovy, and Quoc V. Le. 2020. Self- Training With Noisy Student Improves ImageNet Classification. In2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, W A, USA, June 13-19, 2020 . Computer Vision Founda...

    2020

  49. [58]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Cheng- peng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jin...

    2024 doi arXiv

  50. [59]

    Pournejatian, Hoo Chang Shin, Kaleb E

    Xi Yang, Nima M. Pournejatian, Hoo Chang Shin, Kaleb E. Smith, Christopher Parisien, Colin Compas, Cheryl Martin, Mona G. Flores, Ying Zhang, Tanja Magoc, Christopher A. Harle, Gloria P. Lipori, Duane A. Mitchell, William R. Hogan, Elizabeth A. Shenkman, Jiang Bian, and Yonghu...

    2022 doi arXiv

  51. [60]

    Rosenberg, and Gideon Mann

    Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David S. Rosenberg, and Gideon Mann. 2023. BloombergGPT: A Large Language Model for Finance. CoRR abs/2303.17564 (2023). doi:10.48550/ARXIV.2303.17564 arXiv:2303.17564

    2023 doi arXiv

  52. [61]

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. 2022. STaR: Bootstrapping Reasoning With Reasoning. In Advances in Neural Informa- tion Processing Systems 35: Annual Conference on Neural Information Process- ing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, Novemb...

    2022

  53. [62]

    Wen Zhang, Bibek Paudel, Liang Wang, Jiaoyan Chen, Hai Zhu, Wei Zhang, Abraham Bernstein, and Huajun Chen. 2019. Iteratively Learning Embeddings and Rules for Knowledge Graph Reasoning. In The World Wide Web Conference, WWW 2019, San Francisco, CA, USA, May 13-17, 2019 , Ling ...

    2019

  54. [63]

    Yichi Zhang, Zhuo Chen, Yin Fang, Yanxi Lu, Fangming Li, Wen Zhang, and Huajun Chen. 2024. Knowledgeable Preference Alignment for LLMs in Domain- specific Question Answering. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual ...

    2024 doi

  55. [64]

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: S...

    2024 arXiv

  56. [65]

    Zhaorui Yang, Tianyu Pang, Haozhe Feng, Han Wang, Wei Chen, Minfeng Zhu, and Qian Liu. 2024. Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pap...

    2024

  57. [66]

    Chen, Peilin Zhou, Junling Liu, Yining Hua, Chengfeng Mao, Xian Wu, Zheng Li, and Fenglin Liu

    Hongjian Zhou, Boyang Gu, Xinyu Zou, Yiru Li, Sam S. Chen, Peilin Zhou, Junling Liu, Yining Hua, Chengfeng Mao, Xian Wu, Zheng Li, and Fenglin Liu

  58. [67]

    Zhi Zhou, Jiang-Xin Shi, Peng-Xiao Song, Xiaowen Yang, Yi-Xuan Jin, Lan-Zhe Guo, and Yu-Feng Li. 2024. LawGPT: A Chinese Legal Knowledge-Enhanced Large Language Model. CoRR abs/2406.04614 (2024). doi:10.48550/ARXIV.2406.04614 arXiv:2406.04614

    2024 doi arXiv

  59. [68]

    yes, " “no,

    Andy Zou, Zifan Wang, J. Zico Kolter, and Matt Fredrikson. 2023. Univer- sal and Transferable Adversarial Attacks on Aligned Language Models. CoRR abs/2307.15043 (2023). doi:10.48550/ARXIV.2307.15043 arXiv:2307.15043 Appendix A Dataset Details • SemEval-2018 Task 9 [8] include...

    2023 doi arXiv

  60. [70]

    Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. 2023. LIMA: Less Is More for Alignment. In Advances in Neural Information Processing Sys...

    2023

  61. [2019]

    PubMedQA: A Dataset for Biomedical Research Question Answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Process- ing, EMNLP-IJCNLP 2019, Hong Kong, China, November ...

    2019 doi

  62. [2021]

    In Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, Online Event, August 1-6, 2021 (Findings of ACL, Vol

    Hypernym Discovery via a Recurrent Mapping Model. In Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, Online Event, August 1-6, 2021 (Findings of ACL, Vol. ACL/IJCNLP 2021) , Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.). Association...

    2021 doi

  63. [2023]

    CoRR abs/2311.05112 (2023)

    A Survey of Large Language Models in Medicine: Progress, Application, and Challenge. CoRR abs/2311.05112 (2023). doi:10.48550/ARXIV.2311.05112 arXiv:2311.05112

    2023 doi arXiv

  64. [2024]

    arXiv preprint arXiv:2408.13833 (2024)

    Biomedical Large Languages Models Seem not to be Superior to Generalist Models on Unseen Medical Data. arXiv preprint arXiv:2408.13833 (2024)

    2024 arXiv

  65. [3466]

    doi:10.1109/ICDE55515.2023.00265

    2023

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.