Pith. sign in

REVIEW 4 major objections 7 minor 1 cited by

Have We Designed Generalizable Structural Knowledge Promptings? Systematic Evaluation and Rethinking

T0 review · 4 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Structural knowledge prompting reliably delivers coarse-grained facts to LLMs across task types, but it fails at fine-grained entity identification, according to a systematic nine-task benchmark.

desk verdict Worth engaging: SUBARU is a genuinely useful benchmark and the fine-grained negative result is solid, but the coarse-grained success claim is partly confounded by textual leakage in the MC tasks. read the letter →

arxiv 2501.00244 v1 pith:FCCQNZYQ submitted 2024-12-31 cs.CL

Yichi Zhang , Zhuo Chen , Lingbing Guo , Yajing Xu , Shaokai Chen , Mengshu Sun , Binbin Hu , Zhiqiang Zhang
show 3 more authors
Lei Liang Wen Zhang Huajun Chen
This is my paper · ORCID
classification cs.CL
keywords structuralknowledgepromptinggraphlargelanguagemodelsadapterbenchmarkgranularitycompletionmulti-tasktransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether structural knowledge prompting (SKP) — feeding a frozen LLM embeddings of entities, triples, or subgraphs through an adapter — is a genuinely generalizable way to improve factual accuracy. To answer it, the authors build SUBARU, a benchmark of nine tasks crossing three granularities (entity, triple, subgraph) with three difficulty levels (classification, multiple choice, description), and test 16 combinations of structural encoders and adapters. They find that SKP consistently helps on coarse-grained reasoning such as triple and subgraph multiple choice, but fails on fine-grained tasks like naming an entity from its embedding, where exact-match accuracy is zero. They also find that a simple MLP adapter beats more complex adapters on most tasks, and that performance peaks at three to four adapter layers. The paper's conclusion is that SKP is a useful coarse-grained knowledge channel but not yet a source of precise factual awareness.

What carries the argument

The central machinery is the SKP pipeline itself: a pretrained structural encoder maps KG elements to embeddings, an adapter projects those embeddings into the LLM's token-embedding space, and the frozen LLM performs next-token prediction conditioned on the structural prompt plus a task instruction. The evaluation instrument is SUBARU, a 3x3 benchmark whose entity, triple, and subgraph granularities are crossed with classification, multiple-choice, and description levels, letting the authors separate granularity from difficulty. The benchmark also masks textual names in the prompt templates and includes a text-removal control, which together test whether the LLM is actually using the structural tokens rather than the words in the query.

What would settle it

Run the triple and subgraph multiple-choice tasks with all textual names removed from both query and options, keeping only structural embeddings and arbitrary option labels, and check whether accuracy stays above chance; the paper's own text-removal control on triple multiple-choice predicts it will not for several encoders. A clean version across all tasks would settle whether SKP itself, rather than the LLM's parametric knowledge, drives the coarse-grained results.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes a capability boundary for structural knowledge prompting: SKP can transmit coarse-grained relational and attribute information to LLMs at entity, triple, and subgraph granularities, but it cannot transmit fine-grained, precisely identifying facts. The 16-setting sweep shows the simple MLP adapter dominates, complex adapters such as Qformer and MoE do not help and often hurt on subgraph tasks, and the paradigm transfers across 7B–8B LLM backbones with only small variation. The benchmark's most telling result is the EG-DESC task, where no configuration can recover an entity's name from its structural embedding (0.00 exact match), and the case study in which every subgraph description misses the central entity. From this the paper concludes that future SKP design should add more prompt tokens aimed at activating precise, detailed information.

Load-bearing premise

The evaluation assumes that strong multiple-choice accuracy reflects what the model learned from the structural prompt tokens, but the triple multiple-choice template includes a textual relation name and textual options, and the paper's own text-removal control shows TransE dropping from 86.51 to 49.09 accuracy when the text is removed, so the non-textual signal alone is much weaker.

Editorial extensions

If this is right

  • On triple and subgraph multiple-choice tasks, SKP lifts accuracy well above random choice, so the paradigm is a usable coarse-grained interface between knowledge graphs and LLMs.
  • For applications that need exact entity identification, such as entity linking or fact verification, current SKP provides almost nothing: entity-level description exact match is 0.00 across configurations.
  • Adapter complexity is not buying accuracy; a two-layer MLP is the strongest bridge, and Qformer or MoE can reduce performance on subgraph tasks.
  • Training on additional tasks or granularities does not reliably transfer to the target task, so SKP modules currently need task-specific training; the exception is induction to new entities in triple multiple choice, which works almost as well as seen entities.
  • Adapter depth follows an inverted-U curve, with three to four layer MLPs optimal, so scaling adapter capacity alone is not the path to better structural prompting.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the text-removal results generalize, a large share of the reported multiple-choice gains may come from the LLM's own parametric knowledge activated by the textual query, not from structural tokens; the paper's coarse-grained conclusion would then describe the combined prompt rather than SKP alone.
  • A natural testable extension is to add entity-description or entity-name tokens to the SKP sequence; the paper's own closing recommendation predicts this should close part of the entity-description gap, and SUBARU makes that measurable.
  • The lack of transfer across tasks suggests the trained adapters may be memorizing task-specific mappings rather than learning a general structural semantics; layer-wise attention probing or zero-shot task switches could test this directly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes SUBARU, a benchmark of nine tasks spanning three granularities (entity, triple, subgraph) and three difficulty levels (classification, multiple choice, description), and uses it to evaluate structural knowledge prompting (SKP) across four dimensions: granularity, transferability, scalability, and universality. The evaluation combines four structural encoders (TransE, DistMult, RotatE, R-GCN), four adapters (FC, MLP, MoE, Qformer), and a frozen Llama3-8B-Instruct, with additional experiments on other backbones. The authors report three main conclusions: simple MLP adapters are surprisingly effective; SKP excels at coarse-grained reasoning tasks (TG/SG MC) but struggles with fine-grained entity-level understanding; and SKP shows limited transferability and modest scalability. The central claim is that SKP effectively provides LLMs with coarse-grained information but cannot achieve precise factual awareness.

Significance. If the conclusions hold, the paper provides a useful empirical map of where current SKP methods succeed and fail, and the SUBARU benchmark could serve as a reusable testbed for future structural prompting research. The authors ship code and data, evaluate 16 SKP configurations, and include a case study that illustrates the coarse-vs-fine distinction. The paper is honest about its limitations (benchmark scale, LLM sizes, lack of mechanistic analysis). However, the central claim about coarse-grained reasoning is weakened by textual leakage in the MC prompts and by the absence of variance estimates, so the significance is conditional on correcting these issues.

major comments (4)
  1. [§4.2.2 and Appendix B.3 (Table 4)] The protocol states that "important textual information" is removed so that models solve tasks "using mainly the SKPs rather than the texts," but the TG-MC template in Figure 14 includes a textual relation name (e.g., "occupation") and textual entity-name options. The authors' own control in Table 4 shows this matters: removing text drops TransE TG-MC accuracy from 86.51 to 49.09, near the 25% random baseline, and R-GCN from 94.59 to 78.45, while DistMult and RotatE are less affected. Since Observation 2 and the coarse-grained half of the central claim rely on TG-MC and SG-MC, the main results in Table 2 do not cleanly establish that SKP itself drives the strong MC performance. The w/o-text control should be extended to all MC tasks and adapter/encoder combinations, and the coarse-grained claim should be conditioned on the textual shortcut or removed from the headline.
  2. [Table 2 and all experiments] No error bars, confidence intervals, or multiple-seed results are reported anywhere. Observation 1 ("the simple MLP architecture achieves the best performance on most tasks") and the cross-adapter comparisons in Table 2 rely on single runs; some differences, such as TransE 91.01 vs R-GCN 44.46 on Entity-MC with MLP, are large, but others are small and could be noise. Similarly, the transferability and scalability analyses (Figures 4-6) are presented as qualitative plots without numerical support. The paper should report mean and standard deviation over at least three seeds (or otherwise justify that the evaluation is deterministic) before drawing relative-comparison conclusions.
  3. [§5.2, Observation 2] The claim that "SKP excels in coarse-grained reasoning tasks" conflates task granularity with the presence of textual cues. EG-MC requires selecting an entity type from an entity embedding alone, while TG-MC provides a textual relation name and SG-MC includes a subgraph with a missing core. The Appendix B.3 result for TransE shows that the TG-MC accuracy is largely attributable to text. The observation should be re-framed as an interaction between SKP and textual hints, or the w/o-text results should be integrated into the main comparison; otherwise the coarse-grained superiority claim is over-generalized.
  4. [§5.3.1 and §5.4] The transferability conclusion ("SKP does not exhibit strong transferability") and the scalability conclusion ("an MLP with 3-4 layers is strong enough") are based on figures that report no quantitative values, no variance, and no statistical tests. For a paper whose contribution is a systematic evaluation, these RQ2/RQ3 claims need to be backed by numbers (e.g., delta tables with significance tests) rather than visual inspection of line charts.
minor comments (7)
  1. [§3, sentence after Eq. (3)] The sentence "current approaches often this paradigm directly to specific downstream tasks" is missing the verb "apply"; it should read "often apply this paradigm directly."
  2. [§4.2.2, first sentence] "In this section, we we briefly introduce SUBARU" contains a duplicated "we."
  3. [Figure 2] The legend lists "Entity MC" twice; one of these should presumably be "Subgraph MC" to match the figure labels.
  4. [§5.6, text above Figure 3] The text says "As shown in Figure 3, we present a simple case" but the case study is presented in Table 3; the cross-reference should be corrected.
  5. [§5.1, Evaluation Protocol] "GPT-3.5-tubor" is a typo for "GPT-3.5-turbo."
  6. [Appendix B.1] "with a 3000 epoch training until coverage" should likely read "until convergence" rather than "coverage."
  7. [§5.2, Observation 1] "dominate in most of the colored cells" is vague; please state the exact counts (e.g., "MLP is the best adapter on 6 of 9 tasks") to make the observation precise.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical benchmark study whose conclusions rest on held-out evaluations, not on definitional equivalences or self-citation chains.

full rationale

This paper does not present a derivation in which an output is equivalent to its input by construction. SUBARU is an empirical benchmark, and the central conclusions (Observations 1-3 and the Section 6 claim that SKP provides coarse-grained information but lacks fine-grained factual awareness) are supported by measured accuracy, EM, BLEU, and GPTScore values on held-out test splits. The training loss in Eq. (4) is standard next-token prediction on training data, and evaluation is separate; no fitted parameter is renamed as a prediction. The paper's own prior works (KoPA, NeuralKG) are cited as sources of adapter and encoder implementations, not as evidence for the benchmark conclusions, so the self-citations are not load-bearing. The main validity concern is that TG-MC prompts still contain textual relation names and entity-name options, which may let the frozen LLM answer from parametric knowledge rather than from the SKP tokens alone. However, this is a construct-validity or leakage issue, not circularity: the paper acknowledges the textual shortcut in Appendix B.3 and even reports a control removing the text (e.g., TransE drops from 86.51 to 49.09 accuracy). Similarly, using GPT-3.5 to generate SG-DESC golden answers and then using GPTScore with the same model family is a methodological coupling, but it does not make the evaluation logically identical to the inputs. No equation is shown to reduce to another, no uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via citation. Accordingly, the paper is not circular; the noted issues are correctness and validity risks rather than circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No invented physical entities. The benchmark tasks are new data resources, not new scientific objects. The free parameters are design choices (negative sampling ratio, adapter depth) rather than fitted constants in a physical model, and the axioms are standard assumptions in KG+LLM evaluation, with the GPT-judge and CoDeX source being the main domain constraints.

free parameters (2)
  • Negative sampling ratio (1:1) = 1:1
    Used to construct CLS tasks; the balance is a design choice that affects task difficulty.
  • MLP depth (3-4 layers) = 3-4 for best scalability
    Found by tuning depth from 1 to 6 in Section 5.4; this is a fitted experimental choice, not a predicted law.
assumptions (4)
  • domain assumption Frozen LLM plus trainable adapter is a faithful implementation of the SKP paradigm
    Section 3 defines SKP with a frozen LLM and adapter; this is the accepted paradigm from GNP/KoPA, but it assumes the adapter can bridge the representation gap.
  • domain assumption Task accuracy on CLS/MC/DESC measures SKP generalization
    The benchmark assumes these 9 tasks capture granularity and difficulty; the paper does not validate against external SKP benchmarks.
  • domain assumption GPT-3.5-turbo generated gold descriptions and scores are reliable references
    Appendix A.1 uses GPT-3.5-turbo for SG DESC gold and the judge prompt in Figure 9; the authors manually verified 'acceptable quality'.
  • domain assumption CoDeX KG covers generalizable encyclopedic facts
    All tasks are built from CoDeX; conclusions about SKP generalization are bounded to this KG's distribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Have We Designed Generalizable Structural Knowledge Promptings? Systematic Evaluation and Rethinking." pith.science (2026). https://pith.science/paper/FCCQNZYQ

@misc{pith2026250100244,
  author       = {Pith},
  title        = {Pith review of: Have We Designed Generalizable Structural Knowledge Promptings? Systematic Evaluation and Rethinking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FCCQNZYQ}},
  note         = {Machine review of arXiv:2501.00244}
}
read the original abstract

Large language models (LLMs) have demonstrated exceptional performance in text generation within current NLP research. However, the lack of factual accuracy is still a dark cloud hanging over the LLM skyscraper. Structural knowledge prompting (SKP) is a prominent paradigm to integrate external knowledge into LLMs by incorporating structural representations, achieving state-of-the-art results in many knowledge-intensive tasks. However, existing methods often focus on specific problems, lacking a comprehensive exploration of the generalization and capability boundaries of SKP. This paper aims to evaluate and rethink the generalization capability of the SKP paradigm from four perspectives including Granularity, Transferability, Scalability, and Universality. To provide a thorough evaluation, we introduce a novel multi-granular, multi-level benchmark called SUBARU, consisting of 9 different tasks with varying levels of granularity and difficulty.

Figures

Figures reproduced from arXiv: 2501.00244 by the authors.

Figure 1
Figure 1. An intuition of the structural knowledge [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An overview of the construction pipeline of SUBARU benchmark and the evaluation process. In SUBARU, [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. A general prompt template for all tasks. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: The transferability experiments among differ [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 6
Figure 6. Figure 6: The scalability experiments on the entity / [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Experiments on more different LLMs. additional results about LlaMA2 and LlaMA3.1 are presented in Appendix B.4. 5.6 Case Study and Further Analysis In the three difficulty levels we designed in SUB￾ARU benchmark, the CLS and MC tasks provide clear answers and quantitat…
Figure 8
Figure 8. Figure 8: The prompt template used to generate the golden answer of SG DESC task. [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: The prompt template used for GPT evaluation. [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: The prompt template for EG-CLS. Entity Multiple Choice (EG-MC) <Structural Knowledge Prompting Se > ### Instruction: Given an entity embedding in the front and select the true answer. ### Question: Which is the possible type of entity? A. bell tower B. congestive hear…
Figure 11
Figure 11. Figure 11: The prompt template for EG-MC. Entity Description (EG-DESC) <Structural Knowledge Prompting Se > ### Instruction: Given an entity embedding in the front and describe this entity. ### Output: Bonnie Owens. ### Dessctiption: Bonnie Owens , born Bonnie Campbell, was an A…
Figure 12
Figure 12. Figure 12: The prompt template for EG-DESC. Triple Classification (TG-CLS) <Structural Knowledge Prompting Sh, Sr, St > ### Instruction: Given the embeddings of an knowledge triple in the front. Please answer the following question. ### Question: Is this a positive triple? ### O…
Figure 13
Figure 13. Figure 13: The prompt template for TG-CLS. Triple Multiple Choice (TG-MC) <Structural Knowledge Prompting Sr, St > # Note that the masked answer does not appear in SKP ### Instruction: Given the embeddings of a query and four candidates in the front. Select a correct answer to f…
Figure 14
Figure 14. Figure 14: The prompt template for TG-MC. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_14.png]
Figure 15
Figure 15. Figure 15: The prompt template for TG-DESC. Subgraph Classification (SG-CLS) <Structural Knowledge Prompting Sh, Sr1 , St1 , · · · , Srk , Stk > ### Instruction: Given embeddings from a subgraph in the front and answer the following question. ### Question: Is there any anomaly i…
Figure 16
Figure 16. Figure 16: The prompt template for SG-CLS. Subgraph Multiple Choice (SG-MC) <Structural Knowledge Prompting Sr1 , St1 , · · · , Srk , Stk > # The center entity does not appear. ### Instruction: Given an entity embedding in the front and select the true answer. ### Question: Whic…
Figure 17
Figure 17. Figure 17: The prompt template for SG-MC. Subgraph Description (SG-DESC) <Structural Knowledge Prompting Sh, Sr1 , St1 , · · · , Srk , Stk > ### Instruction: Given embeddings from a subgraph in the front and answer the following question. ### Output: Dominic Howard is a musician…
Figure 18
Figure 18. Figure 18: The prompt template for SG-DESC. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_18.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Abstractive Visual Understanding of Multi-modal Structured Knowledge: A New Perspective for MLLM Evaluation

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A new benchmark, M3STR, renders knowledge-graph subgraphs as images and shows current MLLMs score near random on anomaly detection and poorly on entity counting.

Reference graph

Works this paper leans on

35 extracted references · 14 canonical work pages · cited by 1 Pith paper

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Antoine Bordes, Nicolas Usunier, Alberto Garc \' a - Dur \' a n, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In NIPS , pages 2787--2795

    work page 2013

  4. [4]

    Yuxin Chen, Junfei Tan, An Zhang, Zhengyi Yang, Leheng Sheng, Enzhi Zhang, Xiang Wang, and Tat - Seng Chua. 2024. On softmax direct preference optimization for recommendation. CoRR, abs/2406.09215

    arXiv 2024

  5. [5]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    arXiv 2024

  6. [6]

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Qianyu Guo, Meng Wang, and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey. CoRR, abs/2312.10997

    arXiv 2023

  7. [7]

    Xavier Glorot, Antoine Bordes, and Yoshua Bengio. 2011. Deep sparse rectifier neural networks. In AISTATS , volume 15 of JMLR Proceedings , pages 315--323. JMLR.org

    work page 2011

  8. [8]

    Lingbing Guo, Zhongpu Bo, Zhuo Chen, Yichi Zhang, Jiaoyan Chen, Yarong Lan, Mengshu Sun, Zhiqiang Zhang, Yangyifei Luo, Qian Li, Qiang Zhang, Wen Zhang, and Huajun Chen. 2024 a . Mkgl: Mastery of a three-word language

    work page 2024

Show all 35 references
  1. [9]

    Zirui Guo, Lianghao Xia, Yanhua Yu, Tu Ao, and Chao Huang. 2024 b . Lightrag: Simple and fast retrieval-augmented generation. CoRR, abs/2410.05779

    2024 arXiv

  2. [10]

    Bernal Jim \' e nez Guti \' e rrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su. 2024. Hipporag: Neurobiologically inspired long-term memory for large language models. CoRR, abs/2405.14831

    2024 arXiv

  3. [11]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825

    2023 arXiv

  4. [12]

    Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, Kai Shu, Lu Cheng, and Huan Liu. 2024. From generation to judgment: Opportunities and challenges of llm-as-a-judge. arXiv preprint a...

    2024

  5. [13]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven C. H. Hoi. 2023. BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML , volume 202 of Proceedings of Machine Learning Research, pages 19730--19742. PMLR

    2023

  6. [14]

    Ke Liang, Lingyuan Meng, Meng Liu, Yue Liu, Wenxuan Tu, Siwei Wang, Sihang Zhou, Xinwang Liu, Fuchun Sun, and Kunlun He. 2024. A survey of knowledge graph reasoning on graph types: Static, dynamic, and multi-modal. IEEE Transactions on Pattern Analysis and Machine Intelligence

    2024

  7. [15]

    Youwei Liang, Ruiyi Zhang, li Zhang, and Pengtao Xie. 2023. Drugchat: Towards enabling chatgpt-like capabilities on drug molecule graphs. TechRxiv

    2023

  8. [16]

    Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In ICLR (Poster) . OpenReview.net

    2019

  9. [17]

    Yougang Lyu, Lingyong Yan, Shuaiqiang Wang, Haibo Shi, Dawei Yin, Pengjie Ren, Zhumin Chen, Maarten de Rijke, and Zhaochun Ren. 2024. Knowtuning: Knowledge-aware fine-tuning for large language models. In EMNLP , pages 14535--14556. Association for Computational Linguistics

    2024

  10. [18]

    Qiyao Ma, Xubin Ren, and Chao Huang. 2024. Xrec: Large language models for explainable recommendation. In EMNLP (Findings) , pages 391--402. Association for Computational Linguistics

    2024

  11. [19]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei - Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In ACL , pages 311--318. ACL

    2002

  12. [20]

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32

    2019

  13. [21]

    Tara Safavi and Danai Koutra. 2020. Codex: A comprehensive knowledge graph completion benchmark. In EMNLP (1) , pages 8328--8350. Association for Computational Linguistics

    2020

  14. [22]

    Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling

    Michael Sejr Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In ESWC , volume 10843 of Lecture Notes in Computer Science, pages 593--607. Springer

    2018

  15. [23]

    Zhiqing Sun, Zhi - Hong Deng, Jian - Yun Nie, and Jian Tang. 2019. Rotate: Knowledge graph embedding by relational rotation in complex space. In ICLR (Poster) . OpenReview.net

    2019

  16. [24]

    Chawla, and Panpan Xu

    Yijun Tian, Huan Song, Zichen Wang, Haozhu Wang, Ziqing Hu, Fang Wang, Nitesh V. Chawla, and Panpan Xu. 2024. Graph neural prompting with large language models. In AAAI , pages 19080--19088. AAAI Press

    2024

  17. [25]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton - Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu,...

    2023 arXiv

  18. [26]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 20...

    2017

  19. [27]

    Denny Vrandecic and Markus Kr \" o tzsch. 2014. Wikidata: a free collaborative knowledgebase. Commun. ACM , 57(10):78--85

    2014

  20. [28]

    Yilin Wen, Zifeng Wang, and Jimeng Sun. 2024. Mindmap: Knowledge graph prompting sparks graph of thoughts in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, ...

    2024

  21. [29]

    Bishan Yang, Wen - tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding entities and relations for learning and inference in knowledge bases. In ICLR (Poster)

    2015

  22. [30]

    Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2023. A survey on multimodal large language models. CoRR, abs/2306.13549

    2023 arXiv

  23. [31]

    Wen Zhang, Xiangnan Chen, Zhen Yao, Mingyang Chen, Yushan Zhu, Hongtao Yu, Yufeng Huang, Yajing Xu, Ningyu Zhang, Zezhong Xu, Zonggang Yuan, Feiyu Xiong, and Huajun Chen. 2022. Neuralkg: An open source library for diverse representation learning of knowledge graphs. In SIGIR ,...

    2022

  24. [32]

    Yichi Zhang, Zhuo Chen, Yin Fang, Yanxi Lu, Fangming Li, Wen Zhang, and Huajun Chen. 2024 a . Knowledgeable preference alignment for llms in domain-specific question answering. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtua...

    2024

  25. [33]

    Yichi Zhang, Zhuo Chen, Lingbing Guo, Yajing Xu, Wen Zhang, and Huajun Chen. 2024 b . Making large language models perform better in knowledge graph completion. In ACM Multimedia , pages 233--242. ACM

    2024

  26. [34]

    Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi. 2023. Siren's song in the AI ocean: A survey on hallucination in large language models. CoRR, abs/...

    2023 arXiv

  27. [35]

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian - Yun Nie, and Ji - Ro...

    2023 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.