Pith. sign in

REVIEW 3 major objections 6 minor 51 references

K-ON: Stacking Knowledge On the Head Layer of Large Language Model

T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read K-ON claims that adding K parallel prediction heads to a single LLM head lets the model complete knowledge-graph triples in one step, scoring entire entities instead of one token at a time.

desk verdict A clever and well-ablated head-stacking architecture for LLM-based KG completion, but the headline claim that the head layer is what drives the gains needs a control experiment that is currently missing. read the letter →

arxiv 2502.06257 v1 pith:NPISAJB4 submitted 2025-02-10 cs.CL cs.AI

Lingbing Guo , Yichi Zhang , Zhongpu Bo , Zhuo Chen , Mengshu Sun , Zhiqiang Zhang , Wen Zhang , Huajun Chen This is my paper · ORCID
classification cs.CLcs.AI
keywords knowledgegraphcompletionlargelanguagemodelsmulti-headpredictionentity-levelcontrastivelearningnext-k-tokenheadtrajectorytuninglinkLLMfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

K-ON claims that a large language model can be taught to complete knowledge-graph triples by replacing its single next-token head with K parallel heads that emit the first K tokens of an entity name in one step. The paper argues this fixes a granularity mismatch: knowledge graphs reason over entities, while LLMs generate tokens one at a time. With an entity-level contrastive loss computed over the K token probabilities, the model can score whole candidate entities at once, and the authors report that this beats methods that use text or image side information on DB15K and MKGW. If true, the result would mean entity-level reasoning can be added to an LLM cheaply at the output head, without changing the base model.

What carries the argument

The central mechanism is the K-ON head stack: K head MLPs transform the final hidden state of the query text into K hidden states, a tiny causal Transformer with a causal mask re-establishes dependencies among those K steps, LoRA-adapted score layers produce K token probability distributions, and a weighted sum of the gathered token probabilities gives a scalar entity score for contrastive learning. The K-step gathering step converts every entity name to a fixed-length token sequence of length K, so all candidate entities can be scored in parallel from the same K probability distributions.

What would settle it

Run K-ON on a KG completion dataset where many distinct entities share the same first K tokens (for example, heavily prefixed entity names with long distinguishing suffixes) and compare its MRR against an autoregressive LLM baseline: if the K=8 truncation causes distinct entities to collapse, K-ON's accuracy should drop sharply while the autoregressive baseline does not.

Watch

Extended reading notes

Core claim

The paper's central claim is that the head layer of an LLM is sufficient for injecting knowledge-graph structure, without altering the transformer body. K-ON attaches K independent head MLPs to the final hidden state, each predicting one token of the target entity name, then uses a small causal attention module to reintroduce the sequential dependence that a standard autoregressive model would have. The K probability distributions are gathered into per-entity scores, which support an entity-level contrastive loss that treats the whole token sequence as an atomic unit. Three losses are minimized jointly: supervised fine-tuning on the next-K tokens, token distribution tuning that aligns K-ON's distributions with the original LLM head via KL divergence, and the entity contrastive loss. The authors report that this outperforms state-of-the-art structure, text, and multimodal KG completion baselines on DB15K and MKGW.

Load-bearing premise

The load-bearing premise is that the final hidden state of the query text contains enough information to predict every token of the target entity name, including later tokens that in normal autoregressive generation would be conditioned on earlier generated tokens. A second fragile assumption is that every entity name can be truncated or padded to exactly K tokens without losing identity.

Editorial extensions

If this is right

  • If K-ON works as claimed, LLM fine-tuning for KG completion no longer needs to enumerate candidate entities inside the prompt or generate tokens autoregressively; one forward pass yields scores for all entities at once.
  • Entity-level contrastive learning, long a staple of KG embedding methods, can be ported directly onto an LLM's probability outputs, giving LLM-based KG models a training signal that token-level cross-entropy lacks.
  • The reported reduction from roughly 1,000 training epochs to 5 on DB15K suggests that head-layer adaptation can make LLM-based KG completion computationally practical for large knowledge graphs.
  • The modular design means the base LLM can be swapped without redesigning the K-ON heads, as long as the new model provides a hidden state of matching dimension.
  • Helping an LLM identify entities in one step could extend beyond KG completion to any task where the output is a known set of multi-token names, such as linking mentions to a fixed entity catalog.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper leaves implicit: if K-ON works because the final hidden state contains enough information for all K tokens, then the same head-stacking idea could be applied to other structured-output tasks such as relation extraction or slot filling, where the output vocabulary is a known closed set.
  • The paper's choice of weighted addition over multiplication for the joint probability hints that treating token probabilities as log-additive scores, rather than true joint probabilities, is what makes the contrastive loss well-behaved; this could be tested directly by comparing learnable log-sum-exp aggregation against the reported operators.
  • The truncation assumption that every entity fits in K tokens suggests a concrete failure mode: a dataset with many long, similar entity names should break K-ON because distinct entities collapse to the same truncated sequence. Measuring performance as a function of name-length distribution would isolate this effect.
  • Head trajectory tuning is a general alignment idea: it could be applied to any multi-token prediction head, not just KG entities, to keep the adapted model's token distributions close to the original LLM and therefore preserve general language ability.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes K-ON, a method for knowledge graph completion that attaches K parallel prediction heads to the head layer of a large language model (Llama-2-chat-7B). Each head predicts one token of the target entity, enabling entity-level probability estimation in a single forward pass. The method is trained with a supervised fine-tuning loss, a token-distribution alignment loss (head trajectory tuning, HTT), and an entity-level contrastive loss. Experiments on DB15K and MKGW report state-of-the-art MRR and Hits@k against structure-only, text-only, and multimodal baselines, and ablations show large drops when the entity-level contrastive loss or the multiple head MLPs are removed.

Significance. If the claims hold, K-ON offers a practical way to reconcile the token-level granularity of LLMs with the entity-level granularity of knowledge graphs, and it demonstrates that entity-level contrastive learning can be applied efficiently by reusing the K head output distributions. The paper includes useful analyses of the number of heads K, the number of negative entities, and the joint probability operator. However, the central claim — that the head-stacking architecture is responsible for the gains — is not established without a control that fine-tunes the same backbone with standard next-token prediction. The paper's contrastive loss, as written, also appears theoretically problematic. These issues are load-bearing for the main conclusion, so the contribution is not yet ready for acceptance.

major comments (3)
  1. [Experiment, Table 2] No baseline fine-tunes the same Llama-2-chat-7B backbone with standard next-token prediction (e.g., LoRA SFT) and evaluates it on the same filtered ranking task. All reported LLM baselines (KG-Llama, GPT-3.5) are not fine-tuned on the KG completion objective. Without this control, the reported gains over non-LLM baselines could be due largely to the pretrained 7B model plus ordinary LoRA tuning rather than to the K-ON head-stacking architecture. The ablations in Table 3 remove components of K-ON but never remove the K-ON heads themselves; both 'Shared Head MLP' and 'Shared Score Layer' still use K parallel predictions. Add a plain LoRA-fine-tuned Llama-2-chat-7B baseline (single head, next-token objective) and report its MRR/Hits@1 to isolate the contribution of the head-stacking mechanism.
  2. [Contrastive Loss, Eq. (12)] The entity-level contrastive loss is written as LNCE(e) = −log p_e + (1/|N|) Σ_{ej∈N} log p_ej. This is not a standard contrastive loss: for a negative entity with high probability, log p_ej is close to 0, so the penalty term is near zero; for a low-probability negative, log p_ej is very negative, making the loss unbounded below. Thus the loss fails to strongly penalize false positives and may cause training instability. The same issue appears in the supervised fine-tuning loss in Eq. (13). Since the ablation in Table 3 attributes a large performance drop to removing Lnce, this formulation is load-bearing. Please clarify whether Eq. (12) is the actual implemented loss or a simplified notation; if it is exact, justify it theoretically or replace it with a bounded softmax-based contrastive objective.
  3. [K-step Gathering, Eq. (8)-(10)] Entities are padded/truncated to exactly K=8 tokens. Two different entities whose names share the first 8 tokens after tokenization become indistinguishable because the extraction in Eq. (10) uses the same probability values for both. The paper does not report how many entities are affected on DB15K and MKGW, nor whether this truncation causes rank ambiguity in the filtered evaluation. Provide statistics on token-length distribution and collision rates, and discuss the impact on the reported metrics.
minor comments (6)
  1. [Methodology, Eq. (1)] The notation hm_0:N−1 is defined, but Eq. (3) and later text use hm_0 as the last hidden state; clarify the indexing to avoid confusion.
  2. [Methodology, Eq. (5)] Spelling: 'causual mask' should be 'causal mask'.
  3. [Experiment, Table 2] The KGLM row on MKGW reports Hits@10 as '.46.62'; this appears to be a typographical error and should be 46.62.
  4. [Experiment, Table 3] The entry 'w/o Lsf t' contains an extra space; rename to 'w/o Lsft' for consistency with the text.
  5. [Experiment, Setting] The paper states that training epochs are reduced from 1,000 to 5 compared with conventional methods, but the number of epochs for K-ON and for each baseline is not reported in the tables; specify these values to support the efficiency claim.
  6. [Analysis on the K-ON Heads] The text refers to 'the number of K-ON head layers k' with lowercase k, while the hyper-parameter is K elsewhere; use consistent capitalization throughout.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: K-ON is an end-to-end supervised method evaluated on external KG-completion benchmarks; no prediction reduces by construction to a fitted input or self-citation.

full rationale

The paper's central claim is empirical: K-ON attaches K prediction heads to Llama-2-chat-7B and trains them with a combination of token-level SFT (Eq. 13), KL alignment with the original head (Eq. 14), and an entity-level contrastive loss (Eq. 12). The entity score p_e is defined as a learnable weighted sum of K token probabilities (Eq. 11), then optimized against ground-truth triples from DB15K and MKGW and evaluated with filtered MRR/Hits@K. This is a standard supervised scoring model, not a quantity defined in terms of the fitted parameters or a derived constant. The benchmark results are compared against external non-LLM, multi-modal, and LLM baselines, so the claimed improvement is externally falsifiable. The paper cites prior multi-head work (Medusa, MultiToken) and the authors' own KG papers, but none of these citations carries the load of the derivation: the architecture and losses are specified in the paper, and no uniqueness theorem or unverified self-citation is invoked to force the design. The missing plain-LoRA control noted in the skeptic pass is an experimental-comparison gap (correctness risk), not circularity, because the paper does not define success in terms of K-ON's own components. Accordingly, no circular step can be exhibited with a specific equation reducing to its input.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on a supervised learning pipeline with hyperparameters (K, |N|) and learned weights (alpha_k). The main assumptions concern entity tokenization, the information content of the query hidden state, and the ability of the conditional attention module to reconstruct sequential dependency. No new particles, forces, or entities are introduced.

free parameters (3)
  • K (number of head layers) = 8
    Chosen based on experiments showing performance saturation for K>=8; trade-off between accuracy and computational cost (Figure 3).
  • |N| (number of negative entities) = 128
    Selected by analyzing the effect of negative sample count; performance peaks around 128 while training time stays stable (Figure 4).
  • alpha_k (learnable weights for joint probability) = learned during training
    Used to aggregate token probabilities into an entity score; these weights are trained jointly and shared across entities (Eq. 11).
assumptions (3)
  • domain assumption Entity names can be tokenized into at most K tokens without losing identity
    All entities are padded or truncated to a fixed length K=8. Truncation can cause distinct entities to map to the same token sequence, and padding reduces discriminative signal for short names. This is stated in Eq. (8).
  • domain assumption The last hidden state of the LLM for the query text encodes sufficient information to predict the target entity
    All K head MLPs take hm_0 as input (Eq. 3). If the query representation does not contain enough information about the target entity, the multi-head predictions cannot succeed. This is the core representational assumption.
  • ad hoc to paper The conditional attention module with causal masking can reconstruct the sequential dependencies between tokens
    The small Transformer Ms is introduced to mimic autoregressive conditioning (Eq. 4-5). This is an architectural design choice validated only by ablation (Table 3), not by a formal guarantee.

how reviews work

0 comments
Cite this review

Pith. "Pith review of K-ON: Stacking Knowledge On the Head Layer of Large Language Model." pith.science (2026). https://pith.science/paper/NPISAJB4

@misc{pith2026250206257,
  author       = {Pith},
  title        = {Pith review of: K-ON: Stacking Knowledge On the Head Layer of Large Language Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NPISAJB4}},
  note         = {Machine review of arXiv:2502.06257}
}
read the original abstract

Recent advancements in large language models (LLMs) have significantly improved various natural language processing (NLP) tasks. Typically, LLMs are trained to predict the next token, aligning well with many NLP tasks. However, in knowledge graph (KG) scenarios, entities are the fundamental units and identifying an entity requires at least several tokens. This leads to a granularity mismatch between KGs and natural languages. To address this issue, we propose K-ON, which integrates KG knowledge into the LLM by employing multiple head layers for next k-step prediction. K-ON can not only generate entity-level results in one step, but also enables contrastive loss against entities, which is the most powerful tool in KG representation learning. Experimental results show that K-ON outperforms state-of-the-art methods that incorporate text and even the other modalities.

Figures

Figures reproduced from arXiv: 2502.06257 by the authors.

Figure 1
Figure 1. A comparison of single-step prediction and the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the K-ON architecture. From left to right: (1) The LLM processes the input text containing incomplete [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Performance of K-ON w.r.t. the number of K-ON [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Performance of K-ON w.r.t. the number of negative entities. The results are obtained using [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 30 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al

    Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  4. [4]

    Bordes, A.; Usunier, N.; Garcia-Dur\' a n, A.; Weston, J.; and Yakhnenko, O. 2013. Translating embeddings for modeling multi-relational data. In NIPS, 2787--2795

    work page 2013

  5. [5]

    D.; Chen, D.; and Dao, T

    Cai, T.; Li, Y.; Geng, Z.; Peng, H.; Lee, J. D.; Chen, D.; and Dao, T. 2024. Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads. CoRR, abs/2401.10774

    arXiv 2024

  6. [6]

    Cao, Z.; Xu, Q.; Yang, Z.; He, Y.; Cao, X.; and Huang, Q. 2022. OTKGE : Multi -modal Knowledge Graph Embeddings via Optimal Transport . In NeurIPS

    work page 2022

  7. [7]

    Chen, S.; Liu, X.; Gao, J.; Jiao, J.; Zhang, R.; and Ji, Y. 2021. HittER: Hierarchical Transformers for Knowledge Graph Embeddings. In EMNLP, 10395--10407

    work page 2021

  8. [8]

    MEAformer: Multi-modal Entity Alignment Transformer for Meta Modality Hybrid

    Chen, Z.; Chen, J.; Zhang, W.; Guo, L.; Fang, Y.; Huang, Y.; Geng, Y.; Pan, J. Z.; Song, W.; and Chen, H. 2022. MEAformer: Multi-modal Entity Alignment Transformer for Meta Modality Hybrid. arXiv preprint arXiv:2212.14454

    work page Pith review arXiv 2022

Show all 51 references
  1. [9]

    Z.; Zhang, N.; and Chen, H

    Chen, Z.; Zhang, Y.; Fang, Y.; Geng, Y.; Guo, L.; Chen, X.; Li, Q.; Zhang, W.; Chen, J.; Zhu, Y.; Li, J.; Liu, X.; Pan, J. Z.; Zhang, N.; and Chen, H. 2024. Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey. CoRR, abs/2402.05391

    2024 arXiv

  2. [10]

    Dettmers, T.; Minervini, P.; Stenetorp, P.; and Riedel, S. 2018. Convolutional 2D knowledge graph embeddings. In AAAI, 1811--1818

    2018

  3. [11]

    Dong, Q.; Li, L.; Dai, D.; Zheng, C.; Wu, Z.; Chang, B.; Sun, X.; Xu, J.; and Sui, Z. 2022. A survey on in-context learning. arXiv preprint arXiv:2301.00234

    2022 arXiv

  4. [12]

    Elfwing, S.; Uchibe, E.; and Doya, K. 2018. Sigmoid-weighted linear units for neural network function approximation in reinforcement learning. Neural Networks, 107: 3--11

    2018

  5. [13]

    Y.; Rozi \` e re, B.; Lopez - Paz, D.; and Synnaeve, G

    Gloeckle, F.; Idrissi, B. Y.; Rozi \` e re, B.; Lopez - Paz, D.; and Synnaeve, G. 2024. Better & Faster Large Language Models via Multi-token Prediction. CoRR, abs/2404.19737

    2024 arXiv

  6. [14]

    Guan, X.; Liu, Y.; Lin, H.; Lu, Y.; He, B.; Han, X.; and Sun, L. 2024. Mitigating large language model hallucinations via autonomous knowledge graph-based retrofitting. In AAAI, 18126--18134

    2024

  7. [15]

    Guo, L.; Bo, Z.; Chen, Z.; Zhang, Y.; Chen, J.; Lan, Y.; Sun, M.; Zhang, Z.; Luo, Y.; Li, Q.; Zhang, Q.; Zhang, W.; and Chen, H. 2024 a . MKGL: Mastery of a Three-Word Language. CoRR, abs/2410.07526

    2024 arXiv

  8. [16]

    Guo, L.; Chen, Z.; Chen, J.; Fang, Y.; Zhang, W.; and Chen, H. 2024 b . Revisit and Outstrip Entity Alignment: A Perspective of Generative Models. In ICLR . OpenReview.net

    2024

  9. [17]

    Guo, L.; Sun, Z.; and Hu, W. 2019. Learning to Exploit Long-term Relational Dependencies in Knowledge Graphs. In ICML , 2505--2514

    2019

  10. [18]

    Guo, L.; Wang, W.; Sun, Z.; Liu, C.; and Hu, W. 2020. Decentralized Knowledge Graph Representation Learning. CoRR, abs/2010.08114

    2020 arXiv

  11. [19]

    Guo, L.; Zhang, Q.; and Chen, H. 2022. Unleashing the Power of Transformer for Graphs. CoRR, abs/2202.10581

    2022 arXiv

  12. [20]

    Guo, L.; Zhang, Q.; Sun, Z.; Chen, M.; Hu, W.; and Chen, H. 2022. Understanding and Improving Knowledge Graph Embedding for Entity Alignment. In ICML , volume 162 of Proceedings of Machine Learning Research, 8145--8156. PMLR

    2022

  13. [21]

    J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W

    Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685

    2021 arXiv

  14. [22]

    P.; and Ba, J

    Kingma, D. P.; and Ba, J. 2015. Adam: A Method for stochastic optimization. In ICLR

    2015

  15. [23]

    Kullback, S.; and Leibler, R. A. 1951. On information and sufficiency. The annals of mathematical statistics, 22(1): 79--86

    1951

  16. [24]

    Lee, J.; Chung, C.; Lee, H.; Jo, S.; and Whang, J. J. 2023. VISTA : Visual - Textual Knowledge Graph Representation Learning . In EMNLP ( Findings ) , 7314--7328. Association for Computational Linguistics

    2023

  17. [25]

    Lin, Q.; Mao, R.; Liu, J.; Xu, F.; and Cambria, E. 2023. Fusing topology contexts and logical rules in language models for knowledge graph completion. Information Fusion, 90: 253--264

    2023

  18. [26]

    Liu, Y.; Li, H.; Garc \' a - Dur \' a n, A.; Niepert, M.; O \ n oro - Rubio, D.; and Rosenblum, D. S. 2019. MMKG: Multi-modal Knowledge Graphs. In ESWC , volume 11503 of Lecture Notes in Computer Science, 459--474. Springer

    2019

  19. [27]

    Lu, X.; Wang, L.; Jiang, Z.; He, S.; and Liu, S. 2022. MMKRL: A robust embedding approach for multi-modal knowledge graph representation learning. Appl. Intell., 52(7): 7480--7497

    2022

  20. [28]

    Z.; Razniewski, S.; Kalo, J.-C.; Singhania, S.; Chen, J.; Dietze, S.; Jabeen, H.; Omeliyanenko, J.; Zhang, W.; Lissandrini, M.; et al

    Pan, J. Z.; Razniewski, S.; Kalo, J.-C.; Singhania, S.; Chen, J.; Dietze, S.; Jabeen, H.; Omeliyanenko, J.; Zhang, W.; Lissandrini, M.; et al. 2023. Large language models and knowledge graphs: Opportunities and challenges. arXiv preprint arXiv:2308.06374

    2023 arXiv

  21. [29]

    Pan, S.; Luo, L.; Wang, Y.; Chen, C.; Wang, J.; and Wu, X. 2024. Unifying large language models and knowledge graphs: A roadmap. IEEE Transactions on Knowledge and Data Engineering

    2024

  22. [30]

    Sun, Z.; Deng, Z.-H.; Nie, J.-Y.; and Tang, J. 2019. RotatE: Knowledge graph embedding by relational rotation in complex space. In ICLR

    2019

  23. [31]

    Touvron, H.; Martin, L.; Stone, K.; Albert, P.; Almahairi, A.; Babaei, Y.; Bashlykov, N.; Batra, S.; Bhargava, P.; Bhosale, S.; et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  24. [32]

    Vashishth, S.; Sanyal, S.; Nitin, V.; and Talukdar, P. P. 2020. Composition-based Multi-Relational Graph Convolutional Networks. In ICLR

    2020

  25. [33]

    Wang, B.; Shen, T.; Long, G.; Zhou, T.; Wang, Y.; and Chang, Y. 2021. Structure-augmented text representation learning for efficient knowledge graph completion. In The Web Conference, 1737--1748

    2021

  26. [34]

    Wang, X.; Meng, B.; Chen, H.; Meng, Y.; Lv, K.; and Zhu, W. 2023. TIVA-KG: A Multimodal Knowledge Graph with Text, Image, Video and Audio. In ACM Multimedia , 2391--2399. ACM

    2023

  27. [35]

    Wang, Z.; Li, L.; Li, Q.; and Zeng, D. 2019. Multimodal Data Enhanced Representation Learning for Knowledge Graphs . In IJCNN , 1--8. IEEE

    2019

  28. [36]

    T.; and Zhang, Y

    Wei, Y.; Huang, Q.; Kwok, J. T.; and Zhang, Y. 2024. KICGPT: Large Language Model with Knowledge in Context for Knowledge Graph Completion. arXiv preprint arXiv:2402.02389

    2024 arXiv

  29. [37]

    Wies, N.; Levine, Y.; and Shashua, A. 2024. The learnability of in-context learning. NeurIPS, 36

    2024

  30. [38]

    Xie, R.; Liu, Z.; Luan, H.; and Sun, M. 2017. Image-embodied Knowledge Representation Learning . In IJCAI , 3140--3146. ijcai.org

    2017

  31. [39]

    Xu, D.; Xu, T.; Wu, S.; Zhou, J.; and Chen, E. 2022. Relation-enhanced Negative Sampling for Multimodal Knowledge Graph Completion . In ACM Multimedia , 3857--3866. ACM

    2022

  32. [40]

    Yang, B.; Yih, W.; He, X.; Gao, J.; and Deng, L. 2015. Embedding entities and relations for learning and inference in knowledge bases. In ICLR

    2015

  33. [41]

    Yang, L.; Chen, H.; Li, Z.; Ding, X.; and Wu, X. 2023. Chatgpt is not enough: Enhancing large language models with knowledge graphs for fact-aware language modeling. arXiv preprint arXiv:2306.11489

    2023 arXiv

  34. [42]

    Yao, L.; Mao, C.; and Luo, Y. 2019. KG-BERT: BERT for knowledge graph completion. arXiv preprint arXiv:1909.03193

    2019 arXiv

  35. [43]

    Yao, L.; Peng, J.; Mao, C.; and Luo, Y. 2023. Exploring large language models for knowledge graph completion. arXiv preprint arXiv:2308.13916

    2023 arXiv

  36. [44]

    Youn, J.; and Tagkopoulos, I. 2022. Kglm: Integrating knowledge graph structure in language models for link prediction. arXiv preprint arXiv:2211.02744

    2022 arXiv

  37. [45]

    Zhang, B.; and Sennrich, R. 2019. Root Mean Square Layer Normalization. In NeurIPS, 12360--12371

    2019

  38. [46]

    Zhang, Y.; Chen, M.; and Zhang, W. 2023. Modality-Aware Negative Sampling for Multi-modal Knowledge Graph Embedding. In IJCNN , 1--8. IEEE

    2023

  39. [47]

    Zhang, Y.; Chen, Z.; Guo, L.; Xu, Y.; Hu, B.; Liu, Z.; Zhang, W.; and Chen, H. 2024 a . Mixture of Modality Knowledge Experts for Robust Multi-modal Knowledge Graph Completion. CoRR, abs/2405.16869

    2024 arXiv

  40. [48]

    Zhang, Y.; Chen, Z.; Guo, L.; Xu, Y.; Hu, B.; Liu, Z.; Zhang, W.; and Chen, H. 2024 b . NativE: Multi-modal Knowledge Graph Completion in the Wild. In SIGIR , 91--101. ACM

    2024

  41. [49]

    Zhang, Y.; Chen, Z.; Liang, L.; Chen, H.; and Zhang, W. 2024 c . Unleashing the Power of Imbalanced Modality Information for Multi-modal Knowledge Graph Completion. CoRR, abs/2402.15444

    2024 arXiv

  42. [50]

    Zhang, Y.; Chen, Z.; Zhang, W.; and Chen, H. 2023. Making large language models perform better in knowledge graph completion. arXiv preprint arXiv:2310.06671

    2023 arXiv

  43. [51]

    Zhu, Y.; Wang, X.; Chen, J.; Qiao, S.; Ou, Y.; Yao, Y.; Deng, S.; Chen, H.; and Zhang, N. 2023. Llms for knowledge graph construction and reasoning: Recent capabilities and future opportunities. arXiv preprint arXiv:2305.13168

    2023 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.