Pith. sign in

REVIEW 5 major objections 5 minor 81 references

Dark Side of Modalities: Reinforced Multimodal Distillation for Multimodal Knowledge Graph Reasoning

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A unimodal student trained on soft labels from a reinforcement-selected subset of multimodal teachers beats adaptive-fusion MKGR baselines on five datasets.

desk verdict The neighbor-decoupled distillation is the real contribution; the reinforced teacher selection is under-justified and adds only a small margin, but the paper deserves a serious read. read the letter →

arxiv 2507.20738 v1 pith:FFKMDF6B submitted 2025-07-28 cs.MM

Yu Zhao , Ying Zhang , Xuhui Sui , Baohang Zhou , Haoze Zhu , Jeff Z. Pan , Xiaojie Yuan This is my paper
classification cs.MM
keywords multimodalknowledgegraphreasoningdistillationreinforcementlearningteacherselectionsoftlabelsentitylabelcorrelationscompletion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that multimodal knowledge graph reasoning is best learned not by fusing all modalities into one model, but by training a plain unimodal student to imitate soft label distributions from pre-trained structural, visual, and textual teachers. The soft labels carry 'dark knowledge': correlations between the target entity and other plausible answers, and between distractors, which one-hot cross-entropy discards. The paper further claims that, for each training triple, some modalities are actively misleading, so a reinforcement-trained agent should discretely choose which teachers to keep rather than down-weight the bad ones. On five MKGR benchmarks (DB15K, MKG-W, MKG-Y, FB15K-237, WN18) the resulting DSoM student reports state-of-the-art MRR, Hits@{1,3,10}, and Mean Rank.

What carries the argument

Two coupled mechanisms carry the argument. The first is a reinforced teacher-combination policy: an MLP agent that sees the concatenated score distributions of structural, visual, and textual teachers, samples a subset from the seven possible teacher combinations, and is trained by a REINFORCE-style policy gradient whose reward compares the selected teachers' cross-entropy against the student's. The second is neighbor-decoupled knowledge distillation, an extension of decoupled KD that splits temperature-scaled teacher and student logits into an average binary probability over all neighbor entities (true answers) and a distribution over non-neighbor entities, so the student learns both types of label correlation.

What would settle it

Compare the learned policy's teacher choices on held-out triples with the teacher subset that actually minimizes cross-entropy on those triples; if agreement is near chance, or if replacing the learned agent with a static per-triple best-teacher-by-validation-CE heuristic reproduces the same MRR, the reinforcement component is not doing the claimed work.

Watch

Extended reading notes

Core claim

The paper introduces DSoM, a framework that replaces one-hot hard-label training of a multimodal knowledge graph reasoning model with a distillation routine in which a unimodal ComplEx student learns to imitate soft label distributions emitted by separately pre-trained structural, visual, and textual teacher models. A small policy network receives the teachers' score distributions as state and samples a subset of teachers for each triple, receiving a positive or negative reward depending on whether the averaged chosen teacher logits beat the student's own cross-entropy on that triple. The student is then trained with hard-label cross-entropy plus a neighbor-decoupled knowledge distillation loss that splits teacher and student logits into neighbor and non-neighbor entity groups, so the student learns both correlations among true answers and correlations among distractors. The reported result is consistent gains over adaptive-fusion and ensemble baselines, with the unimodal student reaching 42.68 MRR on DB15K versus the previous best 37.72.

Load-bearing premise

The agent is taught only by whether the averaged chosen teachers have lower loss on a training triple than the student currently does; if that signal tracks the student's memorization instead of genuine modality quality, the selection mechanism and the reported gains could come from the co-training loop itself.

Editorial extensions

If this is right

  • A unimodal student can carry multimodal knowledge: at inference, only the student's entity and relation embeddings score triples, so multimodal encoders are needed only during training.
  • Discrete, per-triple exclusion of a modality removes misleading signals entirely, which the paper argues is stronger than the low-weight treatment used by gating and attention fusion.
  • Soft-label distillation supplies supervision about all entity candidates, so correlations among true answers and among distractors are learned even when they never appear as targets in a one-hot objective.
  • The framework degrades gracefully: with up to 80% of visual and textual modalities missing, and at embedding dimensions as low as 64, the student still beats the strongest full-size baseline on DB15K.
  • The recipe is modality-agnostic: adding another teacher modality only enlarges the policy pool, so the same reinforced distillation could absorb audio, video, or other future modalities.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extension beyond the paper: the non-stationary reward means the learned policy could be a bystander; a simpler per-triple selector that picks the teacher combination with lowest validation-set cross-entropy would settle whether reinforcement is essential to the reported gains.
  • Neighbouring problem: the neighbor-decoupled objective is a general treatment of link prediction with multiple true answers, so other knowledge graph completion settings with plural true tails could reuse it without the multimodal framing.
  • Engineering consequence: because the student is unimodal and can run at 1.72M parameters while outperforming a 22.0M-parameter baseline, the framework licenses replacing heavy multimodal encoders at serving time with a compact embedding model.
  • Testable extension: training the same student with the policy frozen after the first epoch, versus letting it co-train throughout, would reveal how much of the gain comes from the agent adapting to the student rather than from modality quality alone.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes DSoM, a framework for multimodal knowledge graph reasoning (MKGR). DSoM pre-trains three unimodal teachers (structural, visual, textual) and then trains a unimodal student KGR model by knowledge distillation from the teachers' soft labels. A novel reinforced teacher combination module selects, per triple, a subset of teachers via a policy network trained with a binary reward based on cross-entropy comparison with the student. The distillation loss is decoupled into neighbor and non-neighbor entity terms, extending Decoupled KD to MKGR's multiple true answers. Experiments on DB15K, MKG-W, MKG-Y, FB15K-237, and WN18 (plus WN9 in an appendix) report consistent gains over prior MKGR baselines, with the unimodal ComplEx student reaching 42.68 MRR on DB15K versus 37.72 for the previous best MyGO.

Significance. If the empirical results hold, DSoM makes a valuable conceptual contribution: instead of fusing modalities into a multimodal model, it trains a unimodal student on multimodal soft labels, which can exploit inter-entity label correlations and avoid negative transfer from unhelpful modalities. The gains are large and consistent on three medium-scale datasets, and the efficiency experiments (robustness to missing modalities and reduced embedding dimensions) are a genuine strength. The neighbor-decoupled distillation is a sensible adaptation of DKD to KGs. However, the paper is currently weakened by the absence of statistical validation, the incomplete large-scale comparisons, and a reinforcement reward design that is not analyzed well enough to explain the claimed advantage over greedy selection. These issues are fixable and do not, in my view, invalidate the central distillation idea, but they are load-bearing for the SOTA claim.

major comments (5)
  1. [Section 5.1, Tables 1–3] The central claim of state-of-the-art performance rests entirely on single-run results: no standard deviations, no number of seeds, and no significance tests are reported. On several metrics the advantage is small (e.g., WN18 Hits@10 is 97.4 vs. SGMPT's 97.8, and FB15K-237 Hits@1 is 27.5 vs. LAFA's 26.9). Without error bars or paired tests, the superiority over strong baselines is not statistically established. Please report means and standard deviations over at least three random seeds, and perform paired significance tests for the key comparisons in Tables 1 and 3.
  2. [Section 4.4, Eq. (8), and Table 2] The reward in Eq. (8) is non-stationary (the student is updated in the same loop as the policy) and binary (any teacher subset with lower cross-entropy than the student receives the same +1). Under this reward, a greedy BestStrategy that always selects the lowest-cross-entropy subset should be at least as good, yet DSoM beats BestStrategy by about 1 MRR on DB15K (42.68 vs. 41.65). The paper does not analyze the learned policy—for example, how the selected subsets differ from BestStrategy, or whether the advantage comes from exploration, stochasticity, or student co-evolution. Please add such an analysis, or an experiment with a stationary reward from a frozen student, to substantiate the claim that the reinforced module (rather than side effects of co-training) drives the improvement.
  3. [Section 5.3, Table 4] The ablation 'w/ RC on val.' trains the reinforced teacher combination on the validation set and then transfers it to the test set, whereas the full DSoM trains RC jointly during student training. In addition, the full model combines RC and NDKD, so the protocol mismatch makes it difficult to attribute the improvement to the reinforcement mechanism versus the co-training dynamics. Please run a joint-training ablation of RC alone (without NDKD) and an NDKD-only ablation under the same protocol, or explain why the validation-based protocol is equivalent to joint training.
  4. [Section 5.2, Table 3] The large-scale comparison is incomplete: several baselines have missing values (e.g., MyGO's MR and Hits@10, AdaMF-MAT's MR), and the numbers are taken from different publications that may use different evaluation protocols or filtered/unfiltered settings. The claim of superiority on FB15K-237 and WN18 is therefore not fully supported. Please provide a complete comparison under a single evaluation protocol, or clearly mark the source and condition of each baseline number and restrict the SOTA claim to metrics with complete and comparable entries.
  5. [Abstract and Appendix A] The abstract states that 'Codes are available at github.com/OreOZhao/DSoM', but Appendix A says 'We will open-source our code on GitHub after the paper is accepted.' This is a direct contradiction and undermines the reproducibility claim. Please reconcile the availability statement, and in the meantime provide the exact per-dataset hyperparameters (learning rate, batch size, epochs, optimizer) and the settings for gamma, tau, alpha, and beta so that the experiments can be reproduced.
minor comments (5)
  1. [Abstract and Section 5.1 vs. Appendix C] The paper says '5 datasets' in the Abstract and Section 5.1, but Appendix C states that experiments are conducted on 6 datasets and Table 5 includes WN9. Please make the count consistent throughout.
  2. [Appendix A] The appendix header reads 'Pipeling of DSoM'; this should be 'Pipeline of DSoM'.
  3. [Section 5.4] The sentence 'This strategy may not be the best strategy itself, but it is the best for the student' is vague; please define operationally what 'best for the student' means, for example in terms of final held-out MRR or a separate reward computed on validation triples.
  4. [Section 4.4, Eq. (8)] The asymmetric +1/-10 reward scale is not discussed; since the magnitude of the penalty can affect the entropy of the policy and the training dynamics, please justify this choice or provide a sensitivity analysis.
  5. [Figure 4] The label 'SOTA(MyGO)' is imprecise because MyGO is not the best model on every metric or dataset; please use a more neutral label such as 'MyGO (best published baseline)'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported SOTA results are evaluated on held-out test sets, and the Eq. (8) reward is a training-signal design choice rather than a fitted predictor of the reported metrics.

full rationale

The paper's central claim is that DSoM achieves state-of-the-art MKGR performance on five datasets. This claim is supported by held-out test-set evaluation: the student model is trained on T_train (Algorithm 1, lines 8-16) and evaluated on T_test (lines 19-21), and the multimodal teacher soft labels are fixed after pre-training (Eq. 3). No test labels or ranking metrics enter the training objective. The reward in Eq. (8) labels a teacher subset as beneficial exactly when its cross-entropy on a training triple is lower than the student's current cross-entropy. This is a hand-defined training signal, not a fitted parameter that encodes the test MRR or Hits@K. The policy is trained on training triples to maximize this reward, and the improvement over BestStrategy in Table 2 is an empirical consequence of stochastic exploration and student co-evolution, not a definitional equivalence: BestStrategy uses the same lowest-cross-entropy criterion, so the RL advantage is not forced by construction. Self-citations in the paper (notably MoSE [65]) are used as an external published baseline and backbone choice; they are not load-bearing for the derivation, and the NDKD component explicitly builds on the external DKD [64]. The paper does contain a non-circular reproducibility inconsistency: Algorithm 1 trains the policy on T_train, whereas Section 5.3 says the RC ablation is learned on the validation set; this affects experimental reporting, not circularity. The non-stationarity of the Eq. (8) reward is a robustness concern but not a circularity, because the reported test metrics are not used to define the reward. Overall, the derivation chain is self-contained against external benchmarks, so no circular step can be exhibited.

Assumptions & free parameters 8 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new entities or physical quantities. The central method depends on several domain assumptions about the usefulness of soft labels, the reward design, and the transferability of distilled knowledge. The hyperparameters are few but several are unreported; the reward scale and policy architecture are chosen without ablation.

free parameters (8)
  • gamma (weight of NDKD) = 2.0
    Default used for all datasets; chosen from the DB15K hyperparameter sweep in Figure 5.
  • tau (distillation temperature) = 4.0
    Default; higher temperature improves performance until convergence in the Figure 5 sweep.
  • alpha (neighbor KD weight) = 1.0
    Default weight for L_NEKD; the paper sweeps the alpha/beta ratio.
  • beta (non-neighbor KD weight) = 1.0
    Default weight for L_NNKD.
  • negative reward penalty = -10
    Chosen in Equation (8) for non-beneficial teacher subsets; no sensitivity analysis provided.
  • policy network hidden size = 1024
    MLP hidden dimension for the agent, stated in Implementation details; no ablation.
  • embedding dimension (default) = 1000
    The efficiency analysis mentions dim=1000 as the reference DSoM (26.8M parameters), but the main-table dimension is not explicitly stated; likely follows MoSE.
  • training hyperparameters (learning rate, batch size, epochs, optimizer)
    Not reported anywhere in the paper; required to reproduce the student and agent training.
assumptions (5)
  • domain assumption Multimodal teacher soft labels contain useful 'dark knowledge' about inter-entity correlations that can improve a student trained only with hard labels.
    Core premise of the KD objective in Section 4.5; if false, the NDKD loss would not help.
  • domain assumption The cross-entropy difference between a teacher subset and the student is a valid reward for selecting beneficial modalities (Equation 8).
    Defines the entire RC module; the paper provides no analysis of reward noise or misspecification.
  • domain assumption The neighbor set N(h,r) built from training triples covers all true answers relevant for distillation.
    Used in Equation (12) to decouple neighbor and non-neighbor logits.
  • domain assumption A unimodal embedding-based student can absorb multimodal knowledge purely through soft-label supervision on training triples.
    The student has no direct access to image or text features (Section 4.3); this assumption makes the whole framework possible.
  • standard math Standard filtered evaluation protocol and ComplEx scoring are appropriate for MKGR.
    Borrowed from prior work [4, 40, 52]; defines the metrics used in Tables 1 and 3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dark Side of Modalities: Reinforced Multimodal Distillation for Multimodal Knowledge Graph Reasoning." pith.science (2026). https://pith.science/paper/FFKMDF6B

@misc{pith2026250720738,
  author       = {Pith},
  title        = {Pith review of: Dark Side of Modalities: Reinforced Multimodal Distillation for Multimodal Knowledge Graph Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FFKMDF6B}},
  note         = {Machine review of arXiv:2507.20738}
}
read the original abstract

The multimodal knowledge graph reasoning (MKGR) task aims to predict the missing facts in the incomplete MKGs by leveraging auxiliary images and descriptions of entities. Existing approaches are trained with single-target objectives, which neglect the probabilistic correlations of entity labels, especially in non-target entities. Moreover, previous studies incorporate all modalities statically or adaptively, overlooking the negative impacts of irrelevant or misleading information in the incompetent modalities. To address these issues, we introduce a novel Reinforced Multimodal Distillation framework, exploiting the Dark Side of Modalities (DSoM) from two perspectives: (1) Dark knowledge from non-target entities: We propose to train a unimodal KGR model through logit distillation to mimic the multimodal soft labels provided by pre-trained multimodal teacher models. The multimodal soft labels could provide rich supervision signals with subtle correlations among both target and non-target entities from multiple perspectives. We further decouple logits into neighbor entities and non-neighbor entities to divide into two types of correlations. (2) Dark side in unhelpful modalities: To exclude the adverse effects of unhelpful modalities, we introduce a reinforced teacher combination mechanism that dynamically selects the optimal set of multimodal teachers for each triple. The agent is trained to maximize the rewards, which are only assigned to the beneficial multimodal combination strategies for the student model. Comprehensive experiments demonstrate the effectiveness of DSoM framework on 5 MKGR datasets. Codes are available at github.com/OreOZhao/DSoM.

Figures

Figures reproduced from arXiv: 2507.20738 by the authors.

Figure 1
Figure 1. We propose to train the MKGR model with (1) mul [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The DSoM framework for MKGR. First, we pre-train the multimodal teachers. Then we propose the reinforced [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Efficiency analysis on DB15K of DSoM with missing [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figures from the paper (2 more)
Figure 3
Figure 3. Figure 3: The ratio of selected strategies of DSoM. The S, V, [PITH_FULL_IMAGE:figures/full_fig_p008_3.png]
Figure 5
Figure 5. Figure 5: Reward curve 𝛿 during training, and hyper￾parameter analysis of 𝛾, 𝜏 and 𝛼/𝛽 on DB15K dataset [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

81 extracted references · 66 canonical work pages

  1. [1]

    Zeyuan Allen-Zhu and Yuanzhi Li. 2020. Towards Understanding Ensem- ble, Knowledge Distillation and Self-Distillation in Deep Learning. CoRR abs/2012.09816 (2020). arXiv:2012.09816 https://arxiv.org/abs/2012.09816

    arXiv 2020

  2. [2]

    Ivana Balazevic, Carl Allen, and Timothy Hospedales. 2019. TuckER: Tensor Fac- torization for Knowledge Graph Completion. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) . Association for Computational Linguistics

    2019

  3. [3]

    Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor

  4. [4]

    Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Ok- sana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. Advances in neural information processing systems 26 (2013)

    work page 2013

  5. [5]

    Zongsheng Cao, Qianqian Xu, Zhiyong Yang, Yuan He, Xiaochun Cao, and Qingming Huang. 2022. Otkge: Multi-modal knowledge graph embeddings via optimal transport. Advances in Neural Information Processing Systems 35 (2022), 39090–39102

    work page 2022

  6. [6]

    Xiang Chen, Ningyu Zhang, Lei Li, Shumin Deng, Chuanqi Tan, Changliang Xu, Fei Huang, Luo Si, and Huajun Chen. 2022. Hybrid transformer with multi-level fusion for multimodal knowledge graph completion. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 904–915

    work page 2022

  7. [7]

    Zhuo Chen, Yichi Zhang, Yin Fang, Yuxia Geng, Lingbing Guo, Xiang Chen, Qian Li, Wen Zhang, Jiaoyan Chen, Yushan Zhu, et al. 2024. Knowledge graphs meet multi-modal learning: A comprehensive survey. arXiv preprint arXiv:2402.05391 (2024)

    arXiv 2024

  8. [8]

    Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2018. Convolutional 2d knowledge graph embeddings. In AAAI, Vol. 32

    work page 2018

Show all 81 references
  1. [9]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human...

    2019

  2. [10]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al . 2020. An Image is Worth 16x16 Words: Trans- formers for Image Recognition at Scale. In Intern...

    2020

  3. [11]

    Cunhang Fan, Yujie Chen, Jun Xue, Yonghui Kong, Jianhua Tao, and Zhao Lv

  4. [12]

    Lingbing Guo, Yichi Zhang, Zhongpu Bo, Zhuo Chen, Mengshu Sun, Zhiqiang Zhang, Wen Zhang, and Huajun Chen. 2025. K-ON: Stacking Knowledge On the Head Layer of Large Language Model. In AAAI

    2025

  5. [13]

    Byeongho Heo, Jeesoo Kim, Sangdoo Yun, Hyojin Park, Nojun Kwak, and Jin Young Choi. 2019. A comprehensive overhaul of feature distillation. In Pro- ceedings of the IEEE/CVF international conference on computer vision . 1921–1930

    2019

  6. [14]

    Geoffrey Hinton. 2015. Distilling the Knowledge in a Neural Network. arXiv preprint arXiv:1503.02531 (2015)

    2015 arXiv

  7. [15]

    Ying Jin, Jiaqi Wang, and Dahua Lin. 2023. Multi-level logit distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 24276–24285

    2023

  8. [16]

    Jaejun Lee, Chanyoung Chung, Hochang Lee, Sungho Jo, and Joyce Whang. 2023. VISTA: Visual-Textual Knowledge Graph Representation Learning. InFindings of the Association for Computational Linguistics: EMNLP 2023 . 7314–7328

    2023

  9. [17]

    Jens Lehmann, Robert Isele, Max Jakob, Anja Jentzsch, Dimitris Kontokostas, Pablo N Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick Van Kleef, Sören Auer, et al. 2015. Dbpedia–a large-scale, multilingual knowledge base extracted from wikipedia. Semantic web 6, 2 (2015), 167–195

    2015

  10. [18]

    Xinhang Li, Xiangyu Zhao, Jiaxing Xu, Yong Zhang, and Chunxiao Xing. 2023. IMF: Interactive Multimodal Fusion Model for Link Prediction. In Proceedings of the ACM Web Conference 2023. 2572–2580

    2023

  11. [19]

    Yunshui Li, Junhao Liu, Chengming Li, and Min Yang. 2023. Self-distillation with meta learning for knowledge graph completion. arXiv preprint arXiv:2305.12209 (2023)

    2023 arXiv

  12. [20]

    Ke Liang, Lingyuan Meng, Meng Liu, Yue Liu, Wenxuan Tu, Siwei Wang, Si- hang Zhou, Xinwang Liu, and Fuchun Sun. 2022. Reasoning over different types of knowledge graphs: Static, temporal and multi-modal. arXiv preprint arXiv:2212.05767 (2022)

    2022 arXiv

  13. [21]

    Ke Liang, Lingyuan Meng, Yue Liu, Meng Liu, Wei Wei, Suyuan Liu, Wenxuan Tu, Siwei Wang, Sihang Zhou, and Xinwang Liu. 2024. Simple Yet Effective: Structure Guided Pre-trained Transformer for Multi-modal Knowledge Graph Reasoning. In Proceedings of the 32nd ACM International C...

    2024

  14. [22]

    Jiajun Liu, Peng Wang, Ziyu Shang, and Chenxiao Wu. 2023. IterDE: an iter- ative knowledge distillation framework for knowledge graph embeddings. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 37. 4488–4496

    2023

  15. [23]

    Kangzheng Liu, Feng Zhao, Yu Yang, and Guandong Xu. 2024. DySarl: Dynamic Structure-Aware Representation Learning for Multimodal Knowledge Graph Reasoning. In Proceedings of the 32nd ACM International Conference on Multimedia. 8247–8256

    2024

  16. [24]

    Yichen Liu, Jiawei Chen, Defang Chen, Zhehui Zhou, Yan Feng, and Can Wang

  17. [25]

    Ye Liu, Hui Li, Alberto Garcia-Duran, Mathias Niepert, Daniel Onoro-Rubio, and David S Rosenblum. 2019. MMKG: multi-modal knowledge graphs. In The Semantic Web: 16th International Conference, ESWC 2019, Portorož, Slovenia, June 2–6, 2019, Proceedings 16 . Springer, 459–474

    2019

  18. [26]

    Xinyu Lu, Lifang Wang, Zejun Jiang, Shichang He, and Shizhong Liu. 2022. MMKRL: A robust embedding approach for multi-modal knowledge graph repre- sentation learning. Applied Intelligence (2022), 1–18

    2022

  19. [27]

    In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management

    Confidence-aware Self-Semantic Distillation on Knowledge Graph Embed- ding. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management. 1493–1502

  20. [28]

    George A. Miller. 1995. WordNet: a lexical database for English. Commun. ACM 38, 11 (Nov. 1995), 39–41

    1995

  21. [29]

    Seyed Iman Mirzadeh, Mehrdad Farajtabar, Ang Li, Nir Levine, Akihiro Mat- sukawa, and Hassan Ghasemzadeh. 2020. Improved knowledge distillation via teacher assistant. In Proceedings of the AAAI conference on artificial intelligence , Vol. 34. 5191–5198

    2020

  22. [30]

    Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. 2019. Ok-vqa: A visual question answering benchmark requiring external knowledge. In CVPR. 3195–3204

    2019

  23. [31]

    Wonpyo Park, Dongju Kim, Yan Lu, and Minsu Cho. 2019. Relational knowledge distillation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 3967–3976

    2019

  24. [32]

    Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. 2014. Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550 (2014)

    2014 arXiv

  25. [33]

    Hatem Mousselly-Sergieh, Teresa Botschen, Iryna Gurevych, and Stefan Roth

  26. [34]

    Fabian M Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. Yago: a core of semantic knowledge. In Proceedings of the 16th international conference on World Wide Web. 697–706

    2007

  27. [35]

    Xuhui Sui, Ying Zhang, Yu Zhao, Baohang Zhou, and Xiaojie Yuan. 2025. Multi- modal Knowledge Graph Error Detection with Disentanglement VAE and Multi- Grained Triplet Confidence. In Proceedings of the ACM on Web Conference 2025, WWW 2025, Sydney, NSW, Australia, 28 April 2025-...

    2025

  28. [36]

    Rui Sun, Xuezhi Cao, Yan Zhao, Junchen Wan, Kun Zhou, Fuzheng Zhang, Zhongyuan Wang, and Kai Zheng. 2020. Multi-modal knowledge graphs for recommender systems. In CIKM. 1405–1414

    2020

  29. [37]

    Bin Shang, Yinliang Zhao, Jun Liu, and Di Wang. 2024. LAFA: Multimodal knowl- edge graph completion with link aware fusion and aggregation. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 38. 8957–8965

    2024

  30. [38]

    Yun Tang, Jing Huang, Guangtao Wang, Xiaodong He, and Bowen Zhou. 2020. Orthogonal Relation Transforms with Graph Context Modeling for Knowledge Graph Embedding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2713–2722

    2020

  31. [39]

    Kristina Toutanova and Danqi Chen. 2015. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd workshop on continuous vector space models and their compositionality . 57–66

    2015

  32. [40]

    Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In International conference on machine learning . PMLR, 2071–2080

    2016

  33. [41]

    Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2018. RotatE: Knowl- edge Graph Embedding by Relational Rotation in Complex Space. In ICLR

    2018

  34. [42]

    Kai Wang, Yu Liu, Qian Ma, and Quan Z Sheng. 2021. Mulde: Multi-teacher knowledge distillation for low-dimensional knowledge graph embeddings. In MM ’25, October 27–31, 2025, Dublin, Ireland. Yu Zhao et al. Proceedings of the Web Conference 2021 . 1716–1726

    2021

  35. [43]

    Liang Wang, Wei Zhao, Zhuoyu Wei, and Jingming Liu. 2022. SimKGC: Simple Contrastive Knowledge Graph Completion with Pre-trained Language Models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for...

    2022 doi

  36. [44]

    Meng Wang, Sen Wang, Han Yang, Zheng Zhang, Xi Chen, and Guilin Qi. 2021. Is Visual Context Really Helpful for Knowledge Graph? A Representation Learning Perspective. In Proceedings of the 29th ACM International Conference on Multime- dia. 2735–2743

    2021

  37. [45]

    Denny Vrandečić and Markus Krötzsch. 2014. Wikidata: a free collaborative knowledgebase. Commun. ACM 57, 10 (2014), 78–85

    2014

  38. [46]

    Xin Wang, Benyuan Meng, Hong Chen, Yuan Meng, Ke Lv, and Wenwu Zhu

  39. [47]

    Zikang Wang, Linjing Li, Qiudan Li, and Daniel Zeng. 2019. Multimodal data enhanced representation learning for knowledge graphs. In 2019 International Joint Conference on Neural Networks (IJCNN) . IEEE, 1–8

    2019

  40. [48]

    Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning 8 (1992), 229–256

    1992

  41. [49]

    Xintao Wang, Qianyu He, Jiaqing Liang, and Yanghua Xiao. [n. d.]. Language Models as Knowledge Embeddings. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence (Vienna, Austria). International Joint Conferences on Artificial Intelligence...

    doi

  42. [50]

    Ruobing Xie, Zhiyuan Liu, Jia Jia, Huanbo Luan, and Maosong Sun. 2016. Rep- resentation learning of knowledge graphs with entity descriptions. In AAAI, Vol. 30

    2016

  43. [51]

    Ruobing Xie, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. 2017. Image- embodied knowledge representation learning. In IJCAI. 3140–3146

    2017

  44. [52]

    Derong Xu, Tong Xu, Shiwei Wu, Jingbo Zhou, and Enhong Chen. 2022. Relation- enhanced negative sampling for multimodal knowledge graph completion. In Proceedings of the 30th ACM international conference on multimedia . 3857–3866

    2022

  45. [53]

    Haotian Xu, Yuhua Wang, and Jiahui Fan. 2024. Self-Knowledge Distillation for Knowledge Graph Embedding. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). 14595–14605

    2024

  46. [54]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020

  47. [55]

    Liang Yao, Chengsheng Mao, and Yuan Luo. 2019. KG-BERT: BERT for knowledge graph completion. arXiv preprint arXiv:1909.03193 (2019)

    2019 arXiv

  48. [56]

    Jiachen Yu, Yuehong Wu, and Shangsong Liang. 2023. Wasserstein Topology Transfer for Joint Distilling Embeddings of Knowledge Graph Entities and Re- lations. In Proceedings of the 2023 6th International Conference on Algorithms, Computing and Artificial Intelligence. 176–182

    2023

  49. [57]

    Yichi Zhang, Mingyang Chen, and Wen Zhang. 2023. Modality-aware negative sampling for multi-modal knowledge graph embedding. In 2023 International Joint Conference on Neural Networks (IJCNN) . IEEE, 1–8

    2023

  50. [58]

    Yichi Zhang, Zhuo Chen, Lingbing Guo, Yajing Xu, Binbin Hu, Ziqi Liu, Wen Zhang, and Huajun Chen. 2024. Tokenization, Fusion, and Augmentation: To- wards Fine-grained Multi-modal Entity Representation. arXiv:2404.09468 [cs.AI] https://arxiv.org/abs/2404.09468

    2024 arXiv

  51. [59]

    Bishan Yang, Scott Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding Entities and Relations for Learning and Inference in Knowledge Bases. In ICLR

    2015

  52. [60]

    Yichi Zhang, Zhuo Chen, and Wen Zhang. 2023. MACO: A modality adversarial and contrastive framework for modality-missing multi-modal knowledge graph completion. In CCF International Conference on Natural Language Processing and Chinese Computing. Springer, 123–134

    2023

  53. [61]

    Ying Zhang, Tao Xiang, Timothy M Hospedales, and Huchuan Lu. 2018. Deep mutual learning. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4320–4328

    2018

  54. [62]

    Yichi Zhang and Wen Zhang. 2022. Knowledge graph completion with pre- trained multimodal transformer and twins negative sampling. arXiv preprint arXiv:2209.07084 (2022)

    2022 arXiv

  55. [63]

    Ying Zhang, Yu Zhao, Xuhui Sui, Baohang Zhou, Xiangrui Cai, Li Shen, Xiaojie Yuan, and Dacheng Tao. 2025. Hyper-modal Imputation Diffusion Embedding with Dual-Distillation for Federated Multimodal Knowledge Graph Comple- tion. CoRR abs/2506.22036 (2025). https://doi.org/10.485...

    2025 doi arXiv

  56. [64]

    Yichi Zhang, Zhuo Chen, Lei Liang, Huajun Chen, and Wen Zhang. 2024. Unleash- ing the Power of Imbalanced Modality Information for Multi-modal Knowledge Graph Completion. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources...

    2024

  57. [65]

    Yu Zhao, Xiangrui Cai, Yike Wu, Haiwei Zhang, Ying Zhang, Guoqing Zhao, and Ning Jiang. 2022. MoSE: Modality Split and Ensemble for Multimodal Knowledge Graph Completion. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 10527–10536

    2022

  58. [66]

    Yu Zhao, Yike Wu, Xiangrui Cai, Ying Zhang, Haiwei Zhang, and Xiaojie Yuan

  59. [67]

    Yu Zhao, Ying Zhang, Xuhui Sui, and Xiangrui Cai. 2025. ME3A: A Multimodal Entity Entailment framework for multimodal Entity Alignment. Inf. Process. Manag. 62, 1 (2025), 103951. https://doi.org/10.1016/J.IPM.2024.103951

    2025

  60. [68]

    Yu Zhao, Ying Zhang, Baohang Zhou, Xinying Qian, Kehui Song, and Xiangrui Cai

  61. [69]

    Borui Zhao, Quan Cui, Renjie Song, Yiyu Qiu, and Jiajun Liang. 2022. Decoupled knowledge distillation. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition . 11953–11962

    2022

  62. [70]

    Shangfei Zheng, Hongzhi Yin, Tong Chen, Quoc Viet Hung Nguyen, Wei Chen, and Lei Zhao. 2025. Do as I Can, Not as I Get: Topology-Aware Multi-Hop Reasoning on Multi-Modal Knowledge Graphs. IEEE Trans. Knowl. Data Eng. 37, 5 (2025), 2405–2419

    2025

  63. [71]

    Baohang Zhou, Ying Zhang, Yu Zhao, Xuhui Sui, and Xiaojie Yuan. 2025. Mul- timodal Graph-Based Variational Mixture of Experts Network for Zero-Shot Multimodal Information Extraction. In Proceedings of the ACM on Web Conference 2025, WWW 2025, Sydney, NSW, Australia, 28 April 2...

    2025

  64. [72]

    In Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, July 9-14, 2023 , Anna Rogers, Jordan L

    From Alignment to Entailment: A Unified Textual Entailment Framework for Entity Alignment. In Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, July 9-14, 2023 , Anna Rogers, Jordan L. Boyd-Graber, and Naoaki Okazaki (Eds.). Association for ...

    2023

  65. [73]

    Yushan Zhu, Wen Zhang, Mingyang Chen, Hui Chen, Xu Cheng, Wei Zhang, and Huajun Chen. 2022. Dualde: Dually distilling knowledge graph embedding for faster and cheaper reasoning. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining . 1516–1...

    2022

  66. [75]

    In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval

    Contrast then memorize: Semantic neighbor retrieval-enhanced inductive multimodal knowledge graph completion. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval . 102–111

  67. [76]

    Shangfei Zheng, Weiqing Wang, Jianfeng Qu, Hongzhi Yin, Wei Chen, and Lei Zhao. 2023. MMKGR: Multi-hop Multi-modal Knowledge Graph Reasoning. In 39th IEEE International Conference on Data Engineering, ICDE 2023, Anaheim, CA, USA, April 3-7, 2023. IEEE, 96–109

    2023

  68. [79]

    Xiangru Zhu, Zhixu Li, Xiaodan Wang, Xueyao Jiang, Penglei Sun, Xuwu Wang, Yanghua Xiao, and Nicholas Jing Yuan. 2022. Multi-Modal Knowledge Graph Construction and Application: A Survey. arXiv preprint arXiv:2202.05786 (2022)

    2022 arXiv

  69. [81]

    The dataset statistics are listed in Table 5

    and YAGO [34], FB15K-237 dataset [39] is from Freebase KG [3], and WN18, WN9 dataset is from WordNet KG [28], respectively. The dataset statistics are listed in Table 5. D Experiments compared with Structural MKGR methods We compare DSoM with TMR [70], DySarl [23], and MMKGR [...

    2025

  70. [2008]

    In Proceedings of the 2008 ACM SIGMOD international conference on Management of data

    Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data. 1247–1250

    2008

  71. [2018]

    In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics

    A multimodal translation-based approach for knowledge graph represen- tation learning. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics. 225–234

  72. [2023]

    In Proceedings of the 31st ACM International Conference on Multimedia

    TIVA-KG: A multimodal knowledge graph with text, image, video and audio. In Proceedings of the 31st ACM International Conference on Multimedia . 2391–2399

  73. [2024]

    InProceedings of the AAAI Conference on Artificial Intelligence, Vol

    Progressive Distillation Based on Masked Generation Feature Method for Knowledge Graph Completion. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 8380–8388

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.