Pith. sign in

REVIEW 3 major objections 5 minor 34 references

M2GDT claims that aligning multimodal features before diffusion — rather than denoising raw features — yields the best reported completion accuracy on three benchmark knowledge graphs.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 22:49 UTC pith:A6JQZEIK

load-bearing objection Relation-conditioned MoE routing, a frozen MLLM anchor, and a diffusion transformer are sensibly combined for MKGC with small, consistent MRR gains, but the main ablation confuses routing with auxiliary supervision. the 3 major comments →

arxiv 2607.15592 v1 pith:A6JQZEIK submitted 2026-07-17 cs.AI

MGDT: MLLM-Guided Diffusion Transformer with Relation-Adaptive Mixture-of-Experts for Multimodal Knowledge Graph Completion

Xu Hou , Meiyu Liang , Wei Huang , Yawen Li , Zhe Xue , Wu Liu , Guanhua Ye , Lei Shi
show 1 more author
Kangkang Lu
This is my paper
classification cs.AI
keywords Multimodal Knowledge Graph CompletionDiffusion ModelMixture-of-ExpertsMultimodal Large Language ModelSemantic AlignmentRelation-Adaptive RoutingKnowledge Graph EmbeddingEntity Completion
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that prior diffusion-based multimodal knowledge graph completion fails because it forces the denoiser to handle cue selection, cross-modal alignment, and structure-aware generation all at once. It proposes an 'align-then-diffuse' framework: a relation-adaptive mixture-of-experts routes modality-specific features, a frozen multimodal large language model acts as a semantic anchor to pull those routed features into one coherent latent space, and a knowledge graph diffusion transformer then denoises in that aligned space. The central claim is that this decoupling produces better missing-entity representations, and the paper reports new best MRR on MKG-W, MKG-Y, and DB15K. A sympathetic reader would care because the design is transferable: the anchor can be swapped between MLLMs with only minor loss, so the paradigm rather than the specific model is the contribution.

Core claim

The central claim is that multimodal knowledge graph completion is best formulated as an align-then-diffuse pipeline rather than as direct conditional diffusion on raw multimodal features. Specifically, the paper proposes M2GDT, which first uses a relation-adaptive semantic routing mixture-of-experts (RASR-MoE) to select relation-relevant semantic transformation paths per modality, then uses a frozen MLLM (Qwen3-VL) as a semantic anchor, pulling the routed features toward MLLM-consistent representations via cosine-similarity and KL losses, and finally runs a knowledge graph diffusion transformer (KGDT) in that aligned space to generate the missing entity embedding. The paper demonstrates on

What carries the argument

The central object is the align-then-diffuse pipeline, made concrete by three components: (1) Relation-Adaptive Semantic Routing Mixture-of-Experts (RASR-MoE), which uses a relation-conditioned router with Gaussian noise to select modality-specific expert transformations and suppress irrelevant modality interference; (2) an MLLM-anchored alignment stage, in which raw structural/textual/visual features are concatenated, projected, and fed into a frozen MLLM (Qwen3-VL) to produce anchor representations that the routed encoder features are pulled toward via cosine and KL losses; and (3) a Knowledge Graph Diffusion Transformer (KGDT), which injects a time embedding plus the fused aligned conditi

Load-bearing premise

The load-bearing premise is that feeding projected raw multimodal features into a frozen vision-language model produces semantically coherent anchor representations, so the alignment loss genuinely calibrates cross-modal space; if this premise is false, the reported gains could be an artifact of the auxiliary classification losses rather than of the MLLM-guided alignment.

What would settle it

Replace the frozen MLLM anchor with a randomly initialized transformer of the same size (same projection and alignment loss), retraining from scratch: if MRR does not drop materially below the reported 37.77 on MKG-W, then the 'semantic anchor' is not performing semantic alignment and the claimed mechanism is wrong. A complementary test: train the full model with the alignment loss removed but the RASR-MoE and KGDT intact; if MRR stays near 37.77, then the alignment stage is not a necessary cause of the gains.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the align-then-diffuse claim is correct, future diffusion-based MKGC methods should adopt a decoupled pipeline rather than denoising raw multimodal features.
  • The relation-adaptive routing result implies that relations genuinely require different modality emphases, so a fixed fusion strategy is structurally suboptimal.
  • The frozen MLLM anchor demonstrates that a strong pretrained vision-language model can serve as a calibration target without fine-tuning, reducing the need for large MKGC-specific training sets.
  • The replacement experiments suggest the paradigm transfers across MLLM backbones, which is a practical deployment point for lightweight models.
  • The diffusion stage contributes on top of alignment: even after good alignment, a transformer denoiser outperforms a linear one, suggesting iterative refinement adds value.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The ablation places the largest weight on RASR-MoE, but the alignment mechanism relies on the strong premise that feeding arbitrary continuous vectors into a frozen MLLM produces semantically coherent anchors; the reported gains could partly come from the auxiliary BCE losses that supervise each branch, and a cleaner test would be a random-transformer anchor control.
  • The cosine-similarity convergence to about 0.55 suggests the anchor does not fully unify the modalities; if alignment is only partial, a natural extension is to test whether a stronger alignment objective (e.g., contrastive) further improves MRR or, conversely, over-aligns and hurts.
  • A testable extension directly implied by the paper's reasoning: if the paradigm truly decouples selection, alignment, and generation, then swapping the diffusion stage for a non-diffusion generative head (e.g., a flow model) should preserve most of the gain whenever the alignment is good.
  • Because the anchor is frozen, the design is friendly to continual learning: new multimodal knowledge can be incorporated by updating routing and denoiser without altering the MLLM's frozen weights.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes M2GDT, an align-then-diffuse framework for multimodal knowledge graph completion. It introduces RASR-MoE to route structural, visual, and textual features based on the query relation, uses a frozen Qwen3-VL backbone as a 'semantic anchor' to align the routed features, and then generates the missing entity embedding with a knowledge-graph diffusion transformer (KGDT) conditioned on the aligned features. The method is evaluated on MKG-W, MKG-Y, and DB15K against 19 baselines. Table 1 reports the best MRR on all three datasets and best Hits@1/Hits@3 in most cases, with small but consistent margins over the strongest multimodal baseline MCKGC. Ablations on MKG-W suggest that every component contributes, with the largest drop when RASR-MoE is removed; a sensitivity analysis and an MLLM-substitution study are also presented.

Significance. The paper addresses a real problem and the proposed align-then-diffuse decomposition is conceptually clean and likely useful to the MKGC community. The central empirical claim—best MRR on MKG-W, MKG-Y, and DB15K—is supported by Table 1 if the numbers are taken at face value, with gains over strong multimodal baselines (MCKGC, MyGo, MoMoK). The paper also provides hyperparameter sensitivity analysis, an MLLM-compatibility study (Table 3), and a quantitative alignment diagnostic (Figure 6), which are creditable. However, the ablation evidence for the main novelty is currently confounded, statistical reliability is not demonstrated, and the 'MLLM semantic anchor' interpretation is not supported by independent evidence. These issues are addressable but require additional experiments and rewriting.

major comments (3)
  1. [§3.3, Table 2; §2.2, Eq. (10)–(11)] The w/o RASR-MoE ablation removes the routing module but, as described, also removes the four auxiliary branch classifiers p_s, p_v, p_t, p_f and their BCE loss L_moe (Eq. 11; λ1=1.0 in Eq. 33). The paper does not state whether L_moe is retained in this variant. Since L_moe is the largest loss component and provides direct entity-ranking supervision, the MRR drop from 37.77 to 30.88 cannot be attributed solely to relation-adaptive routing. Please specify the variant construction and add ablations that (a) keep L_moe on un-routed features and (b) remove L_moe while keeping routing, to isolate the contribution of the routing mechanism.
  2. [Abstract; §3.2, Table 1] The abstract claims M2GDT 'consistently outperforms strong baselines,' but Table 1 shows MKG-Y Hits@10 is 45.10, below AdaMF-MAT's 45.48 (the paper's own 'Gain vs. SOTA' row shows -0.84%). The text later acknowledges this, but the abstract and contribution bullets overstate the claim. Moreover, no error bars, standard deviations, or significance tests are reported for any dataset; many gains are small (e.g., MKG-W MRR 37.77 vs 36.88, DB15K H@1 32.60 vs 32.38). Please report multiple-seed statistics or paired tests and qualify the 'consistently outperforms' claim.
  3. [§2.3.1, Eq. (13)–(15); §2.3.3, Eq. (20)–(23)] The 'MLLM semantic anchor' is not independent of the input: the frozen Qwen3-VL backbone receives a learnable projection of the concatenated raw structural, visual, and textual features (Eq. 13–14). The alignment loss therefore encourages encoder outputs to match a transformation of the same raw inputs, not a language-grounded external representation. No evidence is provided that feeding arbitrary continuous vectors into Qwen3-VL yields a semantically coherent unified space; in the extreme, a linear map could minimize L_align. The paper's 'MLLM-guided alignment' interpretation (abstract, §1, §3.5) is thus unsupported as stated. Please validate with qualitative examples, an independent semantic probe, or a comparison against a non-MLLM projection baseline.
minor comments (5)
  1. [Title/Abstract] The model name is inconsistent: the title and full text use M2GDT, while the abstract's first sentence uses MGDT. Please unify.
  2. [§3.1] The text says '12 multimodal KGC methods' but then lists 13 (IKRL, TBKGC, TransAE, MMKRL, RSME, VBKGC, OTKGE, IMF, AdaMF-MAT, VISTA, MyGo, MoMoK, MCKGC). With the 6 unimodal methods, the total is 19, so the count should be 13.
  3. [§2.2, Eq. (5)–(9)] Several implementation details of RASR-MoE are missing: the number of experts M, the architecture of each expert, whether routing is top-k or dense, the dimension d_f, and how Σ_k in Eq. (5) is parameterized. These details are essential for reproducibility.
  4. [Table 3] The text says CLIP-ViT-L/14 achieves '36.85% MRR (0.92% drop)'. The drop from 37.77 is 0.92 percentage points, which is 2.44% relative, not 0.92% relative. Please clarify.
  5. [§3.3, Table 2] The relation between 'w/o KGDT' and 'w/o KGDT (Linear Denoiser)' is unclear: the first gives MRR 34.88 and the second gives 36.68, yet the text does not specify what the 'weaker denoising backbone' in the first variant is. Please define both variants.

Circularity Check

0 steps flagged

No significant circularity: the central MRR claims are benchmarked against 19 independent baselines, and the component contributions, while raising some attribution questions, do not reduce to the fitted inputs by construction.

full rationale

I examined the derivation chain for any step where an output or prediction is identical by construction to an input, where a fitted parameter is renamed as a prediction, or where a load-bearing claim rests on self-citation. The main empirical claim—best MRR on MKG-W, MKG-Y, and DB15K—is evaluated against held-out benchmark rankings relative to 19 external baselines, so it is externally falsifiable and not circular. The MLLM anchor is constructed from the same raw structural, visual, and textual features (Eqs. 13–15) and the alignment loss pulls encoder outputs toward projections of that same input (Eqs. 16–20). This weakens the interpretation that the anchor provides an independent semantic signal, but it is a regularizer, not a prediction or a fitted result that is later reported as a discovery. The w/o RASR-MoE ablation (Table 2) removes the routing module, and because the L_moe auxiliary heads (Eqs. 10–11) are defined on RASR-MoE encoder outputs, the reported 6.89 MRR drop could partly reflect loss of auxiliary supervision rather than routing alone. That is a confound in causal attribution, but the paper does not define the ablated variant in a way that makes the gain equal to the routing module by construction. DiffusionCom [8] is a self-citation, but it is used only as background motivation and is not the source of the reported gains. No uniqueness theorem, ansatz smuggling, or renaming of a known result appears. I therefore find no significant circularity.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

M2GDT introduces no new physical or ontological entities; it combines existing components (MoE, frozen MLLM, DDPM, transformer). The ledger's main items are empirically tuned hyperparameters and two domain assumptions: that the frozen MLLM provides a meaningful anchor when fed concatenated raw embeddings, and that the alignment losses preserve discriminative KG information. The weakest external premise is the MLLM-anchor input format.

free parameters (6)
  • loss weights lambda1-lambda4 = lambda1=1.0, lambda2=0.0001, lambda3=0.01, lambda4=0.1
    Chosen via sensitivity analysis on MKG-W (§3.4); the central architecture depends on these weights for balancing routing supervision, anchoring, and diffusion. The lambda labels in §3.4 conflict with §2.5.1.
  • alignment branch weights for structural/visual branches = 1.0
    Heuristically set in §2.3.3 with no sensitivity analysis for this specific choice; textual and fused branches are treated as auxiliary regularizers.
  • number of KGDT transformer blocks = 3
    Tuned on MKG-W (Fig. 5b); gains saturate after 3 blocks.
  • diffusion steps = 100
    Tuned on MKG-W (Fig. 5a); performance saturates at 100 steps.
  • number of MoE experts M = not specified
    Architecture hyperparameter never reported; capacity of the routing module depends on it.
  • KL temperature T = not specified
    Used in Eq. 21–22 for fused-branch alignment; no value or sensitivity analysis provided.
axioms (5)
  • standard math DDPM forward/reverse processes (Eq. 29–31) provide a valid conditional denoising formulation for entity embeddings.
    Assumed without derivation specific to KG embeddings; standard diffusion theory is taken as background.
  • domain assumption A frozen MLLM (Qwen3-VL) fed with a concatenated vector of raw feature embeddings produces hidden states that form a semantically coherent unified multimodal space.
    §2.3.1 Eq. 13–15 treats x_cat = [x_s;x_v;x_t] as continuous input tokens to Qwen3-VL; no evidence is given that this is a valid input format or that the resulting anchor space reduces cross-modal heterogeneity.
  • domain assumption Cosine-similarity and temperature-scaled KL losses (Eq. 20–24) align the MoE encoder to the anchor without destroying KG-specific discriminative information.
    Alignment is imposed as a regularizer; the claim that moderate cosine similarity (0.52–0.58) is optimal is based on one training run without statistical support.
  • domain assumption The benchmark datasets (MKG-W, MKG-Y, DB15K) and standard ranking protocol provide a fair comparison among methods.
    The paper does not report evaluation filtering details or negative-sampling scheme; fair comparison is assumed.
  • domain assumption Relation-conditioned routers in RASR-MoE learn meaningful modality-selection policies rather than collapsing to input-independent weights.
    Routing is trained with only a small CLUB regularizer and BCE branch losses; no routing visualization or analysis is provided.

pith-pipeline@v1.3.0-alltime-deepseek · 13093 in / 12813 out tokens · 140319 ms · 2026-08-01T22:49:41.714403+00:00 · methodology

0 comments
read the original abstract

Multimodal Knowledge Graph Completion (MKGC) requires inferring missing entities from structural, textual, and visual cues. Existing diffusion-based MKGC methods usually denoise directly on raw multimodal features. Such a design forces the denoiser to simultaneously perform relation-dependent cue selection, cross-modal semantic alignment, and structure-aware entity generation, which introduces noisy and semantically inconsistent conditions for diffusion and consequently leads to suboptimal completion performance. To address this limitation, we propose MGDT: MLLM-Guided Diffusion Transformer with Relation-Adaptive Mixture-of-Experts (MGDT), a novel MKGC framework built on an align-then-diffuse paradigm. MGDT first employs a Relation-Adaptive Semantic Routing Mixture-of-Experts (RASR-MoE) module to select relation-relevant multimodal semantic transformation paths and suppress irrelevant modality interference. MGDT then uses a frozen Multimodal Large Language Model (MLLM) as a semantic anchor to align the routed multimodal representations into a unified latent space and reduce cross-modal semantic heterogeneity. Finally, a Knowledge Graph Diffusion Transformer (KGDT) performs graph-conditioned denoising generation in the aligned space to produce the missing entity representation. Experiments on three benchmark datasets show that MGDT consistently outperforms strong baselines.

Figures

Figures reproduced from arXiv: 2607.15592 by Guanhua Ye, Kangkang Lu, Lei Shi, Meiyu Liang, Wei Huang, Wu Liu, Xu Hou, Yawen Li, Zhe Xue.

Figure 1
Figure 1. Figure 1: Core challenges of MKGC. Structural, textual, and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparison between the proposed method and ex [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the M2GDT framework. RASR-MoE first performs relation-adaptive semantic routing over heterogeneous multimodal inputs. A frozen semantic anchor then aligns the routed features into calibrated condition embeddings, which are combined with timestep embeddings for diffusion denoising. Starting from a noisy latent, KGDT iteratively predicts noise and refines the latent toward the target entity repre… view at source ↗
Figure 4
Figure 4. Figure 4: Illustration of the proposed anchor-guided align [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Sensitivity analysis of M2GDT with respect to four hyperparameters on the MKG-W dataset: (a) diffusion steps, (b) number of Transformer blocks, (c) MLLM alignment weight, and (d) diffusion loss weight. 3 Experiments 3.1 Experimental Setup Datasets. We evaluate M2GDT on three widely used multimodal knowledge graph completion benchmarks: MKG-W, MKG-Y [28], and DB15K [12]. These datasets contain relational tr… view at source ↗
Figure 6
Figure 6. Figure 6: Evolution of cosine similarity for text, vision, struc [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 5 linked inside Pith

  1. [1]

    Ivana Balazevic, Carl Allen, and Timothy Hospedales. 2019. TuckER: Tensor Fac- torization for Knowledge Graph Completion. InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, 5184–5193. do...

    doi:10.18653/v1/d19-1522 2019

  2. [2]

    Antoine Bordes, Nicolas Usunier, Alberto Garcia-Durán, Jason Weston, and Ok- sana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. InProceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2(Lake Tahoe, Nevada)(NIPS’13). Curran Associates Inc., Red Hook, NY, USA, 2787–2795

    2013

  3. [3]

    Zongsheng Cao, Qianqian Xu, Zhiyong Yang, Yuan He, Xiaochun Cao, and Qingming Huang. 2022. OTKGE: Multi-modal Knowledge Graph Embeddings via Optimal Transport. InAdvances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), Vol. 35. Curran Associates, Inc., 39090–39102. https://proceedings.neu...

    2022

  4. [4]

    Zhuo Chen, Yin Fang, Yichi Zhang, Lingbing Guo, Jiaoyan Chen, Jeff Z Pan, Hua- jun Chen, and Wen Zhang. 2025. Noise-powered multi-modal knowledge graph representation framework. InProceedings of the 31st International Conference on Computational Linguistics. 141–155

    2025

  5. [5]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Jill Burstein, Christy...

    2019

  6. [6]

    Yuxiao Gao, Fuwei Zhang, Zhao Zhang, Xiaoshuang Min, and Fuzhen Zhuang

  7. [7]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition. 770–778

    2016

  8. [8]

    Wei Huang, Meiyu Liang, Peining Li, Xu Hou, Yawen Li, Junping Du, Zhe Xue, and Zeli Guan. 2025. DiffusionCom: Structure-Aware Multimodal Diffusion Model for Multimodal Knowledge Graph Completion.arXiv preprint arXiv:2504.06543 (2025)

    Pith/arXiv arXiv 2025

  9. [9]

    Jaejun Lee, Chanyoung Chung, Hochang Lee, Sungho Jo, and Joyce Whang. 2023. VISTA: Visual-textual knowledge graph representation learning. InFindings of the association for computational linguistics: EMNLP 2023. 7314–7328

    2023

  10. [10]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. InInternational conference on machine learning. PMLR, 19730–19742

    2023

  11. [11]

    Xinhang Li, Xiangyu Zhao, Jiaxing Xu, Yong Zhang, and Chunxiao Xing. 2023. IMF: interactive multimodal fusion model for link prediction. InProceedings of the ACM web conference 2023. 2572–2580

    2023

  12. [12]

    Ye Liu, Hui Li, Alberto Garcia-Duran, Mathias Niepert, Daniel Onoro-Rubio, and David S Rosenblum. 2019. MMKG: multi-modal knowledge graphs. InEuropean Semantic Web Conference. Springer, 459–474

    2019

  13. [13]

    Xiao Long, Liansheng Zhuang, Aodi Li, Houqiang Li, and Shafei Wang. 2024. Fact embedding through diffusion model for knowledge graph completion. In Proceedings of the ACM Web Conference 2024. 2020–2029

    2024

  14. [14]

    Hatem Mousselly-Sergieh, Teresa Botschen, Iryna Gurevych, and Stefan Roth

  15. [15]

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. 2021. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741(2021)

    Pith/arXiv arXiv 2021

  16. [16]

    Alexander Quinn Nichol and Prafulla Dhariwal. 2021. Improved denoising diffu- sion probabilistic models. InInternational conference on machine learning. PMLR, 8162–8171

    2021

  17. [17]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. InInternational conference on machine learning. PmLR, 8748–8763

    2021

  18. [18]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10684–10695

    2022

  19. [19]

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al

  20. [20]

    Sara Mah- davi, Raphael Gontijo-Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Lit, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mah- davi, Raphael Gontijo-Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. 2022. Photorealistic text-to-image diffusion models with deep language understanding. InProceedings of the 36th Inter...

    2022

  21. [21]

    Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space.CoRR abs/1902.10197 (2019). arXiv:1902.10197 http://arxiv.org/abs/1902.10197

    Pith/arXiv arXiv 2019

  22. [22]

    Yun Tang, Jing Huang, Guangtao Wang, Xiaodong He, and Bowen Zhou. 2020. Orthogonal relation transforms with graph context modeling for knowledge graph embedding. InProceedings of the 58th annual meeting of the association for computational linguistics. 2713–2722

    2020

  23. [23]

    Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. InInternational conference on machine learning. PMLR, 2071–2080

    2016

  24. [24]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need.Advances in neural information processing systems30 (2017). MM ’26, 10–14 November 2026, Rio de Janeiro, Brazil Anonymous Submission

    2017

  25. [25]

    Meng Wang, Sen Wang, Han Yang, Zheng Zhang, Xi Chen, and Guilin Qi. 2021. Is Visual Context Really Helpful for Knowledge Graph? A Representation Learning Perspective. InProceedings of the 29th ACM International Conference on Multime- dia(Virtual Event, China)(MM ’21). Association for Computing Machinery, New York, NY, USA, 2735–2743. doi:10.1145/3474085.3475470

    arXiv 2021

  26. [26]

    Zikang Wang, Linjing Li, Qiudan Li, and Daniel Zeng. 2019. Multimodal data enhanced representation learning for knowledge graphs. In2019 International Joint Conference on Neural Networks (IJCNN). IEEE, 1–8

    2019

  27. [27]

    Ruobing Xie, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. 2017. Image- embodied knowledge representation learning. InProceedings of the 26th Interna- tional Joint Conference on Artificial Intelligence(Melbourne, Australia)(IJCAI’17). AAAI Press, 3140–3146

    2017

  28. [28]

    Derong Xu, Tong Xu, Shiwei Wu, Jingbo Zhou, and Enhong Chen. 2022. Relation- enhanced negative sampling for multimodal knowledge graph completion. In Proceedings of the 30th ACM international conference on multimedia. 3857–3866

    2022

  29. [29]

    Yichi Zhang, Zhuo Chen, Lingbing Guo, Yajing Xu, Binbin Hu, Ziqi Liu, Wen Zhang, and Huajun Chen. 2025. Tokenization, Fusion, and Augmentation: To- wards Fine-grained Multi-modal Entity Representation. InAAAI. AAAI Press, 13322–13330

    2025

  30. [30]

    Yichi Zhang, Zhuo Chen, Lei Liang, Huajun Chen, and Wen Zhang. 2024. Unleash- ing the power of imbalanced modality information for multi-modal knowledge graph completion.arXiv preprint arXiv:2402.15444(2024)

    Pith/arXiv arXiv 2024

  31. [32]

    Yichi Zhang and Wen Zhang. 2022. Knowledge Graph Completion with Pre-trained Multimodal Transformer and Twins Negative Sampling. arXiv:2209.07084 [cs.AI] https://arxiv.org/abs/2209.07084

    Pith/arXiv arXiv 2022

  32. [2015]

    Imagenet large scale visual recognition challenge.International journal of computer vision115, 3 (2015), 211–252

    2015

  33. [2018]

    InProceedings of the seventh joint conference on lexical and computational semantics

    A multimodal translation-based approach for knowledge graph repre- sentation learning. InProceedings of the seventh joint conference on lexical and computational semantics. 225–234

  34. [2025]

    InProceed- ings of the AAAI Conference on Artificial Intelligence, Vol

    Mixed-Curvature Multi-Modal Knowledge Graph Completion. InProceed- ings of the AAAI Conference on Artificial Intelligence, Vol. 39. 11699–11707