Graph-MLLM: Harnessing Multimodal Large Language Models for Multimodal Graph Learning

Chuanhao Ji; Daochen Zha; Dongzhe Fan; Jiacheng Shen; Jiajin Liu; Qiaoyu Tan

REVIEW 4 major objections 6 minor 1 cited by

Graph-MLLM: Harnessing Multimodal Large Language Models for Multimodal Graph Learning

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuned MLLMs beat graph models for multimodal node classification, even without graph structure.

desk verdict A genuinely useful three-paradigm benchmark for multimodal graph learning, but the headline claim that fine-tuned MLLMs beat graph-structured baselines is not yet supported because the comparison ignores model capacity and compute budgets. read the letter →

arxiv 2506.10282 v1 pith:A372MQ7X submitted 2025-06-12 cs.LG

Jiajin Liu , Dongzhe Fan , Jiacheng Shen , Chuanhao Ji , Daochen Zha , Qiaoyu Tan This is my paper

classification cs.LG

keywords multimodalgraphlearninglargelanguagemodelsbenchmarknodeclassificationneuralnetworksLLMsLoRAfine-tuningimage-to-textalignment

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper benchmarks how multimodal large language models (MLLMs) can be used for learning on graphs whose nodes carry both text and images. It compares three paradigms—MLLM as feature encoder for graph neural networks, MLLM as aligner that converts images to text for graph-language models, and MLLM as a standalone fine-tuned predictor—across six product and social-network datasets. The central claim is that fine-tuning a general MLLM on node-level instructions yields the highest classification accuracy in most settings, and that supplying explicit graph structure gives at best marginal gains once the MLLM is fine-tuned. The paper also finds that combining text and image node attributes consistently helps GNN-based methods, and that converting images into textual descriptions helps some graph-language models but not others. It offers the benchmark as an open-sourced framework for fair, unified evaluation of the three paradigms.

What carries the argument

The organizing device is the three-way taxonomy of MLLM roles—Encoder, Aligner, and Predictor—which structures the comparison across six datasets. The predictor result is carried by instruction fine-tuning of MLLMs with LoRA on node-level prompts, while the encoder and aligner results rest on CLIP-based contrastive encoders (including a structure-aware variant CLIP-F-S) and on Qwen-VL-generated image summaries that are folded into textual prompts for graph LLMs.

What would settle it

Run the same six datasets with per-method compute budgets (equal GPU-hours, multiple seeds, tuned hyperparameters) and report mean and standard deviation; if the best graph LLM or a structure-aware MLLM variant with a non-trivial graph encoder matches or exceeds the no-structure fine-tuned Qwen-VL accuracy, the central claim fails.

Watch

Extended reading notes

Core claim

The paper establishes that fine-tuning a general MLLM (Qwen-VL-7B, LLaVA-1.5-7B) on each node's multimodal content, without feeding any adjacency or neighbor information, achieves state-of-the-art node classification accuracy on most of the six datasets, outperforming GNNs with CLIP encoders and graph-language models such as MLaGA and UniGraph2. Injecting neighbor text or images into the MLLM prompt produces improvements of less than one percentage point in most cases, and including neighbor images often hurts. This MLLM-as-Predictor strategy is identified as the most promising of the three paradigms for multimodal graph learning, with the caveat that the value of structure awareness depends heavily on graph density and image quality.

Load-bearing premise

The comparison across paradigms is fair: all methods receive comparable compute, training budget, and hyperparameter tuning, with no seeds or error bars reported and some baseline cells omitted for time efficiency.

Editorial extensions

If this is right

MLLM-as-Predictor should become the default baseline and likely backbone for multimodal graph node classification.
Explicit graph structure is not necessary for strong performance when node text and images are rich; sparse or noisy structure can be ignored, and injecting it may degrade accuracy.
Converting node images to textual descriptions is a viable way to bring visual information into text-only graph LLMs, but its benefit depends on both the underlying model architecture and the dataset.
Structure-aware methods help mainly on dense graphs with high-quality images (such as the CDs dataset) and fail on sparse graphs or when images are weak.
Fine-tuned MLLMs without structure can match or exceed dedicated multimodal graph foundation models, implying the graph-specific inductive bias is not essential for this task class.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the central claim holds, architectural research on graph-specific models may be less urgent for node classification on attribute-rich multimodal graphs than improving instruction-tuning data and MLLM efficiency; the graph's value could lie mainly in providing more node-level supervision rather than topology.
A natural extension is to test MLLM predictors on graphs where node attributes are missing or very weak—there structure may become indispensable, and the paper's ranking would likely reverse.
The consistent degradation when neighbor images are added to prompts suggests MLLMs struggle to process multiple images coherently, so a learned graph tokenizer rather than prompt stuffing might recover the structure gains the paper finds marginal.
A fair-compute re-run that reports per-method GPU budgets, seeds, and error bars could change the headline ranking; that caution is our inference, not the paper's claim.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth a read if you work on multimodal graphs: the paper delivers a systematic benchmark of three MLLM roles (Encoder, Aligner, Predictor) across six datasets, and it ships code. The taxonomy is clean, the datasets span e-commerce and social networks, and the finding that structure-aware prompting helps little once an MLLM is fine-tuned is a real, reproducible observation. The paper is also honest about several null results, which I appreciate.

The genuinely new part is the cross-paradigm comparison. Previous benchmarks (MM-Bench, MGB) only covered GNN encoders, so extending to GraphLLMs and direct MLLM predictors is a useful service. Findings 1–7 are mostly sensible. The problem is Finding 8, the headline: fine-tuned MLLMs achieve SOTA even without graph structure. That may be true, but the evidence is confounded. Qwen-VL-7B is a 7B-parameter model fine-tuned on each dataset, while the GNN and GraphLLM baselines are far smaller and evaluated with frozen CLIP embeddings. No GPU-hours, trainable-parameter counts, or seeds are reported. The comparison is best-accuracy against best-accuracy, so you cannot separate the paradigm from the resource axis. The internal comparison within Qwen (fine-tune with vs. without neighbor info) is controlled and suggests structure gives little, but the claim that this beats graph-structured baselines is not established.

Other soft spots: Table 2 omits several cells, including MLaGA on Reddit and GraphTranslator on multiple datasets, citing time constraints; you can't know whether the omitted numbers would change the ranking. There are no error bars or seeds anywhere, which matters for accuracy differences of 1–2%. UniGraph2 is evaluated without its SPD module, and the authors don't quantify the impact. The self-citation to MLaGA is not hidden — it's reference [5] — but the text doesn't flag that it's their own model, which is a disclosure slip, not a scientific flaw.

Who is this for? Researchers choosing approaches for multimodal graph tasks and anyone building on MM-Bench. The benchmark will be useful even if Finding 8 turns out to be a capacity artifact. It deserves serious peer review, but the revision should report compute budgets, error bars, and full tables, and soften the SOTA claim accordingly.

Referee Report

4 major / 6 minor

Summary. The paper introduces Graph-MLLM, a benchmark for multimodal graph learning that organizes existing methods into three paradigms—MLLM-as-Encoder (GNN-based), MLLM-as-Aligner (GraphLLM-based), and MLLM-as-Predictor (MLLM-based)—and evaluates them on six node-classification datasets from Amazon and Reddit. The central empirical claims are that multimodal node attributes improve GNN performance, that MLLM-based image-to-text alignment has model- and dataset-specific effects, and that fine-tuned 7B MLLMs used directly as predictors achieve state-of-the-art accuracy on most datasets even without explicit graph structure. The benchmark code is open-sourced, and the taxonomy plus dataset coverage are useful contributions.

Significance. If the central claim holds, the paper would provide a valuable benchmark and a clear signal that MLLM-as-Predictor is a promising direction for multimodal graph learning. The open-source library, the six-domain evaluation, and the deliberate comparison of three paradigms are concrete strengths. However, the headline finding currently rests on cross-model comparisons that do not control for model capacity, compute, or tuning budget, and several baseline cells are missing. The result is plausible but not yet demonstrated at the level of the abstract's 'state-of-the-art in most scenarios' claim. The paper would be strengthened by resource-controlled comparisons, variance reporting, and completing the omitted baseline evaluations.

major comments (4)

[§4.5, Fig. 3; Tables 1–3] Finding 8, the paper's central claim, compares the best accuracies of fine-tuned 7B MLLMs against GNN and GraphLLM baselines without controlling for model capacity, parameter count, or compute budget. Qwen-VL-7B is a 7B-parameter model fine-tuned with LoRA, whereas the GNN baselines use frozen CLIP embeddings and the GraphLLM baselines are smaller, task-tuned models. No GPU-hours, trainable-parameter counts, seeds, or error bars are reported. Single-run differences as small as 0.01% (e.g., Table 3: 88.33 vs. 88.34 on Grocery) are treated as meaningful. As presented, the observed MLLM advantage may reflect scale rather than the paradigm, so the conclusion that fine-tuned MLLMs are state of the art 'even without explicit graph structure information' is not yet established. I would like to see either matched-capacity comparisons (e.g., larger GNN/GraphLLM backbones or a smaller MLLM) or, at minimum, a clear statement of compute budgets and variance across seeds.
[Table 2 and §4.5] Several high-performing baseline cells are missing due to 'time efficiency constraints,' including MLaGA on Reddit and GraphTranslator on Arts, CDs, and Reddit. MLaGA is identified in Finding 5 as the strongest GraphLLM, so omitting it on Reddit—one of the six datasets used in the aggregated comparison—removes the closest paradigm competitor from exactly the dataset where the fine-tuned MLLM predictor achieves its largest relative gains. The claim that fine-tuned MLLMs achieve state-of-the-art results in 'most scenarios' is therefore supported by an incomplete baseline matrix. The authors should either complete these runs or explicitly restrict the claim to the five datasets with complete MLaGA results.
[Appendix A.2, Table 1b] UniGraph2, the representative structural GNN foundation model, is evaluated without its shortest-path-distance (SPD) module, with the note that the original implementation is O(n^3). If SPD is a core component of UniGraph2's structural modeling, the benchmark understates the performance of the Encoder paradigm and weakens Finding 9, which attributes limited structure effectiveness to data properties rather than to the ablated baseline. The authors should either report UniGraph2 with the full model (or an efficient approximation) or explicitly state that the reported UniGraph2 numbers are an ablation and temper Finding 9 accordingly.
[§4.2, Findings 1–2; Table 1] Finding 2 states that structure-aware alignment 'does not consistently enable' GNN-based models, yet the paper does not report statistical significance, and several differences in Table 1 are within a few tenths of a percent (e.g., 53.48 vs. 54.11 vs. 55.04 on CDs across MGAT variants). Without seeds, standard deviations, or significance tests, the distinction between 'consistent improvement' and 'no gain' is not reliable. This affects not only Finding 2 but also the paper's broader conclusion about when graph structure helps.

minor comments (6)

[Abstract vs. Contributions (§1)] The contribution bullet says the benchmark covers 'five popular multimodal graph datasets,' but the experiments use six datasets (Movies, Toys, Grocery, Arts, CDs, Reddit). Please correct the count.
[§4.2, Finding 2] There is a duplicated word: 'Notably, Notably, in the Reddit dataset.'
[Throughout] The model name is inconsistently spelled as 'QWen-VL' (e.g., §3.2, §3.3, Appendix A.2) and 'Qwen-VL' (e.g., Table 3). Please use a single spelling.
[Figure 2 and §4.3] Figure 2 is described only as 'Different aligner augmentation methods comparison' without axis labels or a legend in the caption; the text refers to it but does not explain what is plotted. Please add a descriptive caption and define the compared settings.
[§4.4, Finding 7] The phrase 'consistent and stable improvements' is inaccurate when Qwen-VL-7B improves on only 3 out of 6 datasets with structure-aware fine-tuning. Please rephrase to convey the mixed results.
[§3.3 and Appendix A.2] The paper says it 'adheres to official guidelines for LoRA fine-tuning strictly' but does not report LoRA rank, learning rate, batch size, or training steps in the main text or appendix. Including these hyperparameters would improve reproducibility and help readers assess the resource-control issue in the major comments.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark's findings are empirical comparisons, not derivations built from their own inputs.

full rationale

This paper is an empirical benchmark study rather than a derivation, so the standard circularity patterns do not apply. The central claim (Finding 8, Section 4.5) that fine-tuned MLLM predictors achieve state-of-the-art results even without explicit graph structure is supported by held-out test accuracies in Tables 1-3, computed under the same train/validation/test splits as the GNN and GraphLLM baselines. No equation in the paper defines a predicted quantity in terms of the fitted parameter that is then reported as the prediction, and no load-bearing argument reduces to a self-citation. The inclusion of MLaGA [5], prior work by the same authors, is a self-citation, but MLaGA is used as a baseline to be compared against rather than as the source of the paper's main conclusion; the fine-tuned MLLM results are generated independently. Concerns about omitted Table 2 cells, lack of error bars, or unequal compute budgets are experimental-fairness issues, not circularity, and do not make the comparison circular by construction.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The paper's conclusions rest on the representativeness of the six public datasets, the adequacy of CLIP as a universal multimodal encoder, the correct deployment of official implementations, and the fairness of cross-paradigm comparisons; none of these are independently established in the text.

free parameters (4)

Temperature tau in structure-aware contrastive loss (Eq. 5) = not reported
Hand-set temperature parameter for the contrastive objective used in PMM-F-S; no value or tuning procedure is given.
Neighbor sample count m in PMM-F-S = not reported
Number of 1-hop neighbors randomly sampled for each central node in Eq. 5; no value is stated.
Top-k neighbors in structure-aware prompts = 3
Appendix Tables 7 and 9 show three neighbor slots in prompts; this is a hand-chosen constant that affects the structure-aware predictor results.
LoRA rank for MLLM fine-tuning = not reported
LoRA hyperparameter for Qwen-VL and LLaVA adaptation; no rank, learning rate, or epochs are reported.

assumptions (6)

domain assumption Node classification accuracy is the evaluation metric for MMGL.
All results are reported as accuracy only; other tasks like link prediction are not evaluated.
domain assumption The six Amazon and Reddit datasets are representative of multimodal graphs.
Claims about structure density and image quality are generalized from these datasets.
domain assumption CLIP embeddings represent text and image semantics sufficiently for GNN pipelines.
CLIP is used as the standard encoder for Table 1 and for embedding-based GraphLLMs; no validation of embedding quality is provided.
domain assumption Official implementations of prior methods were deployed as intended.
The benchmark states evaluations follow official implementations; deviations such as UniGraph2 without SPD are noted but not analyzed.
domain assumption The 60/20/20 split is a valid protocol.
Standard split, but no seed or cross-validation is reported.
domain assumption LoRA fine-tuning follows official guidelines.
Used to adapt 7B MLLMs; no details on rank, learning rate, or epochs are given.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Graph-MLLM: Harnessing Multimodal Large Language Models for Multimodal Graph Learning." pith.science (2026). https://pith.science/paper/A372MQ7X

@misc{pith2026250610282,
  author       = {Pith},
  title        = {Pith review of: Graph-MLLM: Harnessing Multimodal Large Language Models for Multimodal Graph Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A372MQ7X}},
  note         = {Machine review of arXiv:2506.10282}
}

read the original abstract

Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in representing and understanding diverse modalities. However, they typically focus on modality alignment in a pairwise manner while overlooking structural relationships across data points. Integrating multimodality with structured graph information (i.e., multimodal graphs, MMGs) is essential for real-world applications such as social networks, healthcare, and recommendation systems. Existing MMG learning methods fall into three paradigms based on how they leverage MLLMs: Encoder, Aligner, and Predictor. MLLM-as-Encoder focuses on enhancing graph neural networks (GNNs) via multimodal feature fusion; MLLM-as-Aligner aligns multimodal attributes in language or hidden space to enable LLM-based graph reasoning; MLLM-as-Predictor treats MLLMs as standalone reasoners with in-context learning or fine-tuning. Despite their advances, the MMG field lacks a unified benchmark to fairly evaluate across these approaches, making it unclear what progress has been made. To bridge this gap, we present Graph-MLLM, a comprehensive benchmark for multimodal graph learning by systematically evaluating these three paradigms across six datasets with different domains. Through extensive experiments, we observe that jointly considering the visual and textual attributes of the nodes benefits graph learning, even when using pre-trained text-to-image alignment models (e.g., CLIP) as encoders. We also find that converting visual attributes into textual descriptions further improves performance compared to directly using visual inputs. Moreover, we observe that fine-tuning MLLMs on specific MMGs can achieve state-of-the-art results in most scenarios, even without explicit graph structure information. We hope that our open-sourced library will facilitate rapid, equitable evaluation and inspire further innovative research in this field.

Figures

Figures reproduced from arXiv: 2506.10282 by the authors.

**Figure 2.** Different aligner augmentation methods comparison ★ Finding 3: The performance of aligners exhibits model-specific sensitivity. An analysis of results across different models reveals a clear contrast in the effectiveness of image-totext augmentation. While GraphGPT and GraphPrompter show consistent performance gains in 4 out of 6 and 3 out of 5 datasets, respectively, with aligner augmentation, LLaGA exhibits perf… view at source ↗

**Figure 4.** Extra performance gain of structure-aware to non-structure methods. ★ Finding 8: MLLM-as-Predictor demonstrates the greatest potential for MMGL, while both MLLM-as-Encoder and MLLM-as-Aligner generally lead to performance improvements. Without the involvement of MLLMs, GraphLLMs consistently outperform both GNN-based methods and general-purpose MLLMs across all datasets, highlighting the advantages of integrating la… view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

One Model, Many Graphs: Learning over Attributed Graphs across Heterogeneous Modalities with Vision-Language Models
cs.LG 2026-07 conditional novelty 6.0 of 10

OMG-VLM is a single VLM-based model that handles text-, image-, and multi-attributed graphs through structure-aware adapters, reporting gains on several node/link prediction benchmarks.

Reference graph

Works this paper leans on

36 extracted references · 9 canonical work pages · cited by 1 Pith paper

[5]

Mlaga: Multimodal large language and graph assistant.arXiv preprint arXiv:2506.02568, 2025

Dongzhe Fan, Yi Fang, Jiajin Liu, Djellel Difallah, and Qiaoyu Tan. Mlaga: Multimodal large language and graph assistant.arXiv preprint arXiv:2506.02568, 2025. URL https: //arxiv.org/abs/2506.02568

arXiv 2025
[1]

Qwen technical report.arXiv preprint arXiv:2309.16609, 2023

Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report.arXiv preprint arXiv:2309.16609, 2023

arXiv 2023
[2]

Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023

Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023. URL https: //arxiv.org/abs/2308.12966

arXiv 2023
[3]

Llaga: Large language and graph assistant.arXiv preprint arXiv:2402.08170, 2024

Runjin Chen, Tong Zhao, Ajay Jaiswal, Neil Shah, and Zhangyang Wang. Llaga: Large language and graph assistant.arXiv preprint arXiv:2402.08170, 2024

arXiv 2024
[4]

Node feature extraction by self-supervised multi-scale neighborhood prediction.arXiv preprint arXiv:2111.00064, 2021

Eli Chien, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Jiong Zhang, Olgica Milenkovic, and Inderjit S Dhillon. Node feature extraction by self-supervised multi-scale neighborhood prediction.arXiv preprint arXiv:2111.00064, 2021

arXiv 2021
[6]

Gaugllm: Improving graph contrastive learning for text-attributed graphs with large language models

Yi Fang, Dongzhe Fan, Daochen Zha, and Qiaoyu Tan. Gaugllm: Improving graph contrastive learning for text-attributed graphs with large language models. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 747–758, 2024

work page 2024
[7]

Imagebind: One embedding space to bind them all

Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind: One embedding space to bind them all. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15180–15190, 2023

2023
[8]

From images to textual prompts: Zero-shot vqa with frozen large language models.arXiv preprint arXiv:2212.10846, 2022

Jiaxian Guo, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Boyang Li, Dacheng Tao, and Steven CH Hoi. From images to textual prompts: Zero-shot vqa with frozen large language models.arXiv preprint arXiv:2212.10846, 2022

arXiv 2022

Show all 36 references

[9]

Inductive representation learning on large graphs.Advances in neural information processing systems, 30, 2017

Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs.Advances in neural information processing systems, 30, 2017

2017
[10]

Unigraph: Learning a cross-domain graph foundation model from natural language.arXiv e-prints, pages arXiv–2402, 2024

Yufei He and Bryan Hooi. Unigraph: Learning a cross-domain graph foundation model from natural language.arXiv e-prints, pages arXiv–2402, 2024

2024
[11]

Unigraph2: Learning a unified embedding space to bind multimodal graphs

Yufei He, Yuan Sui, Xiaoxin He, Yue Liu, Yifei Sun, and Bryan Hooi. Unigraph2: Learning a unified embedding space to bind multimodal graphs. InProceedings of the ACM on Web Conference 2025, pages 1759–1770, 2025

2025
[12]

Large language models on graphs: A comprehensive survey.IEEE Transactions on Knowledge and Data Engineering, 2024

Bowen Jin, Gang Liu, Chi Han, Meng Jiang, Heng Ji, and Jiawei Han. Large language models on graphs: A comprehensive survey.IEEE Transactions on Knowledge and Data Engineering, 2024

2024
[13]

Snap datasets: Stanford large network dataset collection.Retrieved December 2021 from http://snap

Leskovec Jure. Snap datasets: Stanford large network dataset collection.Retrieved December 2021 from http://snap. stanford. edu/data, 2014

2021
[14]

Semi-supervised classification with graph convolutional networks.arXiv preprint arXiv:1609.02907, 2016

Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks.arXiv preprint arXiv:1609.02907, 2016

2016 arXiv
[15]

Junnan Li, Dongxu Li, Caiming Xiong, and Steven C. H. Hoi. BLIP: bootstrapping language-image pre-training for unified vision-language understanding and generation.CoRR, abs/2201.12086, 2022. URLhttps://arxiv.org/abs/2201.12086

2022 arXiv
[16]

Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. InInternational conference on machine learning, pages 19730–19742. PMLR, 2023

2023
[17]

Glbench: A comprehensive benchmark for graph with large language models

Yuhan Li, Peisong Wang, Xiao Zhu, Aochuan Chen, Haiyun Jiang, Deng Cai, Victor Wai Kin Chan, and Jia Li. Glbench: A comprehensive benchmark for graph with large language models. arXiv preprint arXiv:2407.07457, 2024. 10

2024 arXiv
[18]

Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

2023
[19]

Can we soft prompt llms for graph learning tasks? InCompanion Proceedings of the ACM Web Conference 2024, pages 481–484, 2024

Zheyuan Liu, Xiaoxin He, Yijun Tian, and Nitesh V Chawla. Can we soft prompt llms for graph learning tasks? InCompanion Proceedings of the ACM Web Conference 2024, pages 481–484, 2024

2024
[20]

Gpt-4 technical report, 2024

OpenAI. Gpt-4 technical report, 2024. URLhttps://arxiv.org/abs/2303.08774

2024 arXiv
[21]

Trustglm: Evaluating the robustness of graphllms against prompt, text, and structure attacks

Zhang Qihai, Xinyue Sheng, Yuanfu Sun, and Qiaoyu Tan. Trustglm: Evaluating the robustness of graphllms against prompt, text, and structure attacks. InProceedings of the 31th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2025

2025
[22]

Learning transferable visual models from natural language supervision

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...

2021
[23]

A survey of large language models for graphs

Xubin Ren, Jiabin Tang, Dawei Yin, Nitesh Chawla, and Chao Huang. A survey of large language models for graphs. InProceedings of the 30th ACM SIGKDD Conference on Knowl- edge Discovery and Data Mining, KDD ’24, page 6616–6626, New York, NY , USA, 2024. Association for Computin...

2024
[24]

Rosenblatt

F. Rosenblatt. The perceptron: A probabilistic model for information storage and organization in the brain.Psychological Review, 65(6):386–408, 1958. ISSN 0033-295X. doi: 10.1037/ h0042519. URLhttp://dx.doi.org/10.1037/h0042519

1958 doi
[25]

The graph neural network model.IEEE Transactions on Neural Networks, 20(1):61–80, 2009

Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model.IEEE Transactions on Neural Networks, 20(1):61–80, 2009. doi: 10.1109/TNN.2008.2005605

2009
[26]

Yuanfu Sun, Zhengnan Ma, Yi Fang, Jing Ma, and Qiaoyu Tan. Graphicl: Unlocking graph learning potential in llms through structured prompt design.Findings of the Association for Computational Linguistics: NAACL 2025, Albuquerque, New Mexico, USA, April 29 - May 4, 2025, 2025

2025
[27]

Graphgpt: Graph instruction tuning for large language models

Jiabin Tang, Yuhao Yang, Wei Wei, Lei Shi, Lixin Su, Suqi Cheng, Dawei Yin, and Chao Huang. Graphgpt: Graph instruction tuning for large language models. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 49...

2024
[28]

Mgat: Multimodal graph attention network for recommendation.Information Processing & Management, 57(5):102277, 2020

Zhulin Tao, Yinwei Wei, Xiang Wang, Xiangnan He, Xianglin Huang, and Tat-Seng Chua. Mgat: Multimodal graph attention network for recommendation.Information Processing & Management, 57(5):102277, 2020

2020
[29]

Gemini: A family of highly capable multimodal models, 2025

Gemini Team. Gemini: A family of highly capable multimodal models, 2025. URL https: //arxiv.org/abs/2312.11805

2025 arXiv
[30]

Graph attention networks.arXiv preprint arXiv:1710.10903, 2017

Petar Veliˇckovi´c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks.arXiv preprint arXiv:1710.10903, 2017

2017 arXiv
[31]

Mmgcn: Multi-modal graph convolution network for personalized recommendation of micro- video

Yinwei Wei, Xiang Wang, Liqiang Nie, Xiangnan He, Richang Hong, and Tat-Seng Chua. Mmgcn: Multi-modal graph convolution network for personalized recommendation of micro- video. InProceedings of the 27th ACM International Conference on Multimedia, MM ’19, page 1437–1445, New Yo...

2019
[32]

When graph meets multimodal: Benchmarking on multimodal attributed graphs learning.arXiv preprint arXiv:2410.09132, 2024

Hao Yan, Chaozhuo Li, Zhigang Yu, Jun Yin, Ruochen Liu, Peiyan Zhang, Weihao Han, Mingzheng Li, Zhengxin Zeng, Hao Sun, et al. When graph meets multimodal: Benchmarking on multimodal attributed graphs learning.arXiv preprint arXiv:2410.09132, 2024. 11

arXiv
[33]

Language is all a graph needs.arXiv preprint arXiv:2308.07134, 2023

Ruosong Ye, Caiqi Zhang, Runhui Wang, Shuyuan Xu, and Yongfeng Zhang. Language is all a graph needs.arXiv preprint arXiv:2308.07134, 2023

2023 arXiv
[34]

Graphtranslator: Aligning graph model to large language model for open-ended tasks

Mengmei Zhang, Mingwei Sun, Peng Wang, Shen Fan, Yanhu Mo, Xiaoxiao Xu, Hong Liu, Cheng Yang, and Chuan Shi. Graphtranslator: Aligning graph model to large language model for open-ended tasks. InProceedings of the ACM Web Conference 2024, pages 1003–1014, 2024

2024
[35]

Minigpt-4: En- hancing vision-language understanding with advanced large language models.arXiv preprint arXiv:2304.10592, 2023

Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: En- hancing vision-language understanding with advanced large language models.arXiv preprint arXiv:2304.10592, 2023

2023 arXiv
[36]

Multimodal graph benchmark.arXiv preprint arXiv:2406.16321, 2024

Jing Zhu, Yuhang Zhou, Shengyi Qian, Zhongmou He, Tong Zhao, Neil Shah, and Danai Koutra. Multimodal graph benchmark.arXiv preprint arXiv:2406.16321, 2024. 12 A Appendix A.1 Dataset Statistics Our experimental benchmark comprises six datasets spanning two distinct domains, wit...

2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [5]

Mlaga: Multimodal large language and graph assistant.arXiv preprint arXiv:2506.02568, 2025

Dongzhe Fan, Yi Fang, Jiajin Liu, Djellel Difallah, and Qiaoyu Tan. Mlaga: Multimodal large language and graph assistant.arXiv preprint arXiv:2506.02568, 2025. URL https: //arxiv.org/abs/2506.02568

arXiv 2025

[2] [1]

Qwen technical report.arXiv preprint arXiv:2309.16609, 2023

Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report.arXiv preprint arXiv:2309.16609, 2023

arXiv 2023

[3] [2]

Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023

Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023. URL https: //arxiv.org/abs/2308.12966

arXiv 2023

[4] [3]

Llaga: Large language and graph assistant.arXiv preprint arXiv:2402.08170, 2024

Runjin Chen, Tong Zhao, Ajay Jaiswal, Neil Shah, and Zhangyang Wang. Llaga: Large language and graph assistant.arXiv preprint arXiv:2402.08170, 2024

arXiv 2024

[5] [4]

Node feature extraction by self-supervised multi-scale neighborhood prediction.arXiv preprint arXiv:2111.00064, 2021

Eli Chien, Wei-Cheng Chang, Cho-Jui Hsieh, Hsiang-Fu Yu, Jiong Zhang, Olgica Milenkovic, and Inderjit S Dhillon. Node feature extraction by self-supervised multi-scale neighborhood prediction.arXiv preprint arXiv:2111.00064, 2021

arXiv 2021

[6] [6]

Gaugllm: Improving graph contrastive learning for text-attributed graphs with large language models

Yi Fang, Dongzhe Fan, Daochen Zha, and Qiaoyu Tan. Gaugllm: Improving graph contrastive learning for text-attributed graphs with large language models. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 747–758, 2024

work page 2024

[7] [7]

Imagebind: One embedding space to bind them all

Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind: One embedding space to bind them all. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15180–15190, 2023

2023

[8] [8]

From images to textual prompts: Zero-shot vqa with frozen large language models.arXiv preprint arXiv:2212.10846, 2022

Jiaxian Guo, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Boyang Li, Dacheng Tao, and Steven CH Hoi. From images to textual prompts: Zero-shot vqa with frozen large language models.arXiv preprint arXiv:2212.10846, 2022

arXiv 2022