Learning Accurate, Efficient, and Interpretable MLPs on Multiplex Graphs via Node-wise Multi-View Ensemble Distillation

Jianhua Zhao; Tao Zheng; Tieke He; Xiang Zhao; Yunhui Liu; Zhen Tao

REVIEW 3 major objections 5 minor 31 references

Learning Accurate, Efficient, and Interpretable MLPs on Multiplex Graphs via Node-wise Multi-View Ensemble Distillation

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that multiplex-GNN knowledge can be moved into a feature-only MLP by distilling each view's soft labels with per-node weights, matching or beating the teacher on five of six transductive datasets while running…

desk verdict First multiplex GNN-to-MLP distillation with a neat low-rank node-wise ensemble, but the headline accuracy claim is built on a transductive protocol that leaks teacher predictions into the evaluation nodes; the inductive numbers tell a more cautious story. read the letter →

arxiv 2502.05864 v1 pith:AS57BCWM submitted 2025-02-09 cs.LG

Yunhui Liu , Zhen Tao , Xiang Zhao , Jianhua Zhao , Tao Zheng , Tieke He This is my paper · ORCID

classification cs.LG

keywords knowledgedistillationmultiplexgraphneuralnetworksGNN-to-MLPinferenceaccelerationnode-wisemulti-viewensemblelow-rankreparameterizationgraphsefficiencyandaccuracy

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper seeks to settle a deployment trade-off: multiplex graph neural networks (MGNNs) are accurate but slow at inference because they must fetch and aggregate neighbors across every edge type, while plain MLPs are fast but weak. Its proposal is to distill the teacher MGNN's knowledge into an MLP student that sees only node features, so that fast inference keeps most of the accuracy. The first model, MGFNN, does this with the teacher's final soft labels alone; the second, MGFNN+, also distills each view-specific GNN's own soft labels and learns a per-node weight vector that decides how much to trust each view. The reported result is that the student matches or beats the teacher on five of six transductive datasets, improves over vanilla MLPs by about ten percentage points on average, and runs 35.40×–89.14× faster than the teacher. If the claim holds, latency-sensitive applications on multiplex graphs can switch to feature-only models without sacrificing accuracy.

What carries the argument

The load-bearing mechanism is a low-rank reparameterization of the node-wise ensemble coefficient matrix $C \in \mathbb{R}^{n \times (r+1)}$, where $n$ is the number of nodes and $r$ the number of views. Rather than learning one weight per node per teacher, which would cost $O(n(r+1))$ parameters and be hard to optimize because each row receives gradients only from its own node, the paper factors $C = ST$: $T \in \mathbb{R}^{m \times (r+1)}$ holds $m$ globally shared 'base view-wise weight assigners,' and $S = \tanh(HW)$ is computed from the student MLP's last hidden layer $H$, so each node's coefficient vector is a weighted combination of the base assigners. This reduces the parameter cost to $O(m(r+1) + hm)$, makes every coefficient differentiable with respect to shared weights $W$, and ties the coefficients to node features, which is what makes them interpretable per node. A mean-entropy regularization term $\mathcal{H}(\bar{c})$ on the average coefficient vector stops the optimizer from collapsing a node onto a single teacher, preserving diversity among the distilled views.

What would settle it

Retrain on a multiplex graph whose labels are a pure function of network position (for instance, class equals degree bucket or triangle membership) with node features randomly shuffled so features carry no label information: the paper's account predicts MGFNN+ collapses toward the vanilla MLP level while the teacher keeps its accuracy. Observing instead that the student tracks the teacher would refute the paper's explanation of why the method works.

Watch

Extended reading notes

Core claim

The central claim is that the teacher's fused prediction $z_v$ is a bottleneck rather than a ceiling. Because an MGNN's integration layer combines the view-specific predictions $z_v^i$ with global, node-agnostic weights, it discards node-specific information; a student that learns its own per-node mixing of the same view predictions can therefore end up at or above the teacher's accuracy. An oracle experiment supports this premise: on ACM, an ideal ensemble that is correct whenever any one view-specific GNN is correct reaches 97.55% versus the fused teacher's 87.53%, quantifying the lost headroom. MGFNN+ harvests this headroom by treating each view-specific GNN together with the whole MGNN as $r+1$ teachers, minimizing a per-node KL loss $\sum_i c_v^i \mathrm{KL}(\hat{y}_v, z_v^i)$ with coefficients learned from the student's own hidden features and regularized by mean-entropy maximization. In the reported experiments the student matches or beats the teacher on five of six transductive datasets (89.10% versus 87.92% on ACM) and remains competitive on five of six production settings, while the inductive columns on the two largest datasets, ArXiv and MAG, drop sharply (66.75% versus 77.99% on ArXiv), which the paper attributes to a training-test distribution shift that feature-only students cannot bridge without neighbor information.

Load-bearing premise

The load-bearing premise, stated in Section 4.3, is that node features and structural roles are highly correlated in real graphs; the method works only if a node's features alone already carry most of the information needed to predict its label, and if labels are decided by network position, such as degree or triangle membership, the feature-only student has nothing to learn from and the accuracy advantage over a plain MLP disappears.

Editorial extensions

If this is right

Distillation can make a feature-only student not merely match but surpass its multiplex-GNN teacher: MGFNN+ beats RSAGE on five of six transductive datasets by 0.56–2.09 points, because per-node ensembling escapes the teacher's fused-prediction bottleneck.
Inference cost becomes almost dataset-independent: MGFNNs take about 0.18 ms per ten nodes on every dataset, yielding 35.40×–89.14× speedups over the RSAGE teacher and roughly 35× over a neighbor-sampled teacher, which is what latency-sensitive graph services need.
The distillation transfers across teacher architectures: with RGCN, RGAT, or HAN as the teacher instead of RSAGE, MGFNN+ still surpasses each teacher's transductive accuracy on the tested datasets (ACM, IMDB, and ArXiv).
The learned per-node coefficients are usable as an interpretability tool: on MAG, for example, node $v_1$ weights the PAP view at 0.3881 while node $v_3$ weights PSP at 0.4058, showing which edge type each node actually relies on.
The boundary of the accuracy parity is inductive generalization on large, shifting graphs: on the ArXiv and MAG inductive splits the student trails the teacher by 11.24 and 7.14 points respectively, so the method's promise is strongest where test nodes resemble training nodes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The oracle gap the paper quantifies (an ideal ensemble at 97.55% versus the fused teacher's 87.53% on ACM) indicates the teacher's fusion layer, not the view encoders, is the accuracy ceiling; a student trained on per-view logits could plausibly be pushed closer to that ideal bound than the 1–2 points MGFNN+ presently gains, since the view predictions disagree enough to correct one another.
Because the base assigners $T$ are node-agnostic, they could in principle transfer across multiplex graphs that share a feature space, letting a new domain initialize its ensemble behavior after seeing only a few nodes — an extension the paper does not test.
The same per-node coefficients can serve as a view-relevance diagnostic: a node whose weight concentrates on one view is likely poorly served by the other edge types, so the mechanism could flag noisy or misaligned views in a multiplex dataset before deployment.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know two things about this paper. First, it does something genuinely new: it is the first GNN-to-MLP distillation framework aimed at multiplex graphs, and the low-rank reparameterization of node-wise ensemble coefficients is a clean, practical trick. Second, the central accuracy claim is not as solid as the abstract suggests. The transductive results—where MGFNN+ matches or beats the MGNN teacher on 5/6 datasets—are obtained by training the student on teacher soft labels for every node, including the test nodes. That is not a generalization measurement; it is fitting to pseudo-labels. The inductive results in Table 3 are the honest ones, and there the student falls behind RSAGE by 3.9 to 11.2 points across all six datasets. The paper's "prod = 0.2*ind + 0.8*tran" metric hides this, and calling the method "accurate" without qualification is an overclaim.

What is good: the motivation is sound, the method is clearly explained, the ablation against view-wise ensemble variants (MEAN, PARA, ATTN, GRAD) is reasonable, and the speedups of 35-89x are real. The code is public. The low-rank coefficient learning is an elegant way to avoid O(n) parameters while staying node-adaptive. All of that deserves credit.

Soft spots, in rough order of importance. The evaluation protocol is the main issue: if the deployment scenario is latency-sensitive inference on new nodes, the inductive accuracy is the relevant metric, and it is consistently worse. The discussion of ArXiv/MAG attributes the gap to distribution shift, but the same pattern appears on the small datasets too, so that explanation is incomplete. Second, there is a mathematical inconsistency in the entropy regularization: c is defined as a scalar average, then H(c) is written as a sum over i of c_i log c_i. The intent is probably a per-view average coefficient vector, but as written it is incoherent. Fixable, but sloppy. Third, the paper does not discuss its relationship to the authors' own prior work on heterogeneous graph distillation (ref [12]), which may be a close cousin. Fourth, there are no adapted strong baselines like NOSMOG or VQGraph for multiplex graphs, so the comparison to the state of the art in GNN-to-MLP distillation is incomplete.

Who is this for? Researchers working on graph inference acceleration and practitioners who care about multiplex graphs in latency-sensitive settings. It deserves a serious referee, but the referee should push for a major revision: report inductive accuracy as the primary result, reframe the claims, fix the entropy notation, and clarify the relationship with [12]. If those are addressed, the paper would be a solid contribution; as is, the headline does not match the evidence.

Referee Report

3 major / 5 minor

Summary. The paper proposes MGFNN and MGFNN+, MLP students distilled from multiplex GNN (MGNN) teachers. MGFNN distills from the teacher's final soft labels, while MGFNN+ additionally distills per-view soft labels using node-wise ensemble coefficients learned through a low-rank reparameterization with entropy regularization. Experiments on six multiplex graph datasets report accuracy close to or above the teacher MGNN in transductive and 'production' settings, together with 35.40x–89.14x inference speedups over the teacher, and visualizations of node-wise coefficients for interpretability.

Significance. If the accuracy claims are properly scoped, the paper makes a useful contribution to GNN-to-MLP distillation for multiplex graphs. Its concrete strengths are that it is the first to target multiplex GNNs in this distillation paradigm, the low-rank node-wise coefficient design gives a parameter-efficient way to combine per-view teachers, the ablation against view-wise ensemble variants is informative, and the evaluation covers six datasets, multiple teacher architectures, inductive split rates, and feature noise. The released code is another positive. However, the headline 'accurate' claim is currently supported mainly by transductive numbers in which every evaluation node's teacher soft label is used during student training, and the inductive results show consistent deficits; this makes the deployment claim, and the unqualified abstract language, stronger than the evidence justifies.

major comments (3)

[Section 5.1, Table 3, Abstract] The transductive protocol generates soft labels for every node in V (including the nodes later used for evaluation), and Eq. (6) trains the student on the KL term over all v in V. Thus the reported transductive accuracy measures fit to teacher pseudo-labels on the evaluation nodes, not generalization to unseen queries. In the inductive rows of Table 3, MGFNN+ trails RSAGE by 5.32, 3.87, 4.65, 4.75, 11.24, and 7.14 points on ACM, IMDB, IMDB5K, DBLP, ArXiv, and MAG. The interpolated metric prod = 0.2*ind + 0.8*tran dilutes these deficits, and the abstract's unqualified 'accurate' claim omits them. The deployment claim should be restricted to the transductive setting unless the inductive behavior is analyzed and addressed.
[Section 4.2, Eq. (6)] The entropy regularization is not well-defined as written. The text defines c = (1/(n(r+1))) sum_{v,i} c_i^v, which is a scalar, but then writes H(c) = - sum_{i=1}^{r+1} c_i log c_i, which requires c_i as a vector of per-view averages. The loss in Eq. (6) is therefore ambiguous and cannot be reproduced as a precise objective. Please define c_i = (1/n) sum_v c_i^v, state the normalization (e.g., whether c is a probability vector), and rewrite Eq. (6) accordingly.
[Section 4.3, Section 5.2, Table 3] The information-theoretic justification in Section 4.3 rests on the assumption that I(X[v]; y_v | E[v]) is large, but the paper provides no empirical measure of this quantity or of feature-structure correlation. The paper's own inductive results on ArXiv and MAG (Table 3) show the regime where the assumption fails, and Section 5.2 attributes the drop to 'distribution shift.' This is an internal tension: the paper uses the transductive setting, where teacher soft labels for test nodes are available, to support a general 'accurate' claim, while the inductive setting is the one most relevant to the latency-sensitive deployment scenario. Please either add an analysis of when feature-only students can match MGNNs or substantially qualify the claims.

minor comments (5)

[Section 4.2, Eq. (5)] The text says 'we decompose C = ST, where U in R^{n x m} and T in R^{m x (r+1)}' but the first matrix is later used as S. Please fix the variable name to S.
[Section 1, Contributions] The last contribution bullet contains a duplicated phrase: 'Experiments on six datasets show MGFNNs that MGFNNs achieve...' Please correct this typo.
[Section 4.2] The sentence 'these view-wise coefficients only reflect the global importance of each view without but make no specific discrimination for individual nodes' has a grammatical error and should be rewritten.
[Table 2 caption] The caption contains a typo: 'Classificatiom' should be 'Classification'.
[Figure 5(b)] The y-axis appears to show duplicated tick labels ('60 65 70' repeated). Please check the figure export.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity: distillation targets are teacher logits and accuracy is scored on true labels, but the transductive component of the headline accuracy is fitted to teacher soft labels on the evaluation nodes, and the only self-citation is contextual.

full rationale

The paper's derivation chain is not circular. MGFNN/MGFNN+ train an MLP student against soft labels produced by independently trained teacher MGNNs; the student's reported accuracy is measured on ground-truth labels, not on the distillation loss, so the central accuracy/efficiency comparison is externally evaluated. The node-wise ensemble coefficients are learned parameters, not fitted constants relabeled as predictions; their 'interpretability' is a visualization of the learned weights. The only self-citation is reference [12] (Liu et al., arXiv:2411.14035), which appears in Related Work as one of several GNN-to-MLP distillation frameworks and is not used to justify the method or to forbid alternatives; it is therefore not load-bearing. One caveat is worth noting but does not constitute derivation-level circularity: in the transductive setting (Section 5.1) soft labels are generated for every node v in V, and Eq. (6) trains the student with KL terms over all v in V, so the transductive test accuracy partly measures the student's fidelity to teacher logits on the evaluation nodes. The paper's own inductive results (Table 3, ind rows) show MGFNN+ below RSAGE by 3.87-11.24 points, and the production metric 'prod = 0.2*ind + 0.8*tran' dilutes these deficits; this is a limitation of the evaluation protocol, not a circular derivation. The inductive experiments, where teacher soft labels are withheld, provide an independent check and are honestly reported. Efficiency gains (35.40x-89.14x) and the multi-view ensemble mechanism are independent of any circularity concern.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim depends on hyperparameters chosen by validation (m, gamma, lambda, hidden size, weight decay) and on the domain assumption that node features are informative enough to support labels. No new physical or conceptual entities are introduced; the node-wise coefficients are learned model parameters.

free parameters (5)

lambda (trade-off weight) = 0 (fixed)
Set to 0 following GLNN [29]; the student is trained purely by distillation, with no true-label cross-entropy.
low-rank dimension m = searched over {1,2,3}
Controls capacity of node-wise coefficient decomposition; chosen by validation accuracy.
entropy regularization weight gamma = searched over {0.1,0.01,0.001}
Controls strength of entropy maximization on average coefficients; chosen by validation accuracy.
hidden dimension = 128 for small datasets, 1024 for ArXiv and MAG
Architectural choice; affects capacity and results.
weight decay = selected from {1e-3, 5e-3, 5e-4, 0}
Regularization hyperparameter for all models, chosen per dataset.

assumptions (4)

domain assumption Node features and structural roles are highly correlated in real-world node classification, making I(X[v]; y_v | E[v]) large enough to support MLP-only inference.
Section 4.3 justifies the viability of MGFNNs; if false, the student MLP cannot recover structural labels.
domain assumption The teacher MGNN is well-trained and its per-view logits provide useful, partially complementary information beyond the integrated logits.
Motivates multi-view ensemble distillation; supported by Figure 2 oracle experiment but not guaranteed in general.
ad hoc to paper Low-rank factorization with S = tanh(HW) can represent a sufficiently expressive node-wise coefficient matrix.
Section 4.2 introduces this reparameterization; it is a modeling choice without independent justification.
standard math Standard facts from linear algebra and KL divergence are used to justify the distillation objective.
Eqs. (2)-(6) rely on basic properties of cross-entropy, KL divergence, and matrix multiplication.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Learning Accurate, Efficient, and Interpretable MLPs on Multiplex Graphs via Node-wise Multi-View Ensemble Distillation." pith.science (2026). https://pith.science/paper/AS57BCWM

@misc{pith2026250205864,
  author       = {Pith},
  title        = {Pith review of: Learning Accurate, Efficient, and Interpretable MLPs on Multiplex Graphs via Node-wise Multi-View Ensemble Distillation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AS57BCWM}},
  note         = {Machine review of arXiv:2502.05864}
}

abstract

Multiplex graphs, with multiple edge types (graph views) among common nodes, provide richer structural semantics and better modeling capabilities. Multiplex Graph Neural Networks (MGNNs), typically comprising view-specific GNNs and a multi-view integration layer, have achieved advanced performance in various downstream tasks. However, their reliance on neighborhood aggregation poses challenges for deployment in latency-sensitive applications. Motivated by recent GNN-to-MLP knowledge distillation frameworks, we propose Multiplex Graph-Free Neural Networks (MGFNN and MGFNN+) to combine MGNNs' superior performance and MLPs' efficient inference via knowledge distillation. MGFNN directly trains student MLPs with node features as input and soft labels from teacher MGNNs as targets. MGFNN+ further employs a low-rank approximation-based reparameterization to learn node-wise coefficients, enabling adaptive knowledge ensemble from each view-specific GNN. This node-wise multi-view ensemble distillation strategy allows student MLPs to learn more informative multiplex semantic knowledge for different nodes. Experiments show that MGFNNs achieve average accuracy improvements of about 10% over vanilla MLPs and perform comparably or even better to teacher MGNNs (accurate); MGFNNs achieve a 35.40$\times$-89.14$\times$ speedup in inference over MGNNs (efficient); MGFNN+ adaptively assigns different coefficients for multi-view ensemble distillation regarding different nodes (interpretable).

Figures

Figures reproduced from arXiv: 2502.05864 by the authors.

**Figure 1.** The number of nodes fetched and inference time of MGNNs are both magnitudes more than MLPs and grow exponentially with the number of layers. (a) The total number of nodes fetched for inference. (b) The total inference time. (Inductive inference for 10 random nodes on MAG.) introduces a multi-view ensemble distillation strategy, which adaptively injects multiplex semantic knowledge from view-specific GNNs into studen… view at source ↗

**Figure 2.** Classification accuracy of MGNN, each view-specific GNN, and the ideal ensemble classifier on ACM, IMDB, and MAG. Empirical Analysis. To verify our argument, we conduct an exploratory experiment from an oracle perspective. Specifically, we first evaluate the performance of each view-specific GNN in a well-trained MGNN (instantiated as RSAGE). We then compute the performance of an ideal ensemble classifier, which c… view at source ↗

**Figure 3.** Visualization of learned node-wise ensemble coefficients for 6 randomly selected nodes on ACM, IMDB, and MAG. 5.5 Ablation Studies (RQ4) To show the superiority of our node-wise multi-view ensemble distillation, we substitute node-wise ensemble coefficients in Eq. (4) with view-wise ensemble coefficients in Eq. (3). These coefficients can be uniformly set to 1/(r+1) (MEAN), defined as (r+ 1)-dimensional learnable p… view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Transductive Accuracy vs. Teacher MGNN Architectures. MGFNNs can learn from different MGNN teachers to improve over MLPs and achieve comparable results. Inductive Split Rate In [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]

**Figure 5.** Figure 5: shows that altering the inductive:transductive ratio in the production setting does not affect the accuracy much. We only consider rates up to 50- 50 since having 50% or more inductive nodes is exceedingly rare in practical scenarios. In cases where a substantial influ…

**Figure 6.** Figure 6: Accuracy vs. Feature Noise Ratio, Hidden Dimension, and λ on ArXiv. 6 Conclusion In this paper, we propose MGFNN and MGFNN+ to combine both MGNNs’ superior performance and MLPs’ efficient inference. MGFNN directly trains student MLPs with node features as input and so…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 29 canonical work pages

[12]

Teaching MLPs to Master Heterogeneous Graph-Structured Knowledge for Efficient and Accurate Inference

Liu, Y., Gao, X., He, T., Zhao, J., Yin, H.: Teaching mlps to master heterogeneous graph-structured knowledge for efficient and accurate inference. arXiv preprint arXiv:2411.14035 (2024)

work page Pith review arXiv 2024
[1]

In: ICLR (2021)

Chen, L., Chen, Z., Bruna, J.: On graph neural networks versus graph-augmented mlps. In: ICLR (2021)

work page 2021
[2]

In: CIKM (2020)

Dou, Y., Liu, Z., Sun, L., Deng, Y., Peng, H., Yu, P.S.: Enhancing graph neural network-based fraud detectors against camouflaged fraudsters. In: CIKM (2020)

work page 2020
[3]

NeurIPS (2020)

Du, S., You, S., Li, X., Wu, J., Wang, F., Qian, C., Zhang, C.: Agree to disagree: Adaptive ensemble knowledge distillation in gradient space. NeurIPS (2020)

work page 2020
[4]

In: ICLR (2024)

Feng, Y., Luo, Y., Ying, S., Gao, Y.: LightHGNN: Distilling hypergraph neural networks into MLPs for 100x faster inference. In: ICLR (2024)

work page 2024
[5]

In: ICML

Guo, Z., Shiao, W., Zhang, S., Liu, Y., Chawla, N.V., Shah, N., Zhao, T.: Linkless link prediction via relational distillation. In: ICML. pp. 12012–12033 (2023)

work page 2023
[6]

NeurIPS (2017)

Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. NeurIPS (2017)

2017
[7]

arXiv preprint (2015)

Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint (2015)

work page 2015

Show all 31 references

[8]

NeurIPS (2020)

Hu, W., Fey, M., Zitnik, M., Dong, Y., Ren, H., Liu, B., Catasta, M., Leskovec, J.: Open graph benchmark: Datasets for machine learning on graphs. NeurIPS (2020)

2020
[9]

TKDE 35(1), 521–532 (2021)

Ji, H., Wang, X., Shi, C., Wang, B., Philip, S.Y.: Heterogeneous graph propagation network. TKDE 35(1), 521–532 (2021)

2021
[10]

In: ICLR (2017) 16 Y

Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: ICLR (2017) 16 Y. Liu et al

2017
[11]

Applied Network Science5, 1–24 (2020)

Lerique, S., Abitbol, J.L., Karsai, M.: Joint embedding of structure and features via graph convolutional networks. Applied Network Science5, 1–24 (2020)

2020
[13]

In: KDD (2024)

Lu, W., Guan, Z., Zhao, W., Yang, Y.: Adagmlp: Adaboosting gnn-to-mlp knowl- edge distillation. In: KDD (2024)

2024
[14]

In: ICML

Mo, Y., Lei, Y., Shen, J., Shi, X., Shen, H.T., Zhu, X.: Disentangled multiplex graph representation learning. In: ICML. pp. 24983–25005 (2023)

2023
[15]

In: The Semantic Web

Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: The Semantic Web. pp. 593–607. Springer (2018)

2018
[16]

In: ICDM

Tan, Q., Zha, D., Liu, N., Choi, S.H., Li, L., Chen, R., Hu, X.: Double wins: Boosting accuracy and efficiency of graph neural networks by reliable knowledge distillation. In: ICDM. pp. 1343–1348. IEEE (2023)

2023
[17]

In: ICLR (2023)

Tian, Y., Zhang, C., Guo, Z., Zhang, X., Chawla, N.: Learning MLPs on graphs: A unified view of effectiveness, robustness, and efficiency. In: ICLR (2023)

2023
[18]

In: ICLR (2018)

Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., Bengio, Y.: Graph attention networks. In: ICLR (2018)

2018
[19]

Wang, X., Ji, H., Shi, C., Wang, B., Ye, Y., Cui, P., Yu, P.S.: Heterogeneous graph attention network. In: WWW. pp. 2022–2032 (2019)

2019
[20]

In: ICML (2019)

Wu, F., Souza, A., Zhang, T., Fifty, C., Yu, T., Weinberger, K.: Simplifying graph convolutional networks. In: ICML (2019)

2019
[21]

In: AAAI

Wu, L., Lin, H., Huang, Y., Fan, T., Li, S.Z.: Extracting low-/high-frequency knowledge from graph neural networks and injecting it into mlps: An effective gnn-to-mlp distillation framework. In: AAAI. pp. 10351–10360 (2023)

2023
[22]

In: ICML

Wu, L., Lin, H., Huang, Y., Li, S.Z.: Quantifying the knowledge in gnns for reliable distillation into mlps. In: ICML. pp. 37571–37581. PMLR (2023)

2023
[23]

In: DASFAA

Yang, B.W., Chang, M.Y., Lu, C.H., Shen, C.Y.: Two heads are better than one: Teaching mlps with multiple graph neural networks via knowledge distillation. In: DASFAA. pp. 452–462. Springer (2024)

2024
[24]

In: ICLR (2024)

Yang, L., Tian, Y., Xu, M., Liu, Z., Hong, S., Qu, W., Zhang, W., CUI, B., Zhang, M., Leskovec, J.: VQGraph: Rethinking graph representation space for bridging GNNs and MLPs. In: ICLR (2024)

2024
[25]

In: WWW (2024)

Yao, T., Sun, J., Cao, D., Zhang, K., Chen, G.: Mugsi: Distilling gnns with multi- granularity structural information for graph classification. In: WWW (2024)

2024
[26]

arXiv preprint (2020)

Yu, L., Shen, J., Li, J., Lerer, A.: Scalable graph neural networks for heterogeneous graphs. arXiv preprint (2020)

2020
[27]

Yu, P., Fu, C., Yu, Y., Huang, C., Zhao, Z., Dong, J.: Multiplex heterogeneous graph convolutional network. In: KDD. pp. 2377–2387 (2022)

2022
[28]

In: AAAI

Zhang, C., Liu, J., Dang, K., Zhang, W.: Multi-scale distillation from multiple graph neural networks. In: AAAI. pp. 4337–4344 (2022)

2022
[29]

In: ICLR (2022)

Zhang, S., Liu, Y., Sun, Y., Shah, N.: Graph-less neural networks: Teaching old MLPs new tricks via distillation. In: ICLR (2022)

2022
[30]

Scientific Reports13(1), 19171 (2023)

Zhang, S., Liu, Y., Xie, L.: A universal framework for accurate and efficient geo- metric deep learning of molecular systems. Scientific Reports13(1), 19171 (2023)

2023
[31]

In: CIKM

Zhang, W., Mao, J., Cao, Y., Xu, C.: Multiplex graph neural networks for multi- behavior recommendation. In: CIKM. pp. 2313–2316 (2020)

2020

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [12]

Teaching MLPs to Master Heterogeneous Graph-Structured Knowledge for Efficient and Accurate Inference

Liu, Y., Gao, X., He, T., Zhao, J., Yin, H.: Teaching mlps to master heterogeneous graph-structured knowledge for efficient and accurate inference. arXiv preprint arXiv:2411.14035 (2024)

work page Pith review arXiv 2024

[2] [1]

In: ICLR (2021)

Chen, L., Chen, Z., Bruna, J.: On graph neural networks versus graph-augmented mlps. In: ICLR (2021)

work page 2021

[3] [2]

In: CIKM (2020)

Dou, Y., Liu, Z., Sun, L., Deng, Y., Peng, H., Yu, P.S.: Enhancing graph neural network-based fraud detectors against camouflaged fraudsters. In: CIKM (2020)

work page 2020

[4] [3]

NeurIPS (2020)

Du, S., You, S., Li, X., Wu, J., Wang, F., Qian, C., Zhang, C.: Agree to disagree: Adaptive ensemble knowledge distillation in gradient space. NeurIPS (2020)

work page 2020

[5] [4]

In: ICLR (2024)

Feng, Y., Luo, Y., Ying, S., Gao, Y.: LightHGNN: Distilling hypergraph neural networks into MLPs for 100x faster inference. In: ICLR (2024)

work page 2024

[6] [5]

In: ICML

Guo, Z., Shiao, W., Zhang, S., Liu, Y., Chawla, N.V., Shah, N., Zhao, T.: Linkless link prediction via relational distillation. In: ICML. pp. 12012–12033 (2023)

work page 2023

[7] [6]

NeurIPS (2017)

Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. NeurIPS (2017)

2017

[8] [7]

arXiv preprint (2015)

Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint (2015)

work page 2015