Pith. sign in

REVIEW 3 major objections 7 minor 2 cited by

Teaching MLPs to Master Heterogeneous Graph-Structured Knowledge for Efficient and Accurate Inference

T0 review · 3 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Distilled feature-only MLPs can match or beat heterogeneous GNN teachers on node classification while running inference up to 379× faster, the authors argue.

desk verdict Useful empirical extension of GNN-to-MLP distillation to heterogeneous graphs, with real speedups, but the headline MLP comparison is inflated by transductive soft-label access and the novelty claim ignores existing heterogeneous distillation baselines. read the letter →

arxiv 2411.14035 v2 pith:N5BAFQSD submitted 2024-11-21 cs.LG

Yunhui Liu , Xinyi Gao , Tieke He , Jianhua Zhao , Hongzhi Yin This is my paper · ORCID
classification cs.LG
keywords heterogeneousgraphneuralnetworksknowledgedistillationMLPstudentmeta-pathinferenceaccelerationnodeclassificationreliablepseudo-labelingtransductivelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that the accuracy of heterogeneous graph neural networks (HGNNs) can be transplanted into plain MLPs that never look at the graph at inference time. The authors propose two students, HG2M and HG2M+, trained on the soft labels of a teacher HGNN; the plus version also distills reliable meta-path-based neighbor knowledge so that heterogeneous semantics survive without structure. On six datasets, the distilled MLPs match or beat the teacher HGNN on most benchmarks, improve on vanilla MLPs by about 6–8% on average, and infer 39.81× to 379.24× faster. The payoff, if correct, is that latency-constrained deployments can use heavy relational models only offline, during teacher training, and serve predictions with lightweight feature-only MLPs.

What carries the argument

The load-bearing machinery is teacher-student knowledge distillation instantiated on heterogeneous graphs. A teacher HGNN (the heterogeneous GraphSAGE variant RSAGE by default) produces soft labels $z_v$ for every target-type node; the student MLP minimizes $\mathcal{L} = \lambda \mathcal{L}_{CE} + (1-\lambda)\mathcal{L}_{KL}$ in HG2M, and in HG2M+ a combined loss of reliable node distillation and reliable meta-path distillation (Eq. 6), where only high-confidence, low-entropy nodes $R$ and reliable intra-class meta-path neighbor pairs $M_P$ contribute. The meta-path distillation term uses a logistic-regression scorer on attribute similarity, connection strength, and soft-label similarity to estimate whether a neighbor shares the anchor's class, so structural semantics are injected without requiring graph access at inference. The efficiency claim follows because the deployed model is an MLP: no neighbor fetching, no relation-specific aggregation, only a forward pass on the node's own features.

What would settle it

Re-run the transductive experiments on one small dataset (e.g., TMDB) with soft labels generated only from training-time nodes, so validation and test nodes are unseen by the teacher, and check whether HG2M and HG2M+ still outperform vanilla MLPs and stay within about 1% of the teacher. If the margin over MLPs shrinks or disappears, the reported gains depend on the asymmetric access; if it holds, the distillation claim is robust.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that an MLP trained solely on node features can reproduce the node-classification behavior of a heterogeneous GNN teacher when the supervision consists of the teacher's soft labels extended with reliable and intra-class meta-path-based neighbors. HG2M uses only the standard distillation objective, a weighted combination of cross-entropy against true labels and KL divergence against teacher logits. HG2M+ adds reliable node distillation, which discards low-confidence, high-entropy teacher outputs, and reliable meta-path distillation, which trains the student to match the teacher's prediction for an anchor node's selected meta-path neighbors. Across TMDB, CroVal, ArXiv, IGB-549K-19, IGB-549K-2K, and IGB-3M-19, HG2M+ ranks first on 5 of 6 transductive benchmarks and in the production (transductive-plus-inductive) setting beats the teacher by 0.34% on average, while inference on five nodes takes 0.23–0.34 ms versus 10.75–125.15 ms for the teacher.

Load-bearing premise

The load-bearing premise is that giving the student MLP teacher soft labels for validation and test nodes, computed from the whole graph, while giving vanilla MLP and HGNN baselines no such extra supervision still counts as a fair accuracy comparison.

Editorial extensions

If this is right

  • If the central claim holds, latency-sensitive systems can serve node classifiers from MLPs while retaining HGNN-level accuracy, since graph-dependent computation is confined to offline teacher training.
  • The large-scale result (379× speedup and 28× lower inference memory on IGB-3M-19) implies that structure-free inference can be practical for large heterogeneous graphs under tight latency budgets.
  • Because HG2M+ improves over HG2M, the paper implies that explicitly distilling relational semantics through reliable meta-path neighbors closes most of the residual gap left by logit-only distillation.
  • The insensitivity of performance to the trade-off weight $\lambda$ (setting it to 0 works best) suggests that filtered teacher soft labels alone carry enough supervision for the student, without direct label loss.
  • The noisy-feature experiments suggest that distilled MLPs inherit some of the teacher's structure-awareness even when input features degrade, as long as reliable nodes and meta-path pairs are used.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the transductive protocol uses the full graph to generate teacher soft labels for validation and test nodes, while vanilla MLP and HGNN baselines receive no such extra supervision, the reported margin over MLPs may partly reflect asymmetric access rather than pure distillation.
  • Editorial inference: the same recipe could plausibly be carried to link prediction or graph classification by choosing task-appropriate distillation targets, though the meta-path machinery would need adaptation beyond node classification.
  • Editorial inference: the reliable meta-path distillation relies on a logistic-regression classifier trained on labeled and reliable unlabeled pairs, so a natural stress test is whether the gains persist on graphs with weak homophily or semantically meaningless meta-paths, a limitation the paper itself flags.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This paper proposes HG2M and HG2M+, two knowledge-distillation methods for compressing heterogeneous graph neural networks (HGNNs) into feature-only MLP students. HG2M trains an MLP on node features using soft labels from a pretrained HGNN teacher, while HG2M+ additionally filters teacher predictions by confidence/uncertainty (reliable node distillation) and injects meta-path-based relational supervision through a learned intra-class neighbor selector (reliable meta-path distillation). The authors evaluate on six heterogeneous graph datasets under transductive, inductive, and combined production protocols, reporting accuracy competitive with or better than the RSAGE teacher and inference speedups of 39.81x to 379.24x over the teacher. The paper also includes ablations, alternative teacher architectures, feature-noise robustness, and hyperparameter sensitivity studies.

Significance. If the reported accuracy held under a fair comparison, the paper would provide a practical recipe for latency-sensitive deployment of heterogeneous graph models, and the breadth of experiments (six datasets, including IGB-3M-19; multiple teacher architectures; ablations; noise robustness) is a genuine strength. The public code release is also commendable. However, the central accuracy claim is currently supported mainly by a transductive protocol in which the student receives teacher soft labels for the evaluation nodes, while the vanilla MLP and HGNN baselines do not. The inductive rows of Table IV, which are the cleanest control, show much smaller gains over MLPs and clear degradation below the teacher. The efficiency result is robust by construction, since the student is an MLP; the accuracy contribution needs re-scoping and re-evaluation before the paper can be accepted.

major comments (3)
  1. [Section V-A5, Eq. (2), Section V-A3] The transductive evaluation is asymmetric. The paper states in Section V-A5 that in the tran setting the entire graph, including validation and test nodes, is used to generate soft labels, and Eq. (2) is trained with lambda = 0, so the student is trained exclusively on teacher predictions for every target node, including the nodes on which it is later evaluated. The vanilla MLP and RSAGE baselines receive no such supervision, so Table III and the tran/prod rows of Table IV do not provide a fair comparison for the headline claim of significantly outperforming vanilla MLPs. Additionally, model selection is performed on validation accuracy (Section V-A3), but validation nodes are themselves used as training targets in this protocol, which makes the reported numbers optimistic. Please rerun the transductive experiments with soft labels generated only for labeled and observed nodes (as in the ind protocol), or add baselines that receive exactly the same teacher soft-label supervision, and condition all abstract and RQ1 claims on the resulting protocol.
  2. [Section V-B, Table IV] The inductive rows are the natural control and contradict the generality of the claim. In Table IV, on the ind rows HG2M+ improves over MLP by only 1.40 to 4.81 points, whereas the tran rows show gains of 6.10 to 15.98 points; and on the ind rows HG2M+ is below RSAGE by 11.60 points on ArXiv, 7.85 points on TMDB, and 2.30 points on CroVal. Since the inductive setting is the deployment-relevant one (teacher soft labels for new nodes are not available at inference), the statement in the abstract and Section V-B that HG2Ms achieve competitive or even better performance than HGNNs should be restricted to the transductive case or substantially tempered.
  3. [Section V-A3, Tables III and IV] No significance tests are reported. With five seeds and overlapping standard deviations (e.g., Table III, IGB-549K-19: HG2M+ 59.62 +/- 0.49 vs. RSAGE 58.53 +/- 0.55; Table VIII, IGB-549K-19: w/RND 59.24 +/- 0.47 vs. HG2M+ 59.62 +/- 0.49), the word 'significant' is not statistically supported. Paired significance tests across the five seeds would strengthen the main comparison.
minor comments (7)
  1. [Section IV-B2] The intra-class classifier in RMPD uses teacher soft-label similarity f3_uv = z_u dot z_v as a feature to decide which teacher soft labels z_u will be distilled; this is a mild circularity because the same teacher outputs are used both to select and to supervise the selected pairs. Please discuss this limitation and, ideally, ablate the f3 feature.
  2. [Section IV-C] The information-theoretic analysis is informal: Eq. (7) is a mutual-information decomposition, but no estimates of I(E;y) or I(X;y|E) are provided, so the paragraph should be framed as intuition rather than a proof.
  3. [Section I and Section V-C] The contribution bullet claiming to be the first to integrate HGNNs with MLPs through knowledge distillation is difficult to reconcile with the inclusion of HIRE [45] as a heterogeneous knowledge distillation baseline; please clarify the specific distinction or soften the novelty claim.
  4. [Table II] The hyperparameter table is ambiguous: the # layers and hidden dim rows appear to have an extra column relative to the listed model names. Please reformat so each value is clearly aligned with its model.
  5. [Table VI] Inference times are reported on only five randomly chosen nodes without variance across node choices; please add standard deviations or describe the sampling procedure more carefully.
  6. [Table V] The row labeled 'HG2M/GLNN' is unclear because GLNN is a homogeneous-graph method; please clarify whether this entry is HG2M with a GLNN-style objective or a separate baseline.
  7. [Limitations and Future Work] The limitations paragraph discusses RMPD feature generalization and task scope, but does not mention the transductive soft-label access issue raised above; adding this as a limitation would improve transparency.

Circularity Check

1 steps flagged · score 6.0 of 10

Transductive/prod accuracy is fitted to teacher soft labels on the evaluation nodes, making the headline gains over MLPs partially circular; the inductive control shows the non-circular effect is much smaller.

  1. fitted input called prediction [Section V-A5 (Transductive vs. Inductive) and Eq. (2)]
    "During distillation, the entire graph including the validation and test nodes is used to generate soft labels zv for every target-type node v ∈ Vt."

    Eq. (2) minimizes (1-λ)/|Vt| Σ_{v∈Vt} LKL(ŷ_v, z_v), and the paper sets λ=0, so the student is trained by directly matching teacher soft labels on all target-type nodes. Since Vt includes the validation and test nodes, the student's predictions on the evaluation nodes are fitted to the teacher's outputs for exactly those nodes. The vanilla MLP baseline receives no such soft-label supervision and is trained only on Y^L, making the headline 'significantly outperform vanilla MLPs' an asymmetric comparison. The transductive column—and the 80%-weighted prod score—therefore largely measures teacher-output imitation on the eval nodes rather than independent generalization.

full rationale

The paper's core technical contribution—training an MLP on teacher soft labels plus meta-path-based auxiliary objectives—is a legitimate empirical method with no equation-level circularity in the loss itself. There is no load-bearing self-citation: references [37] and [44] are dataset and hyperparameter precedents, not uniqueness theorems or unverified foundational claims. The information-theoretic explanation in Section IV-C is heuristic rather than a derivation, and while it is loose about what Eq. (2) actually transfers, it does not reduce a prediction to an input. The one significant circular element is the evaluation protocol: the transductive setting supplies teacher soft labels for the exact validation/test nodes and then trains the student (with λ=0) to match those soft labels, so the reported accuracy on those nodes is a fit to the teacher's outputs rather than an independent prediction. The paper does include a genuinely non-circular inductive control, and there HG2M+ still beats MLP by modest margins, so the circularity is partial rather than total. However, the abstract and RQ1 conclusions state the unconditional accuracy claim without conditioning on the asymmetric soft-label access, making the central headline comparison partially circular. Score 6 reflects that one central 'prediction' (transductive/prod accuracy) reduces by construction, while the method retains independent inductive content.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No new physical or conceptual entities are introduced. The method relies on standard ML assumptions about label noise, pseudo-labeling, and feature-label correlation, plus a protocol assumption about transductive access to test nodes.

free parameters (2)
  • reliable node proportion p = 0.9
    Set consistently across all datasets based on empirical findings (Section V-I2, Figure 8); trades supervision amount against label precision.
  • trade-off weight lambda = 0
    Authors report slight tuning and choose 0 where only the distillation loss is active (Section V-A4, Figure 11).
assumptions (4)
  • domain assumption Node features and structural roles are often highly correlated, so feature-only MLPs can approximate HGNN performance.
    Invoked in Section IV-C after Eq. 7 to justify why distillation can close the gap between MLP and HGNN.
  • domain assumption High-confidence, low-entropy teacher predictions on unlabeled nodes are likely correct.
    Used in Reliable Node Distillation (Eq. 3) to define the reliable unlabeled set; standard pseudo-labeling assumption.
  • domain assumption Nodes with similar attributes, soft labels, and meta-path connection strength are likely in the same class.
    Used to construct the 3-dimensional feature vector in Reliable Meta-Path Distillation (Section IV-B2).
  • domain assumption Using test node features and teacher soft labels for student training is a legitimate transductive evaluation protocol.
    Section V-A5 states that soft labels are generated for all target nodes including validation and test nodes in the transductive setting.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Teaching MLPs to Master Heterogeneous Graph-Structured Knowledge for Efficient and Accurate Inference." pith.science (2026). https://pith.science/paper/N5BAFQSD

@misc{pith2026241114035,
  author       = {Pith},
  title        = {Pith review of: Teaching MLPs to Master Heterogeneous Graph-Structured Knowledge for Efficient and Accurate Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N5BAFQSD}},
  note         = {Machine review of arXiv:2411.14035}
}
abstract

Heterogeneous Graph Neural Networks (HGNNs) have achieved promising results in various heterogeneous graph learning tasks, owing to their superiority in capturing the intricate relationships and diverse relational semantics inherent in heterogeneous graph structures. However, the neighborhood-fetching latency incurred by structure dependency in HGNNs makes it challenging to deploy for latency-constrained applications that require fast inference. Inspired by recent GNN-to-MLP knowledge distillation frameworks, we introduce HG2M and HG2M+ to combine both HGNN's superior performance and MLP's efficient inference. HG2M directly trains student MLPs with node features as input and soft labels from teacher HGNNs as targets, and HG2M+ further distills reliable and heterogeneous semantic knowledge into student MLPs through reliable node distillation and reliable meta-path distillation. Experiments conducted on six heterogeneous graph datasets show that despite lacking structural dependencies, HG2Ms can still achieve competitive or even better performance than HGNNs and significantly outperform vanilla MLPs. Moreover, HG2Ms demonstrate a 379.24$\times$ speedup in inference over HGNNs on the large-scale IGB-3M-19 dataset, showcasing their ability for latency-sensitive deployments.

Figures

Figures reproduced from arXiv: 2411.14035 by the authors.

Figure 1
Figure 1. The number of nodes fetched and inference time for HGNNs exceed [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An example of a heterogeneous graph (TMDB). (a) TMDB consists of [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overview of our proposed HG2M+. In offline training, a well-trained teacher HGNN is first used to generate soft labels on the graph. Subsequently, [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Effectiveness of our reliable node selection in RND and intra-class [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Transductive Accuracy vs. Teacher HGNN Architectures. HG2Ms can learn from different HGNN teachers to improve over MLPs and achieve [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Transductive Accuracy vs. Node Feature Noise. HG2Ms have comparable performance to HGNNs. Adding more noise decreases HG2Ms’ performance [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Accuracy vs. Inductive:Transductive Ratio under the production setting. Altering the inductive split rate doesn’t affect the accuracy much. [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Impact of the proportion of selected reliable nodes on the performance of HG2M+. [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Impact of the proportion of selected reliable nodes on the accuracy of selected reliable nodes. [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Impact of model width on the prod performance. The teacher HGNN is RSAGE with 128-dim hidden layers. Therefore, we evaluate the robustness of HG2Ms with regards to different noise levels across TMDB, CroVal, and IGB￾549K-19 datasets. Specifically, we introduce differe…
Figure 10
Figure 10. Figure 10: On the ArXiv dataset, we observe that as the hidden [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Transductive Accuracy vs. Trade-off Weight [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Multi-Scale Heterogeneous Text-Attributed Graph Datasets From Diverse Domains

    cs.LG 2024-12 conditional novelty 6.0 of 10

    Six multi-scale, text-attributed heterogeneous graph benchmark datasets (TMDB, CroVal, ArXiv, Book, DBLP, Patent) with raw text, PLM features, and time-based splits for node classification.

  2. Learning Accurate, Efficient, and Interpretable MLPs on Multiplex Graphs via Node-wise Multi-View Ensemble Distillation

    cs.LG 2025-02 conditional novelty 4.0 of 10

    A node-wise multi-view ensemble distillation method lets plain MLPs match or beat multiplex GNN teachers on 5 of 6 datasets while running 35-89x faster.

Reference graph

Works this paper leans on

47 extracted references · 38 canonical work pages · cited by 2 Pith papers

  1. [1]

    Heterogeneous network representation learning: A unified framework with survey and benchmark,

    C. Yang, Y . Xiao, Y . Zhang, Y . Sun, and J. Han, “Heterogeneous network representation learning: A unified framework with survey and benchmark,” IEEE Transactions on Knowledge and Data Engineering , vol. 34, no. 10, pp. 4854–4873, 2020

    2020

  2. [2]

    Online user representation learning across heterogeneous social networks,

    W. Wang, H. Yin, X. Du, W. Hua, Y . Li, and Q. V . H. Nguyen, “Online user representation learning across heterogeneous social networks,” in International ACM SIGIR Conference on Research and Development in Information Retrieval, 2019, pp. 545–554

    work page 2019

  3. [3]

    Oag: Linking entities across large-scale heterogeneous knowledge graphs,

    F. Zhang, X. Liu, J. Tang, Y . Dong, P. Yao, J. Zhang, X. Gu, Y . Wang, E. Kharlamov, B. Shao et al., “Oag: Linking entities across large-scale heterogeneous knowledge graphs,” IEEE Transactions on Knowledge and Data Engineering , vol. 35, no. 9, pp. 9225–9239, 2022

    work page 2022

  4. [4]

    Heterogeneous informa- tion network embedding for recommendation,

    C. Shi, B. Hu, W. X. Zhao, and S. Y . Philip, “Heterogeneous informa- tion network embedding for recommendation,” IEEE Transactions on Knowledge and Data Engineering , vol. 31, no. 2, pp. 357–370, 2018

    work page 2018

  5. [5]

    Connecting embeddings based on multiplex relational graph attention networks for knowledge graph entity typing,

    Y . Zhao, H. Zhou, A. Zhang, R. Xie, Q. Li, and F. Zhuang, “Connecting embeddings based on multiplex relational graph attention networks for knowledge graph entity typing,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 5, pp. 4608–4620, 2022

    work page 2022

  6. [6]

    Single-cell biological network inference using a heterogeneous graph transformer,

    A. Ma, X. Wang, J. Li, C. Wang, T. Xiao, Y . Liu, H. Cheng, J. Wang, Y . Li, Y . Changet al. , “Single-cell biological network inference using a heterogeneous graph transformer,” Nature Communications , vol. 14, no. 1, p. 964, 2023

    work page 2023

  7. [7]

    Modeling relational data with graph convolutional networks,

    M. Schlichtkrull, T. N. Kipf, P. Bloem, R. van den Berg, I. Titov, and M. Welling, “Modeling relational data with graph convolutional networks,” in The Semantic Web. Springer, 2018, pp. 593–607

    work page 2018

  8. [8]

    Interpretable and efficient heterogeneous graph convolutional network,

    Y . Yang, Z. Guan, J. Li, W. Zhao, J. Cui, and Q. Wang, “Interpretable and efficient heterogeneous graph convolutional network,” IEEE Trans- actions on Knowledge and Data Engineering , vol. 35, no. 2, pp. 1637– 1650, 2021

    work page 2021

Show all 47 references
  1. [9]

    Are we really making much progress? revisiting, benchmarking and refining heterogeneous graph neural networks,

    Q. Lv, M. Ding, Q. Liu, Y . Chen, W. Feng, S. He, C. Zhou, J. Jiang, Y . Dong, and J. Tang, “Are we really making much progress? revisiting, benchmarking and refining heterogeneous graph neural networks,” in ACM SIGKDD Conference on Knowledge Discovery & Data Mining , 2021, pp...

    2021

  2. [10]

    Hgamlp: Heterogeneous graph attention mlp with de-redundancy mech- anism,

    Y . Liang, W. Zhang, Z. Sheng, L. Yang, J. Jiang, Y . Tong, and B. Cui, “Hgamlp: Heterogeneous graph attention mlp with de-redundancy mech- anism,” in International Conference on Data Engineering , 2024, pp. 2779–2791

    2024

  3. [11]

    Heterogeneous graph attention network,

    X. Wang, H. Ji, C. Shi, B. Wang, Y . Ye, P. Cui, and P. S. Yu, “Heterogeneous graph attention network,” in The Web Conference, 2019, pp. 2022–2032

    2019

  4. [12]

    Magnn: Metapath aggregated graph neural network for heterogeneous graph embedding,

    X. Fu, J. Zhang, Z. Meng, and I. King, “Magnn: Metapath aggregated graph neural network for heterogeneous graph embedding,” in The Web Conference, 2020, pp. 2331–2341

    2020

  5. [13]

    Heterogeneous graph propagation network,

    H. Ji, X. Wang, C. Shi, B. Wang, and S. Y . Philip, “Heterogeneous graph propagation network,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 1, pp. 521–532, 2021

    2021

  6. [14]

    Reliable node sim- ilarity matrix guided contrastive graph clustering,

    Y . Liu, X. Gao, T. He, T. Zheng, J. Zhao, and H. Yin, “Reliable node sim- ilarity matrix guided contrastive graph clustering,” IEEE Transactions on Knowledge and Data Engineering , vol. 36, no. 12, pp. 9123–9135, 2024

    2024

  7. [15]

    Paths2pair: Meta-path based link prediction in billion-scale commercial heterogeneous graphs,

    J. Hang, Z. Hong, X. Feng, G. Wang, G. Yang, F. Li, X. Song, and D. Zhang, “Paths2pair: Meta-path based link prediction in billion-scale commercial heterogeneous graphs,” in ACM SIGKDD Conference on Knowledge Discovery & Data Mining , 2024, pp. 5082–5092

    2024

  8. [16]

    Igb: Addressing the gaps in labeling, features, heterogeneity, and size of public graph datasets for deep learning research,

    A. Khatua, V . S. Mailthody, B. Taleka, T. Ma, X. Song, and W.-m. Hwu, “Igb: Addressing the gaps in labeling, features, heterogeneity, and size of public graph datasets for deep learning research,” in ACM SIGKDD Conference on Knowledge Discovery & Data Mining , 2023, pp. 4284–...

    2023

  9. [17]

    Graph-less neural networks: Teaching old MLPs new tricks via distillation,

    S. Zhang, Y . Liu, Y . Sun, and N. Shah, “Graph-less neural networks: Teaching old MLPs new tricks via distillation,” in International Confer- ence on Learning Representations , 2022

    2022

  10. [18]

    Learning MLPs on graphs: A unified view of effectiveness, robustness, and efficiency,

    Y . Tian, C. Zhang, Z. Guo, X. Zhang, and N. Chawla, “Learning MLPs on graphs: A unified view of effectiveness, robustness, and efficiency,” in International Conference on Learning Representations , 2023

    2023

  11. [19]

    Quantifying the knowledge in gnns for reliable distillation into mlps,

    L. Wu, H. Lin, Y . Huang, and S. Z. Li, “Quantifying the knowledge in gnns for reliable distillation into mlps,” in International Conference on Machine Learning. PMLR, 2023, pp. 37 571–37 581

    2023

  12. [20]

    VQGraph: Rethinking graph representation space for bridging GNNs and MLPs,

    L. Yang, Y . Tian, M. Xu, Z. Liu, S. Hong, W. Qu, W. Zhang, B. CUI, M. Zhang, and J. Leskovec, “VQGraph: Rethinking graph representation space for bridging GNNs and MLPs,” in International Conference on Learning Representations, 2024

    2024

  13. [21]

    Distilling the knowledge in a neural network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531 , 2015

    2015 arXiv

  14. [22]

    Semi-supervised classification with graph convolutional networks,

    T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in International Conference on Learning Rep- resentations, 2017

    2017

  15. [23]

    Graph attention networks,

    P. Veli ˇckovi´c, G. Cucurull, A. Casanova, A. Romero, P. Li `o, and Y . Bengio, “Graph attention networks,” in International Conference on Learning Representations, 2018

    2018

  16. [24]

    Double wins: Boosting accuracy and efficiency of graph neural networks by reliable knowledge distillation,

    Q. Tan, D. Zha, N. Liu, S.-H. Choi, L. Li, R. Chen, and X. Hu, “Double wins: Boosting accuracy and efficiency of graph neural networks by reliable knowledge distillation,” in ICDM, 2023, pp. 1343–1348

    2023

  17. [25]

    Extracting low-/high- frequency knowledge from graph neural networks and injecting it into mlps: An effective gnn-to-mlp distillation framework,

    L. Wu, H. Lin, Y . Huang, T. Fan, and S. Z. Li, “Extracting low-/high- frequency knowledge from graph neural networks and injecting it into mlps: An effective gnn-to-mlp distillation framework,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 37, no. 9,...

    2023

  18. [26]

    Linkless link prediction via relational distillation,

    Z. Guo, W. Shiao, S. Zhang, Y . Liu, N. V . Chawla, N. Shah, and T. Zhao, “Linkless link prediction via relational distillation,” in International Conference on Machine Learning . PMLR, 2023, pp. 12 012–12 033

    2023

  19. [27]

    Mugsi: Distilling gnns with multi-granularity structural information for graph classification,

    T. Yao, J. Sun, D. Cao, K. Zhang, and G. Chen, “Mugsi: Distilling gnns with multi-granularity structural information for graph classification,” in The Web Conference, 2024, pp. 709–720

    2024

  20. [28]

    LightHGNN: Distilling hy- pergraph neural networks into MLPs for 100x faster inference,

    Y . Feng, Y . Luo, S. Ying, and Y . Gao, “LightHGNN: Distilling hy- pergraph neural networks into MLPs for 100x faster inference,” in International Conference on Learning Representations , 2024

    2024

  21. [29]

    In defense of pseudo-labeling: An uncertainty-aware pseudo-label selection frame- work for semi-supervised learning,

    M. N. Rizve, K. Duarte, Y . S. Rawat, and M. Shah, “In defense of pseudo-labeling: An uncertainty-aware pseudo-label selection frame- work for semi-supervised learning,” in International Conference on Learning Representations, 2021

    2021

  22. [30]

    Re- liable data distillation on graph convolutional network,

    W. Zhang, X. Miao, Y . Shao, J. Jiang, L. Chen, O. Ruas, and B. Cui, “Re- liable data distillation on graph convolutional network,” in Proceedings of the 2020 ACM SIGMOD international conference on management of data, 2020, pp. 1399–1414

    2020

  23. [31]

    Deep insights into noisy pseudo labeling on graph data,

    B. Wang, J. Li, Y . Liu, J. Cheng, Y . Rong, W. Wang, and F. Tsung, “Deep insights into noisy pseudo labeling on graph data,” in Neural Information Processing Systems , 2023

    2023

  24. [32]

    Self-supervised heterogeneous graph neural network with co-contrastive learning,

    X. Wang, N. Liu, H. Han, and C. Shi, “Self-supervised heterogeneous graph neural network with co-contrastive learning,” in ACM SIGKDD Conference on Knowledge Discovery & Data Mining , 2021, pp. 1726– 1736

    2021

  25. [33]

    OGB- LSC: A large-scale challenge for machine learning on graphs,

    W. Hu, M. Fey, H. Ren, M. Nakata, Y . Dong, and J. Leskovec, “OGB- LSC: A large-scale challenge for machine learning on graphs,” in Neural Information Processing Systems , 2021

    2021

  26. [34]

    On graph neural networks versus graph-augmented mlps,

    L. Chen, Z. Chen, and J. Bruna, “On graph neural networks versus graph-augmented mlps,” in International Conference on Learning Rep- resentations, 2021

    2021

  27. [35]

    Rethinking softmax with cross- entropy: Neural network classifier as mutual information estimator,

    Z. Qin, D. Kim, and T. Gedeon, “Rethinking softmax with cross- entropy: Neural network classifier as mutual information estimator,” arXiv preprint arXiv:1911.10688 , 2019

    1911 arXiv

  28. [36]

    Joint embedding of struc- ture and features via graph convolutional networks,

    S. Lerique, J. L. Abitbol, and M. Karsai, “Joint embedding of struc- ture and features via graph convolutional networks,” Applied Network Science, vol. 5, pp. 1–24, 2020

    2020

  29. [37]

    Multi-scale heterogeneous text-attributed graph datasets from diverse domains,

    Y . Liu, Q. Xie, J. Shi, J. Shen, and T. He, “Multi-scale heterogeneous text-attributed graph datasets from diverse domains,” in Companion Proceedings of the ACM Web Conference 2025 , 2025

    2025

  30. [38]

    Minilm: Deep self-attention distillation for task-agnostic compression of pre- trained transformers,

    W. Wang, F. Wei, L. Dong, H. Bao, N. Yang, and M. Zhou, “Minilm: Deep self-attention distillation for task-agnostic compression of pre- trained transformers,” Neural Information Processing Systems , vol. 33, pp. 5776–5788, 2020

    2020

  31. [39]

    Sentence-BERT: Sentence embeddings using Siamese BERT-networks,

    N. Reimers and I. Gurevych, “Sentence-BERT: Sentence embeddings using Siamese BERT-networks,” in EMNLP-IJCNLP. Association for Computational Linguistics, 2019, pp. 3982–3992

    2019

  32. [40]

    Roberta: A robustly optimized bert pretraining approach,

    Y . Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V . Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” arXiv preprint arXiv:1907.11692 , 2019

    1907 arXiv

  33. [41]

    Inductive representation learn- ing on large graphs,

    W. Hamilton, Z. Ying, and J. Leskovec, “Inductive representation learn- ing on large graphs,” Neural Information Processing Systems , vol. 30, 2017

    2017

  34. [42]

    Relational graph attention networks,

    D. Busbridge, D. Sherburn, P. Cavallo, and N. Y . Hammerla, “Relational graph attention networks,” arXiv preprint arXiv:1904.05811 , 2019

    1904 arXiv

  35. [43]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International Conference on Learning Representations , 2015

    2015

  36. [44]

    Learning accurate, efficient, and interpretable mlps on multiplex graphs via node- wise multi-view ensemble distillation,

    Y . Liu, Z. Tao, X. Zhao, J. Zhao, T. Zheng, and T. He, “Learning accurate, efficient, and interpretable mlps on multiplex graphs via node- wise multi-view ensemble distillation,” in International Conference on Database Systems for Advanced Applications . Springer, 2025

    2025

  37. [45]

    Hire: Distilling high-order relational knowledge from heterogeneous graph neural networks,

    J. Liu, T. Zheng, and Q. Hao, “Hire: Distilling high-order relational knowledge from heterogeneous graph neural networks,” Neurocomput- ing, vol. 507, pp. 67–83, 2022

    2022

  38. [46]

    A teacher-free graph knowledge distillation framework with dual self-distillation,

    L. Wu, H. Lin, Z. Gao, G. Zhao, and S. Z. Li, “A teacher-free graph knowledge distillation framework with dual self-distillation,” IEEE Transactions on Knowledge and Data Engineering , 2024

    2024

  39. [47]

    Single teacher, multiple perspectives: Teacher knowledge augmentation for enhanced knowledge distillation,

    M. I. Hossain, S. Akhter, C. S. Hong, and E.-N. Huh, “Single teacher, multiple perspectives: Teacher knowledge augmentation for enhanced knowledge distillation,” in The Thirteenth International Conference on Learning Representations, 2025

    2025

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.