Pith. sign in

REVIEW 3 major objections 5 minor 102 references

OMEGA: A Low-Latency GNN Serving System for Large Graphs

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Pith's one-line read OMEGA serves GNNs on billion-edge graphs with up to 159× lower latency

desk verdict A solid, well-evaluated GNN serving system whose deployed recomputation heuristic is better than the theory used to justify it; referee with revisions. read the letter →

arxiv 2501.08547 v1 pith:U4IN3O5J submitted 2025-01-15 cs.DC cs.LG

classification cs.DCcs.LG
keywords graphneuralnetworksGNNservingprecomputedembeddingsselectiverecomputationcomputationparallelismlow-latencyinferencedistributedsystems
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

OMEGA is a system for serving graph neural networks on large, distributed graphs. Its central claim is that by reusing precomputed embeddings and selectively recomputing only the few that new query nodes affect most, a GNN service can avoid neighborhood explosion and cut latency by orders of magnitude while keeping accuracy loss under one percentage point. The paper also introduces computation graph parallelism, which spreads graph construction and local aggregation across machines so that communication no longer dominates serving time. If the claim holds, latency-critical GNN applications—recommendation, fraud detection, traffic prediction—can run on graphs with billions of edges without the slow full-neighborhood computation or the accuracy penalty of random sampling.

What carries the argument

The central mechanism is SRPE's recomputation policy, which ranks candidate nodes by the ratio $|N_Q(u)|/|N(u)|$ — the number of edges from query nodes to node $u$ divided by $u$'s total degree. This ratio proxies the theoretically optimal variance-minimizing weights $p_u \propto ||\sum_l q^{(l)}_u||$ while avoiding the impossible computation of full query embeddings. CGP then carries the execution: each machine aggregates messages from its local partition, exchanges partial aggregations through all-to-all, and applies a model-specific merge function (identity for sum, max, softmax with logits, or power-mean with pow operations) before the update function.

What would settle it

Run the reported Yelp GAT workload with a recomputation budget tuned to keep accuracy drop under 1% and then re-run it with the trained attention weights re-initialized or with edges rewired so that high-degree nodes have low query-edge ratios; if accuracy drops exceed one point under budget, the top-query-edges-ratio proxy is not carrying the accuracy claim.

Watch

Extended reading notes

Core claim

OMEGA's core discovery is that approximation errors in precomputed embeddings are highly skewed: a small fraction of nodes produces most of the error when a new query node connects to them. Recomputing just that top fraction restores accuracy almost fully. The system therefore proposes Selective Recomputation of Precomputed Embeddings (SRPE), with a top-query-edges-ratio policy that recomputes embeddings of nodes whose neighborhood has the highest ratio of query edges, and proves in Theorem 1 that recomputation probabilities proportional to $||\sum_{l=1}^{k-1} q^{(l)}_u||$ minimize estimator variance. To remove the remaining communication bottleneck, OMEGA adds Computation Graph Parallelism (CGP), where each machine builds and executes a local partition of the computation graph with local aggregation, then merges partial results with all-to-all collectives and custom merge functions for sum, max, power-mean, normalized-moment, and softmax-based aggregations. The evaluation reports up to 159× lower latency than full-computation-graph DGL serving and up to 10.8× lower latency than sampling-based DGL serving, with accuracy within 1 point of the full model.

Load-bearing premise

The load-bearing premise is that a recomputation candidate's error can be predicted by the ratio of query edges to its total degree, with aggregation treated as a mean of neighbor messages and GNN layers treated as statistically independent; for attention-based aggregators with correlated layers, this proxy has no proven guarantee and accuracy recovery depends on per-dataset budgets.

Editorial extensions

If this is right

  • Serving latency for GNNs on billion-edge graphs can drop from seconds to tens of milliseconds, making real-time inference feasible on datasets like the 10-billion-edge FB10B workload.
  • Accuracy stays within one percentage point of the full model when the recomputation budget is set per dataset, whereas neighborhood sampling can lose 2–6 points on attention-based and convolutional models.
  • Computation graph parallelism turns communication from a dominant bottleneck into a few megabytes of collective traffic, and it scales with GPUs: OMEGA's latency drops 67% from 2 to 8 GPUs while sampling-based serving barely improves.
  • Because PEs shrink computation graphs to direct neighbors, deeper GNN layers cost roughly linearly in latency rather than exponentially, as shown with GCNII up to six layers.
  • The system also serves models with attention and generalized arithmetic aggregation by translating their local aggregations into merge functions, so the approach is not limited to sum or mean aggregators.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The variance-minimization proof relies on treating GNN layers as statistically independent and aggregation as a mean; for attention models with learned, input-dependent weights and correlated layers, the top-query-edges-ratio policy has no formal error guarantee beyond the empirical budgets reported.
  • One testable extension is caching recomputed embeddings for frequently queried nodes across requests, which the paper does not explore but which could reduce recomputation costs further on skewed query workloads.
  • The system assumes query nodes attach only to existing training nodes; handling dynamic edge insertions or node deletions after deployment would require a staleness or invalidation mechanism that OMEGA explicitly leaves for future work.
  • The reported accuracy numbers are measured against a fixed training/test split with 25% of test nodes held out; a realistic deployment with drifting query distributions could require re-tuning the recomputation budget, and the paper does not provide an online adaptation rule.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents OMEGA, a distributed GNN serving system that combines two techniques: selective recomputation of precomputed embeddings (SRPE), which reuses layer embeddings of training nodes and recomputes a small fraction of error-prone embeddings, and computation graph parallelism (CGP), which partitions the construction and execution of computation graphs across machines with custom merge functions for different GNN aggregations. The system is implemented on top of DGL and evaluated on six graph datasets and three GNN models. The paper reports large latency reductions, up to 159× versus full-graph serving and up to 10.8× versus sampling-based serving, with accuracy drops kept below 1 percentage point. A theoretical contribution is claimed: Theorem 1 states that the recomputation probabilities minimize the variance of unbiased embedding estimators, and the paper argues that the deployed top-query-edges-ratio policy statistically minimizes approximation errors.

Significance. If the claims are scoped carefully, the paper is a solid systems contribution. The evaluation is extensive: six datasets, three models, multiple batch sizes, scaling studies, latency breakdowns, and a latency-throughput analysis. The system design is plausible and the reported latency benefits are large. The paper also provides an analytical latency model for CGP in Appendix D, which is a useful check on the empirical results. The main weakness is that the theoretical statement is used to support a guarantee that the deployed heuristic does not actually satisfy, and the accuracy-loss claims are partly established by tuning the recomputation budget on the same workload used for evaluation. These issues do not invalidate the empirical latency findings, but they require a careful reframing of the paper's accuracy and optimality claims.

major comments (3)
  1. [§5.2.1–5.2.2, Appendix A] Theorem 1 derives optimal recomputation probabilities p_u ∝ ||Σ_{l=1}^{k-1} Σ_{v∈N_Q(u)} m_v^(l)/|N(u)| ||, which depend on the actual query-node messages m_v^(l). The deployed top-query-edges-ratio policy replaces this quantity with |N_Q(u)|/|N(u)|, effectively dropping the message terms. As stated in §5.2.2, the exact calculation is infeasible because it requires full query embeddings. Consequently, the abstract's phrase 'statistically minimizes approximation errors' and the §5.2.2 reference to 'optimal probabilities' are not properties of the system as implemented. The paper should either prove a separate approximation guarantee for the degree-ratio proxy under explicit conditions (e.g., homogeneous message norms), or clearly label the proxy as a heuristic and remove the language that ties its accuracy to Theorem 1.
  2. [§5.2.2, Table 3, §8.2] The 'minimal accuracy loss' claim is weaker than it appears because for the majority of dataset-model combinations in Table 3 the recomputation budget is γ=0. For Reddit, Products, and Papers, all models use γ=0, meaning SRPE does not perform any recomputation and the reported accuracy drops are entirely due to plain PE reuse. For the configurations where γ>0 (e.g., Yelp GCN with γ=20% and Yelp GAT with γ=7%), the budget is explicitly selected so that the accuracy drop is below 1 percentage point on the same workload used to report the final accuracy. This makes the <1% drop a tuning target rather than an independent prediction. The paper should separate the measured accuracy of the policy at a fixed budget from the budget-selection procedure, and should report results at representative budgets without per-dataset tuning.
  3. [§5.2.2, §8.2, Table 4] For attention-based models such as GAT, the degree-ratio proxy has no demonstrated error guarantee. GAT's softmax attention weights are learned and can be highly nonuniform per edge, so a single high-attention query edge may dominate approximation error regardless of the ratio |N_Q(u)|/|N(u)|. The empirical comparison in Fig. 18 shows that the policy works on the tested GAT workloads, but this does not establish the general claim that it 'statistically minimizes approximation errors.' The paper should either include an analysis or experiments specifically probing attention-weight distributions (e.g., comparing the degree-ratio ranking against rankings based on actual attention-weighted messages), or explicitly restrict the theoretical claim to mean-aggregation models and present the GAT results as empirical evidence only.
minor comments (5)
  1. [§6.1, Eq. (3)] The symbol U is used both for the GNN update function in Eq. (1) and for the CGP merge function in Eq. (3); these are different operations and should use distinct symbols to avoid confusion.
  2. [§5.2.1] The notation q_u^(l) is introduced in the text but the estimator \f\u005e(l)_u is written with q_u^(l) and t_u^(l) without explicitly defining t_u^(l) in the main text; it appears only in Appendix A. A short definition in the main text would improve readability.
  3. [Figure 6 (right)] The legend lists '10' next to the OMEGA line, which appears to be a leftover artifact; the labeled curves are RANDOM, AE, IS, and OMEGA, so the stray '10' should be removed.
  4. [§7] The paper does not state whether the implementation or evaluation scripts will be released; a reproducibility or artifact availability statement would strengthen the systems contribution.
  5. [Throughout] The abstract in the submitted text does not include the quantitative speedup numbers (159×, 10.8×) that are emphasized in the paper's evaluation and in the reader's summary; adding these to the abstract would make the contribution more visible, though it is not required.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: SRPE's theorem is adapted from external prior work, the deployed heuristic is an explicitly approximate policy, and the γ budget is a transparently configured tradeoff parameter.

full rationale

The paper's derivation chain is self-contained rather than circular. Theorem 1 is explicitly adapted from Theorem 3.2 of GraphSAINT [90], an external result, and the proof in Appendix A states its simplifying assumption (independent GNN layers) rather than smuggling in the conclusion. The deployed top-query-edges-ratio policy is transparently a simplification: §5.2.2 says the exact optimal probabilities require full query-node messages, 'which is infeasible in serving', and therefore the policy approximates p_u ∝ |N_Q(u)|/|N(u)| 'without considering the message terms'. The accuracy claims are therefore supported by empirical comparisons against RANDOM, IS, and DGL baselines (§5.2.2, Fig. 6, Fig. 18, Table 4), not by equating the heuristic with the theorem. The only parameter that could look fitted, γ, is explicitly a user-controlled budget: Table 3's caption states that γ is 'the recomputation budget to achieve less than 1% points of accuracy drop', and §5.2.2 says users adjust it based on acceptable accuracy drop and latency. Selecting a threshold and then reporting the resulting accuracy is a configuration choice, not a fitted quantity renamed as a prediction, and the latency speedups (Figs. 10, 11) are measured end-to-end against DGL baselines independently of the accuracy target. The only coauthor self-citation is [34], used as related-work context and not load-bearing for any central claim. No circular step can be exhibited, so the appropriate finding is no significant circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The system's theoretical contribution rests on the GraphSAINT variance argument restated under mean aggregation and layer independence, and its empirical accuracy claims rest on per-dataset recomputation budgets chosen on the test workload. No new entities are introduced.

free parameters (1)
  • Recomputation budget gamma = Yelp GCN 20%, Amazon GCN 3%, Yelp GAT 7%, Amazon GAT 1%, 0% for others (Table 3)
    Chosen per dataset and model to make SRPE accuracy drop less than 1 point, then used in the main latency and accuracy comparison. The accuracy claim therefore depends on test-set tuning.
assumptions (4)
  • domain assumption Each GNN layer independently learns embeddings, enabling statistical analysis despite nonlinear activations.
    Invoked in Appendix A to prove Theorem 1. Not true for end-to-end trained GNNs and limits the theorem's scope.
  • domain assumption Aggregation is a mean of neighbor messages in the SRPE error model.
    In Section 5.2.1, q and t are defined with mean denominators, and the unbiased estimator relies on this. Does not cover attention-based GAT or power-mean aggregation, for which CGP's merge functions are separate engineering.
  • domain assumption Approximation error is dominated by direct neighbors of query nodes, so recomputation candidates are restricted to direct neighbors.
    In Section 5.2, candidates R are direct neighbors; errors in higher-hop embeddings are assumed recoverable through recomputed direct-neighbor outputs. This is not proven.
  • domain assumption Synthetic serving workload, created by removing 25% random test nodes and adding query edges, represents real serving traffic.
    In Section 8.1, no public large-scale GNN serving workload is used, so evaluation relies on this synthetic request model. Real workloads with dynamic updates or skew may differ.

how reviews work

0 comments
Cite this review

Pith. "Pith review of OMEGA: A Low-Latency GNN Serving System for Large Graphs." pith.science (2026). https://pith.science/paper/U4IN3O5J

@misc{pith2026250108547,
  author       = {Pith},
  title        = {Pith review of: OMEGA: A Low-Latency GNN Serving System for Large Graphs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U4IN3O5J}},
  note         = {Machine review of arXiv:2501.08547}
}
read the original abstract

Graph Neural Networks (GNNs) have been widely adopted for their ability to compute expressive node representations in graph datasets. However, serving GNNs on large graphs is challenging due to the high communication, computation, and memory overheads of constructing and executing computation graphs, which represent information flow across large neighborhoods. Existing approximation techniques in training can mitigate the overheads but, in serving, still lead to high latency and/or accuracy loss. To this end, we propose OMEGA, a system that enables low-latency GNN serving for large graphs with minimal accuracy loss through two key ideas. First, OMEGA employs selective recomputation of precomputed embeddings, which allows for reusing precomputed computation subgraphs while selectively recomputing a small fraction to minimize accuracy loss. Second, we develop computation graph parallelism, which reduces communication overhead by parallelizing the creation and execution of computation graphs across machines. Our evaluation with large graph datasets and GNN models shows that OMEGA significantly outperforms state-of-the-art techniques.

Figures

Figures reproduced from arXiv: 2501.08547 by the authors.

Figure 1
Figure 1. (Left) An example graph dataset with 8 nodes and F-dimensional feature vectors. We use this graph dataset as our running example. (Right) The 2-hop computation graph for node 0 where the boxes below represent feature vectors. • We show that OMEGA is able to outperform state-of-the￾art techniques and to achieve up to orders of magnitude lower latency with minimal accuracy loss (§8). 2 Background A Primer on GNNs. Unl… view at source ↗
Figure 3
Figure 3. (Left) Latency breakdown of [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 4
Figure 4. High-level illustration of (Left) Selective Recomputa￾tion of Precomputed Embeddings (SRPE) and (Right) Computa￾tion Graph Parallelism (CGP). The query node 8 is connected to the existing nodes 2 and 3 of the example graph dataset in [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (11 more)
Figure 6
Figure 6. Figure 6: (Left) CDF of the approximation errors of PEs, derived from the workload in [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 5
Figure 5. Figure 5: OMEGA end-to-end workflow. customized merge functions for GNN models; 5 the master server collects and returns the output embeddings. Problem Scope. This paper focuses on achieving low-latency, large-scale GNN serving for new query nodes that are un￾seen during trainin…
Figure 8
Figure 8. Figure 8: (a) An example serving request on the graph dataset [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: Example of distributed layer execution of CGP. At [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: End-to-end serving latency of OMEGA and baseline systems (in log scale) across three models and six datasets. The numbers within each bar represent the latency values for OMEGA or the relative speedup of OMEGA compared to each system. ‘OOM’ indicates that a system fai…
Figure 13
Figure 13. Figure 13: Normalized Latency of OMEGA and DGL (NS) vary￾ing number of GPUs with FB10B dataset. 0 2 4 6 8 10 12 14 Throughput (req/s) 0 10 0 10 1 10 2 10 3 10 4 DGL (NS) GPUS=2 OMEGA GPUS=2 DGL (NS) GPUS=4 OMEGA GPUS=4 DGL (NS) GPUS=8 OMEGA GPUS=8 Median Latency (s) [PITH_FULL_…
Figure 14
Figure 14. Figure 14: Latency-throughput results of OMEGA and DGL (NS) varying request rates with SAGE model and FB10B dataset. data type size: (L − 1) ∗ H ∗ D bytes. Although the mem￾ory footprint depends on the GNN model configuration and graph dataset size, PEs are typically smaller tha…
Figure 12
Figure 12. Figure 12: Latency and accuracy trade-off of OMEGA varying recomputation budget with GCN and GAT models and Amazon and Yelp datasets. most of a dataset, limiting local aggregation effectiveness. Nevertheless, CGP significantly enhances SRPE, where com￾putation graphs mainly cons…
Figure 15
Figure 15. Figure 15: Normalized latency of OMEGA and DGL (NS) varying feature cache size with SAGE model and FB10B dataset. Systems Yelp + R.H. Yelp + Metis Amazon + R.H. Amazon + Metis DGL (FULL) 652.6 ± 30.8 774.2 ± 37.4 8181.3 ± 215.4 10602.8 ± 1087.6 DGL (NS) 159.1 ± 3.2 167.0 ± 8.4 9…
Figure 17
Figure 17. Figure 17: Latency of OMEGA and DGL (NS) varying number of layers with GCNII model and Yelp dataset. test nodes for each model-dataset pair. We assess the perfor￾mance of recomputation policies using these batches and PEs and aggregate the results [PITH_FULL_IMAGE:figures/full_…
Figure 18
Figure 18. Figure 18: The recovered accuracy varies under different recomputation policies ( [PITH_FULL_IMAGE:figures/full_fig_p020_18.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

102 extracted references · 68 canonical work pages

  1. [1]

    https : / / www.nvidia.com/en- us/data- center/a100/

    NVIDIA A100 Tensor Core GPU. https : / / www.nvidia.com/en- us/data- center/a100/ . [Ac- cessed 06-16-2024]

  2. [2]

    https://www.nvidia.com/ en - us / networking / ethernet / connectx - 4 - lx/

    NVIDIA ConnectX-4 Lx. https://www.nvidia.com/ en - us / networking / ethernet / connectx - 4 - lx/. [Accessed 06-16-2024]

  3. [3]

    https://www.nvidia.com/en- us/networking/ethernet/connectx-5/

    NVIDIA ConnectX-5. https://www.nvidia.com/en- us/networking/ethernet/connectx-5/. [Accessed 06-16-2024]

  4. [4]

    https://www.nvidia.com/ en - us / networking / ethernet / connectx - 6 - dx/

    NVIDIA ConnectX-6 Dx. https://www.nvidia.com/ en - us / networking / ethernet / connectx - 6 - dx/. [Accessed 06-16-2024]

  5. [5]

    https : / / www.nvidia.com/en- us/data- center/h100/

    NVIDIA H100 Tensor Core GPU. https : / / www.nvidia.com/en- us/data- center/h100/ . [Ac- cessed 06-16-2024]

  6. [6]

    https://www.nvidia.com/en- us/data- center/tesla- p100/

    NVIDIA Tesla P100. https://www.nvidia.com/en- us/data- center/tesla- p100/ . [Accessed 06-16- 2024]

  7. [7]

    https : / / www.nvidia.com/en- us/data- center/v100/

    NVIDIA V100 Tensor Core GPU. https : / / www.nvidia.com/en- us/data- center/v100/ . [Ac- cessed 06-16-2024]

  8. [8]

    Build a GNN-based real-time fraud detection solu- tion using Amazon Sagemaker, Amazon Neptune, and the Deep Graph Library. https://aws .amazon.com/ blogs / machine - learning / build - a - gnn - based - real - time - fraud - detection - solution - using - amazon - sagemaker - amazon - neptune - and - the - deep-graph-library/, 2024. [Accessed 10-09-2024]

Show all 102 references
  1. [9]

    https: //github.com/facebookincubator/gloo, 2024

    Gloo: a collective communications library. https: //github.com/facebookincubator/gloo, 2024. [Ac- cessed 10-09-2024]

  2. [10]

    https://github.com/NVIDIA/nccl,

    NVIDIA NCCL. https://github.com/NVIDIA/nccl,

  3. [11]

    Abdine, M

    H. Abdine, M. Chatzianastasis, C. Bouyioukos, and M. Vazirgiannis. Prot2text: Multimodal protein’s func- tion generation with gnns and transformers. In Proceed- ings of the AAAI Conference on Artificial Intelligence, volume 38, pages 10757–10765, 2024

  4. [12]

    Achiam, S

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

  5. [13]

    Atkinson, A

    O. Atkinson, A. Bhardwaj, C. Englert, V . S. Ngairang- bam, and M. Spannowsky. Anomaly detection with convolutional graph neural networks. Journal of High Energy Physics, 2021(8):1–19, 2021

  6. [14]

    L. Bai, L. Yao, C. Li, X. Wang, and C. Wang. Adap- tive graph convolutional recurrent network for traffic forecasting. Advances in neural information processing systems, 33:17804–17815, 2020

  7. [15]

    Battaglia, J

    P. Battaglia, J. B. C. Hamrick, V . Bapst, A. Sanchez, V . Zambaldi, M. Malinowski, A. Tacchetti, D. Raposo, A. Santoro, R. Faulkner, C. Gulcehre, F. Song, A. Bal- lard, J. Gilmer, G. E. Dahl, A. Vaswani, K. Allen, C. Nash, V . J. Langston, C. Dyer, N. Heess, D. Wierstra, P. K...

  8. [16]

    J. S. Bridle. Probabilistic interpretation of feedforward classification network outputs, with relationships to sta- tistical pattern recognition. In Neurocomputing: Algo- rithms, architectures and applications, pages 227–236. Springer, 1990

  9. [17]

    Brody, U

    S. Brody, U. Alon, and E. Yahav. How atten- tive are graph attention networks? arXiv preprint arXiv:2105.14491, 2021

  10. [18]

    Brown, B

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Ka- plan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-V oss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. ...

  11. [19]

    J. Chen, T. Ma, and C. Xiao. FastGCN: Fast learn- ing with graph convolutional networks via importance sampling. In International Conference on Learning Representations, 2018

  12. [20]

    J. Chen, J. Zhu, and L. Song. Stochastic training of graph convolutional networks with variance reduction. In In- ternational Conference on Machine Learning , pages 941–949, 2018

  13. [21]

    M. Chen, Z. Wei, Z. Huang, B. Ding, and Y . Li. Sim- ple and deep graph convolutional networks. In Inter- national conference on machine learning, pages 1725–

  14. [22]

    Chiang, X

    W.-L. Chiang, X. Liu, S. Si, Y . Li, S. Bengio, and C.-J. Hsieh. Cluster-GCN: An efficient algorithm for train- ing deep and large graph convolutional networks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’19, pages ...

  15. [23]

    Chien, W.-C

    E. Chien, W.-C. Chang, C.-J. Hsieh, H.-F. Yu, J. Zhang, O. Milenkovic, and I. S. Dhillon. Node feature ex- traction by self-supervised multi-scale neighborhood prediction. In International Conference on Learning Representations (ICLR), 2022

  16. [24]

    Corso, L

    G. Corso, L. Cavalleri, D. Beaini, P. Liò, and P. Veliˇckovi´c. Principal neighbourhood aggregation for graph nets. Advances in Neural Information Processing Systems, 33:13260–13271, 2020

  17. [25]

    Damania, S

    P. Damania, S. Li, A. Desmaison, A. Azzolini, B. Vaughan, E. Yang, G. Chanan, G. J. Chen, H. Jia, H. Huang, J. Spisak, L. Wehrstedt, L. Hosseini, M. Kr- ishnan, O. Salpekar, P. Belevich, R. Varma, S. Gera, W. Liang, S. Xu, S. Chintala, C. He, A. Ziashahabi, S. Avestimehr, and ...

  18. [26]

    T. Dao, D. Fu, S. Ermon, A. Rudra, and C. Ré. FlashAt- tention: Fast and memory-efficient exact attention with io-awareness. In S. Koyejo, S. Mohamed, A. Agar- wal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 163...

  19. [27]

    S. Deng, H. Rangwala, and Y . Ning. Learning dynamic context graphs for predicting social events. In Proceed- ings of the 25th ACM SIGKDD International Confer- ence on Knowledge Discovery & Data Mining , KDD ’19, pages 1007–1016, New York, NY , USA, 2019. As- sociation for Com...

  20. [28]

    Derrow-Pinion, J

    A. Derrow-Pinion, J. She, D. Wong, O. Lange, T. Hes- ter, L. Perez, M. Nunkesser, S. Lee, X. Guo, B. Wilt- shire, et al. Eta prediction with graph neural networks in google maps. In Proceedings of the 30th ACM In- ternational Conference on Information & Knowledge Management, p...

  21. [29]

    Edunov, D

    S. Edunov, D. Logothetis, C. Wang, A. Ching, and M. Kabiljo. Generating synthetic social graphs with Darwini. In 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), pages 567– 577, 2018

  22. [30]

    W. Fan, Y . Ma, Q. Li, Y . He, E. Zhao, J. Tang, and D. Yin. Graph neural networks for social recommenda- tion. In The World Wide Web Conference, WWW ’19, page 417–426, New York, NY , USA, 2019. Association for Computing Machinery

  23. [31]

    Fey and J

    M. Fey and J. E. Lenssen. Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019

  24. [32]

    M. Fey, J. E. Lenssen, F. Weichert, and J. Leskovec. GN- NAutoScale: Scalable and expressive graph neural net- works via historical embeddings. In International con- ference on machine learning, pages 3294–3304. PMLR, 2021

  25. [33]

    A. Fout, J. Byrd, B. Shariat, and A. Ben-Hur. Protein interface prediction using graph convolutional networks. Advances in neural information processing systems, 30, 2017

  26. [34]

    Gandhi and A

    S. Gandhi and A. P. Iyer. P3: Distributed deep graph learning at scale. In 15th USENIX Symposium on Oper- ating Systems Design and Implementation (OSDI 21), pages 551–568, 2021

  27. [35]

    Geyer and S

    F. Geyer and S. Bondorf. DeepTMA: Predicting ef- fective contention models for network calculus using graph neural networks. In IEEE INFOCOM 2019 - IEEE Conference on Computer Communications, pages 1009–1017, 2019

  28. [36]

    Gilmer, S

    J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl. Neural message passing for quantum chem- istry. In International conference on machine learning, pages 1263–1272. PMLR, 2017

  29. [37]

    W. L. Hamilton, R. Ying, and J. Leskovec. Inductive representation learning on large graphs. In Proceedings of the 31st International Conference on Neural Infor- mation Processing Systems, NIPS’17, page 1025–1035, Red Hook, NY , USA, 2017. Curran Associates Inc. 14

  30. [38]

    W. L. Hamilton, R. Ying, and J. Leskovec. Represen- tation learning on graphs: Methods and applications. arXiv preprint arXiv:1709.05584, 2017

  31. [39]

    Hochreiter and J

    S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997

  32. [40]

    Hoffmann, S

    J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. d. L. Casas, L. A. Hendricks, J. Welbl, A. Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556 , 2022

  33. [41]

    H. Hu, F. Liu, Q. Pei, Y . Yuan, Z. Xu, and L. Wang. λGrapher: A resource-efficient serverless system for gnn serving through graph sharing. In The World Wide Web Conference, WWW ’24. Association for Computing Machinery, 2024

  34. [42]

    W. Hu, M. Fey, H. Ren, M. Nakata, Y . Dong, and J. Leskovec. OGB-LSC: A large-scale challenge for machine learning on graphs. In J. Vanschoren and S. Ye- ung, editors, Proceedings of the Neural Information Pro- cessing Systems Track on Datasets and Benchmarks 1, NeurIPS Datase...

  35. [43]

    W. Hu, M. Fey, M. Zitnik, Y . Dong, H. Ren, B. Liu, M. Catasta, and J. Leskovec. Open Graph Benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems , 33:22118– 22133, 2020

  36. [44]

    Huang, T

    W. Huang, T. Zhang, Y . Rong, and J. Huang. Adap- tive sampling towards fast graph representation learning. Advances in neural information processing systems, 31, 2018

  37. [45]

    Z. Jia, S. Lin, M. Gao, M. Zaharia, and A. Aiken. Im- proving the accuracy, scalability, and performance of graph neural networks with ROC. In I. Dhillon, D. Pa- pailiopoulos, and V . Sze, editors,Proceedings of Ma- chine Learning and Systems, volume 2, pages 187–198, 2020

  38. [46]

    Z. Jia, S. Lin, R. Ying, J. You, J. Leskovec, and A. Aiken. Redundancy-free computation for graph neural net- works. In Proceedings of the 26th ACM SIGKDD Inter- national Conference on Knowledge Discovery & Data Mining, KDD ’20, page 997–1005, New York, NY , USA,

  39. [47]

    Jiang and J

    W. Jiang and J. Luo. Graph neural network for traffic forecasting: A survey. Expert Systems with Applications, page 117921, 2022

  40. [48]

    M. Jin, Y . Liu, Y . Zheng, L. Chi, Y .-F. Li, and S. Pan. ANEMONE: graph anomaly detection with multi-scale contrastive learning. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 3122–3126, 2021

  41. [49]

    Kaler, N

    T. Kaler, N. Stathas, A. Ouyang, A.-S. Iliopoulos, T. Schardl, C. E. Leiserson, and J. Chen. Accelerat- ing training and inference of graph neural networks with fast sampling and pipelining. Proceedings of Machine Learning and Systems, 4:172–189, 2022

  42. [50]

    Karypis and V

    G. Karypis and V . Kumar. A fast and high quality multi- level scheme for partitioning irregular graphs. SIAM J. Sci. Comput., 20(1):359–392, Dec. 1998

  43. [51]

    D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014

  44. [52]

    T. N. Kipf and M. Welling. Semi-Supervised Classifica- tion with Graph Convolutional Networks. In Proceed- ings of the 5th International Conference on Learning Representations, ICLR ’17, 2017

  45. [53]

    G. Li, M. Muller, A. Thabet, and B. Ghanem. Deep- GCNs: Can GCNs go as deep as CNNs? In Proceedings of the IEEE/CVF international conference on computer vision, pages 9267–9276, 2019

  46. [54]

    G. Li, C. Xiong, A. Thabet, and B. Ghanem. Deep- erGCN: All you need to train deeper gcns. arXiv preprint arXiv:2006.07739, 2020

  47. [55]

    Q. Li, Z. Han, and X.-M. Wu. Deeper insights into graph convolutional networks for semi-supervised learn- ing. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018

  48. [56]

    D. Lin, S. Sun, J. Ding, X. Ke, H. Gu, X. Huang, C. Song, X. Zhang, L. Yi, J. Wen, et al. Platogl: Effective and scalable deep graph learning system for graph-enhanced real-time recommendation. In Proceedings of the 31st ACM International Conference on Information & Knowl- edg...

  49. [57]

    Z. Lin, C. Li, Y . Miao, Y . Liu, and Y . Xu. PaGraph: Scal- ing gnn training on large graphs via computation-aware caching. In Proceedings of the 11th ACM Symposium on Cloud Computing, SoCC ’20, pages 401–415, New York, NY , USA, 2020. Association for Computing Machinery

  50. [58]

    T. Liu, Y . Chen, D. Li, C. Wu, Y . Zhu, J. He, Y . Peng, H. Chen, H. Chen, and C. Guo. BGL: GPU-Efficient GNN training by optimizing graph data I/O and prepro- cessing. In 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23), pages 103–118, Boston, M...

  51. [59]

    Y .-C. Lo, S. E. Rensi, W. Torng, and R. B. Altman. Ma- chine learning in chemoinformatics and drug discovery. Drug Discovery Today, 23(8):1538 – 1546, 2018

  52. [60]

    M. Lu, Z. Han, S. X. Rao, Z. Zhang, Y . Zhao, Y . Shan, R. Raghunathan, C. Zhang, and J. Jiang. Bright-graph neural networks in real-time fraud detection. In Pro- ceedings of the 31st ACM International Conference on Information & Knowledge Management, pages 3342– 3351, 2022

  53. [61]

    L. Ma, Z. Yang, Y . Miao, J. Xue, M. Wu, L. Zhou, and Y . Dai. NeuGraph: Parallel deep neural network compu- tation on large graphs. In 2019 USENIX Annual Tech- nical Conference (USENIX ATC 19) , pages 443–458, Renton, W A, July 2019. USENIX Association

  54. [62]

    V . Md, S. Misra, G. Ma, R. Mohanty, E. Georganas, A. Heinecke, D. Kalamkar, N. K. Ahmed, and S. Avan- cha. DistGNN: Scalable distributed training for large- scale graph neural networks. In Proceedings of the International Conference for High Performance Com- puting, Networkin...

  55. [63]

    Ong and P

    E. Ong and P. Veli ˇckovi´c. Learnable commutative monoids for graph neural networks. In Learning on Graphs Conference, pages 43–1. PMLR, 2022

  56. [64]

    Paszke, S

    A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer. Automatic differentiation in pytorch. 2017

  57. [65]

    J. Peng, Z. Chen, Y . Shao, Y . Shen, L. Chen, and J. Cao. Sancus: staleness-aware communication-avoiding full- graph decentralized training in large-scale graph neu- ral networks. Proceedings of the VLDB Endowment , 15(9):1937–1950, 2022

  58. [66]

    J. Shi, V . Chaurasiya, Y . Liu, S. Vij, Y . Wu, S. Kanduri, N. Shah, P. Yu, N. Srivastava, L. Shi, G. Venkataraman, and J. Yu. Embedding based retrieval in friend recom- mendation. In H. Chen, W. E. Duh, H. Huang, M. P. Kato, J. Mothe, and B. Poblete, editors, Proceedings of ...

  59. [67]

    Y . Shi, A. Zhang, E. Zhang, Z. Liu, and X. Wang. ReLM: Leveraging language models for enhanced chemical re- action prediction. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023

  60. [68]

    Sinha, Z

    A. Sinha, Z. Shen, Y . Song, H. Ma, D. Eide, B.-J. P. Hsu, and K. Wang. An overview of Microsoft Academic Service (MAS) and applications. In Proceedings of the 24th International Conference on World Wide Web, WWW ’15 Companion, page 243–246, New York, NY , USA, 2015. Associati...

  61. [69]

    J. M. Stokes, K. Yang, K. Swanson, W. Jin, A. Cubillos- Ruiz, N. M. Donghia, C. R. MacNair, S. French, L. A. Carfrae, Z. Bloom-Ackermann, V . M. Tran, A. Chiappino-Pepe, A. H. Badran, I. W. Andrews, E. J. Chory, G. M. Church, E. D. Brown, T. S. Jaakkola, R. Barzilay, and J. J....

  62. [70]

    Suarez-Varela, P

    J. Suarez-Varela, P. Almasan, M. Ferriol-Galmes, K. Rusek, F. Geyer, X. Cheng, X. Shi, S. Xiao, F. Scarselli, A. Cabellos-Aparicio, and P. Barlet-Ros. Graph neural networks for communication networks: Context, use cases and opportunities. IEEE Network, pages 1–8, 2022

  63. [71]

    J. Sun, L. Su, Z. Shi, W. Shen, Z. Wang, L. Wang, J. Zhang, Y . Li, W. Yu, J. Zhou, and F. Wu. Legion: Automatically pushing the envelope of Multi-GPU sys- tem for Billion-Scale GNN training. In 2023 USENIX Annual Technical Conference (USENIX ATC 23), pages 165–179, Boston, MA...

  64. [72]

    Z. Tan, X. Yuan, C. He, M.-K. Sit, G. Li, X. Liu, B. Ai, K. Zeng, P. Pietzuch, and L. Mai. Quiver: Sup- porting GPUs for low-latency, high-throughput GNN serving with workload awareness. arXiv preprint arXiv:2305.10863, 2023

  65. [73]

    Thorpe, Y

    J. Thorpe, Y . Qiao, J. Eyolfson, S. Teng, G. Hu, Z. Jia, J. Wei, K. V ora, R. Netravali, M. Kim, and G. H. Xu. Dorylus: Affordable, scalable, and accurate GNN train- ing with distributed CPU servers and serverless threads. In 15th USENIX Symposium on Operating Systems De- sig...

  66. [74]

    Y . Tian, H. Song, Z. Wang, H. Wang, Z. Hu, F. Wang, N. V . Chawla, and P. Xu. Graph neural prompting with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19080–19088, 2024

  67. [75]

    Veliˇckovi´c, G

    P. Veliˇckovi´c, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y . Bengio. Graph attention networks. In International Conference on Learning Representations, 2018

  68. [76]

    Virinchi, A

    S. Virinchi, A. Saladi, and A. Mondal. Recommend- ing related products using graph neural networks in di- rected graphs. In Joint European Conference on Ma- chine Learning and Knowledge Discovery in Databases, pages 541–557. Springer, 2022. 16

  69. [77]

    C. Wan, Y . Li, A. Li, N. S. Kim, and Y . Lin. BNS-GCN: Efficient full-graph training of graph convolutional net- works with partition-parallelism and random boundary node sampling. Proceedings of Machine Learning and Systems, 4:673–693, 2022

  70. [78]

    M. Wang, D. Zheng, Z. Ye, Q. Gan, M. Li, X. Song, J. Zhou, C. Ma, L. Yu, Y . Gai, T. Xiao, T. He, G. Karypis, J. Li, and Z. Zhang. Deep Graph Library: A graph- centric, highly-performant package for graph neural net- works. arXiv preprint arXiv:1909.01315, 2019

  71. [79]

    Y . Wang, B. Feng, G. Li, S. Li, L. Deng, Y . Xie, and Y . Ding. GNNAdvisor: An adaptive and efficient run- time system for GNN acceleration on GPUs. In 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI 21), pages 515–531. USENIX Association, July 2021

  72. [80]

    Y . Wang, B. Feng, Z. Wang, T. Geng, K. Barker, A. Li, and Y . Ding. MGG: Accelerating graph neural net- works with Fine-Grained Intra-Kernel Communication- Computation pipelining on Multi-GPU platforms. In 17th USENIX Symposium on Operating Systems Design and Implementation (...

  73. [81]

    J. Wei, Y . Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Met- zler, E. H. Chi, T. Hashimoto, O. Vinyals, P. Liang, J. Dean, and W. Fedus. Emergent abilities of large language models. Transactions on Machine Learning Research, 2022

  74. [82]

    Wen and Y

    Z. Wen and Y . Fang. Augmenting low-resource text clas- sification with graph-grounded pre-training and prompt- ing. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in In- formation Retrieval, SIGIR ’23, page 506–516, New York, NY , US...

  75. [83]

    Y . Wu, D. Lian, Y . Xu, L. Wu, and E. Chen. Graph convo- lutional networks with markov random field reasoning for social spammer detection. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 1054–1061, 2020

  76. [84]

    Y . Wu, K. Ma, Z. Cai, T. Jin, B. Li, C. Zheng, J. Cheng, and F. Yu. Seastar: vertex-centric programming for graph neural networks. In Proceedings of the Sixteenth European Conference on Computer Systems, pages 359– 375, 2021

  77. [85]

    K. Xu, C. Li, Y . Tian, T. Sonobe, K.-i. Kawarabayashi, and S. Jegelka. Representation learning on graphs with jumping knowledge networks. In International confer- ence on machine learning , pages 5453–5462. PMLR, 2018

  78. [86]

    H. Yang. AliGraph: A comprehensive graph neural network platform. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Dis- covery & Data Mining , KDD ’19, pages 3165–3166, New York, NY , USA, 2019. Association for Computing Machinery

  79. [87]

    J. Yang, Z. Liu, S. Xiao, C. Li, D. Lian, S. Agrawal, A. Singh, G. Sun, and X. Xie. GraphFormers: GNN- nested transformers for representation learning on tex- tual graph. In M. Ranzato, A. Beygelzimer, Y . Dauphin, P. Liang, and J. W. Vaughan, editors,Advances in Neu- ral Info...

  80. [88]

    J. Yang, D. Tang, X. Song, L. Wang, Q. Yin, R. Chen, W. Yu, and J. Zhou. GNNLab: a factored system for sample-based gnn training over gpus. In Proceedings of the Seventeenth European Conference on Computer Systems, pages 417–434, 2022

  81. [89]

    R. Ying, R. He, K. Chen, P. Eksombatchai, W. L. Hamil- ton, and J. Leskovec. Graph convolutional neural net- works for web-scale recommender systems. In Proceed- ings of the 24th ACM SIGKDD International Confer- ence on Knowledge Discovery & Data Mining , KDD ’18, pages 974–98...

  82. [90]

    H. Zeng, H. Zhou, A. Srivastava, R. Kannan, and V . K. Prasanna. GraphSAINT: Graph sampling based induc- tive learning method. In8th International Conference on Learning Representations, ICLR, Addis Ababa, Ethiopia, April 26-30, 2020

  83. [91]

    L. Zeng, P. Huang, K. Luo, X. Zhang, Z. Zhou, and X. Chen. Fograph: Enabling real-time deep graph infer- ence with fog computing. In Proceedings of the ACM Web Conference 2022, pages 1774–1784, 2022

  84. [92]

    Zhang, X

    D. Zhang, X. Huang, Z. Liu, J. Zhou, Z. Hu, X. Song, Z. Ge, L. Wang, Z. Zhang, and Y . Qi. AGL: A scalable system for industrial-purpose graph machine learning. Proc. VLDB Endow., 13(12):3125–3137, Aug. 2020

  85. [93]

    Zhang, Y

    S. Zhang, Y . Liu, Y . Sun, and N. Shah. Graph-less neural networks: Teaching old mlps new tricks via distillation. In The Tenth International Conference on Learning Rep- resentations, ICLR, Virtual Event, April 25-29, 2022

  86. [94]

    Zhang, A

    X. Zhang, A. Bosselut, M. Yasunaga, H. Ren, P. Liang, C. D. Manning, and J. Leskovec. GreaseLM: Graph REASoning enhanced language models. In Interna- tional Conference on Learning Representations, 2021. 17

  87. [95]

    Zheng, H

    C. Zheng, H. Chen, Y . Cheng, Z. Song, Y . Wu, C. Li, J. Cheng, H. Yang, and S. Zhang. ByteGNN: efficient graph neural network training at large scale. Proceed- ings of the VLDB Endowment, 15(6):1228–1242, 2022

  88. [96]

    Zheng, C

    D. Zheng, C. Ma, M. Wang, J. Zhou, Q. Su, X. Song, Q. Gan, Z. Zhang, and G. Karypis. DistDGL: Dis- tributed graph neural network training for billion-scale graphs. In 2020 IEEE/ACM 10th Workshop on Irregu- lar Applications: Architectures and Algorithms (IA3) , pages 36–44, Los...

  89. [97]

    Zheng, X

    D. Zheng, X. Song, C. Yang, D. LaSalle, and G. Karypis. Distributed hybrid cpu and gpu training for graph neural networks on billion-scale heterogeneous graphs. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages 4582– 4591, 2022

  90. [98]

    G. Zhou, X. Zhu, C. Song, Y . Fan, H. Zhu, X. Ma, Y . Yan, J. Jin, H. Li, and K. Gai. Deep interest network for click-through rate prediction. In Y . Guo and F. Farooq, editors, Proceedings of the 24th ACM SIGKDD Inter- national Conference on Knowledge Discovery & Data Mining,...

  91. [99]

    J. Zhou, G. Cui, S. Hu, Z. Zhang, C. Yang, Z. Liu, L. Wang, C. Li, and M. Sun. Graph neural networks: A review of methods and applications. AI Open, 1:57–81, 2020. A Proof of Theorem 1 We describe the formal proof of Theorem 1. To prove the theorem, we assume each GNN layer in...

  92. [102]

    B Evaluation on Recomputation Policies In this section, we evaluate the performance of OMEGA ’s recomputation policy against RANDOM and IS policies (§5.2.2)

    ≥ ( ∑ u∈R || k−1 ∑ l=1 q(l) u ||)2 (6) Since γ = ∑u∈R √pu2 and the right-hand side are constants, the first term is minimized when the following equality con- dition holds: ∀u ∈ R, || k−1 ∑ l=1 q(l) u || 1√pu ∝ √pu (7) Therefore, we conclude S is minimized if pu ∝ || ∑k−1 l=1 ...

  93. [2020]

    Association for Computing Machinery

  94. [2024]

    [Accessed 10-09-2024]

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.