Pith. sign in

REVIEW 3 major objections 4 minor 2 cited by

Transformers are Graph Neural Networks

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper establishes an exact equivalence between multi-head attention and message passing, identifying Transformers as GNNs on fully connected token graphs.

desk verdict A readable tutorial on a known equivalence, but the 'formal equivalence' overreaches: the equations miss the output projection and use a non-pairwise message, so it's an analogy, not an identity. read the letter →

arxiv 2506.22084 v1 pith:A67FZDI7 submitted 2025-06-27 cs.LG cs.AI

Chaitanya K. Joshi This is my paper
classification cs.LGcs.AI
keywords TransformerGraphNeuralNetworksmessagepassingmulti-headattentionpositionalencodingshardwarelotteryfullyconnectedgraphs
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that a Transformer layer is a message-passing Graph Neural Network whose input is a fully connected graph with one node per token. In this view, multi-head self-attention is exactly the message construction and aggregation step of a graph attention network, with the neighbor set taken to be all tokens, and the token-wise MLP, residual, and normalization playing the role of the update function. The point matters because it unifies two seemingly separate architecture families: graph structure becomes a soft hint supplied by positional encodings rather than a hard constraint, and the practical dominance of Transformers over GNNs is reframed as a hardware advantage, not a fundamental algorithmic gap. If the paper is right, results and design ideas from one family transfer directly to the other.

What carries the argument

The load-bearing identity is that the Transformer's global attention score $w_{ij}$ equals a GAT's local attention score when the neighbor set is the whole token set: both are $\mathrm{softmax}_j(W_Q h_i \cdot W_K h_j)$. The message function $\psi$ is the value projection weighted by this score, aggregation is summation, and the update $\phi$ is the token-wise MLP with residual and LayerNorm. This per-head message passing, followed by concatenation and output projection for multi-head attention, yields exactly the update equations of Section 1, making the fully connected token graph the central object.

What would settle it

Find a Transformer variant used in practice whose aggregation over tokens is not a sum of per-pair messages — for example, an attention layer that uses max-pooling over values, or a message function that depends on the full set of tokens beyond the softmax denominator — and show it cannot be written in the message passing form of equations (11)-(13). Such a counterexample would falsify the claim that all Transformers are GNNs under the paper's mapping.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that the multi-head attention update of a Transformer can be instantiated directly in the message passing framework: for each head, the message from token $j$ to token $i$ is the value projection $W_V^{\ell,k} h_j^\ell$ weighted by the softmax attention score $w^k_{ij}$, and token $i$ updates by summing these messages over all $j \in S$ and applying the token-wise feed-forward network as $\phi$. Because $S$ is the set of all tokens, the Transformer is a GNN operating on a complete graph, and a GAT is exactly a Transformer whose attention is masked to the local neighborhood $N_i$. Positional encodings are then soft hints about ordering or graph structure rather than enforced inductive biases. The paper concludes that Transformers are expressive set-processing networks that currently win the hardware lottery because their dense matrix implementation runs far faster on modern accelerators than sparse message passing.

Load-bearing premise

The paper's central claim rests on treating the Transformer's token-wise MLP, LayerNorm, and residual connection as the update function of a message passing layer, without proving that these components are exactly captured by the standard message passing update; if they cannot be, the word 'exactly' in the equivalence fails.

Editorial extensions

If this is right

  • GATs are Transformers with sparse or masked attention: any graph structure can be encoded as a mask on the complete token graph.
  • Graph Transformers that interleave local message passing and global attention inherit both the expressivity of global attention and the bias of graph structure, overcoming known limits of sparse message passing.
  • Since Transformers make no a priori graph assumption, they can be applied to arbitrary sets and learn relational structure from data; positional encodings are used to softly inject order or geometry without hard constraints.
  • The practical gap between Transformers and GNNs is mostly a hardware implementation gap: dense matrix multiplication on GPUs and TPUs far outperforms sparse gather and scatter, so Transformers are GNNs currently winning the hardware lottery.
  • At sufficient scale, Transformers may learn inductive biases such as locality from data rather than requiring them to be encoded in the architecture.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A consequence the paper leaves implicit: if the equivalence is literal, expressivity and failure-mode results for message passing GNNs, such as over-squashing or limited graph-isomorphism discrimination, should apply to Transformers with the complete graph in place of a sparse one, making sequence length the analogue of graph diameter.
  • The hardware-lottery framing suggests a testable prediction: if future accelerators make sparse message passing as fast as dense attention, the practical advantage of Transformers over GNNs should shrink, and the field's choice of architecture should shift accordingly.
  • The paper's equivalence is per-layer; extending it to training dynamics would require showing that gradients through attention match gradients through graph attention, which is not addressed. One could test whether deep Transformers exhibit GNN-like over-smoothing as the complete graph gets large.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper argues that Transformer self-attention can be viewed as message passing in Graph Attention Networks (GATs) over a fully connected graph of tokens, with positional encodings as soft structural hints. It first presents a simplified account of single-head and multi-head attention (Eqs. 1-9), then formalizes GNN message passing (Eqs. 11-16), and in Section 3 claims a formal equivalence, stating that Eqs. 17-19 instantiate multi-head attention in the message-passing framework. Section 4 discusses why Transformers, despite this equivalence, are faster in practice because dense attention matrices exploit modern hardware better than sparse message passing.

Significance. The manuscript is a clearly written expository note rather than an original research contribution. If the formal equivalence were correct as stated, it would provide a clean unifying perspective on Transformers and GNNs, and the hardware-lottery framing is a useful observation. The paper correctly reproduces the standard simplified equations for attention and message passing, and it cites relevant prior work. However, the central formal claim is stronger than what Eqs. 17-19 actually establish, and the paper does not supply the missing identifications (multi-head output projection, set-dependent message construction) that would make the equivalence exact. The pedagogical value is real, but the paper needs revision before it can support the word 'formal'.

major comments (3)
  1. [Section 3, Eqs. (17)-(19) vs Eq. (8)] The claimed instantiation of multi-head attention omits the output projection O^l. Eq. (8) defines tilde_h_i^l = Concat(head_i^1,...,head_i^K) O^l, whereas Eq. (19) updates with MLP(LayerNorm(h_i^l + sum_{j in S} psi(...))), where each psi is the single-head message of Eq. (18). For K>1, summing one head's messages never yields the concatenated K-head vector that O^l consumes; for K=1, the update is still not Eq. (9) unless O^l is the identity. This is a load-bearing gap because the text states 'exactly the same set of update equations'. The gap is repairable by defining psi to return the concatenated per-head weighted values and applying O^l after aggregation or inside phi, but the paper does not provide that mapping.
  2. [Section 3, Eqs. (17)-(18) vs Eq. (11)] The message function psi is not of the pairwise form required by Eq. (11). In Eq. (18), the softmax denominator sums over all j' in S, so psi(h_i, h_j) depends on every token in the sentence, not only on h_i and h_j. The same issue already appears in the GAT equation (15) with the neighborhood N_i, which means the paper's general message-passing definition (11) is inconsistent with the attention message construction it later uses. If the equivalence is meant to be formal, the message-passing framework must allow set-dependent message construction or a normalized aggregator; the paper should state this explicitly.
  3. [Section 3, final paragraph] The statement 'We have arrived at exactly the same set of update equations as in Section 1' is too strong. The Transformer in Section 1 includes multi-head concatenation with output projection and a token-wise MLP with residual and LayerNorm; the message-passing update in Eq. (13) is generic, and Eq. (19) only matches Eq. (9) after choosing phi and after solving the issues in the two comments above. The informal analogy is valuable, but the wording should distinguish 'can be viewed as' from 'is formally identical'.
minor comments (4)
  1. [Throughout] Typos: 'apriori' in the abstract should be 'a priori'; 'propogate' in Section 4 should be 'propagate'; 'allows allows' appears in Section 2; 'This lead to' in Section 1 should be 'This led to'.
  2. [Eq. (12)] The aggregation operator in Eq. (12) is typeset as 'L' but is not defined; it should be a big operator such as ⨁ over the neighbor set.
  3. [Section 2, Eq. (14)] The notation for LocalAttention in Eq. (14) passes a set of key/value vectors, which is inconsistent with the pairwise signature of psi in Eq. (11); the authors should either generalize Eq. (11) or write the message function with the full neighbor set as an explicit argument.
  4. [References] The reference [Joshi, 2020] contains a typo in the URL ('gaph-neural-networks' instead of 'graph-neural-networks'), and the footnote on the first page should be integrated with the reference entry.

Circularity Check

1 steps flagged · score 2.0 of 10

No significant circularity: the Transformer/GNN identity is a direct equation match with independent content; the flagged step is the Section 3 instantiation, which is definitional in flavor and omits Eq. 8's Concat/O^l mapping, but no input is fitted and no load-bearing self-citation carries the argument.

  1. self definitional [Section 3, Eqs. 17–19 and the closing sentence 'We have arrived at exactly the same set of update equations as in Section 1.']
    "The multi-head attention in equation 8 can be directly instantiated in the message passing framework as follows (for each head): ... h ℓ+1 i = φ (h ℓ i , m ℓ i ) = MLP(LayerNorm(h ℓ i + Σ j∈S ψ(h ℓ i , h ℓ j ))). (19) ... We have arrived at exactly the same set of update equations as in Section 1."

    Eq. 19 is declared equal to Eq. 9 after defining ψ as the single-head attention message (Eq. 18) and φ as MLP∘LayerNorm with residual, so the equality holds by construction rather than by a derived mapping. The construction drops Eq. 8's multi-head output: Eq. 19 sums only one per-head ψ, with no Concat(head^1,...,head^K) and no O^l, so the identity holds only for the K=1, O^l=I case the construction selects; Eq. 8 is not instantiated as written. Also, ψ in Eq. 18 depends on all of S through the softmax denominator, so it is not the pairwise ψ: R^{2d}→R^d of Eq. 11; the equivalence silently widens the message-passing definition. The conclusion 'Transformers are GNNs' is thus partly built into the chosen ψ/φ and the omitted multi-head mapping.

full rationale

The derivation chain is self-contained: Transformer equations (Section 1, Eqs. 1–9) come from Vaswani et al. (2017), the message-passing framework and GAT (Section 2, Eqs. 11–16) come from Veličković et al. (2018), and Section 3 substitutes the fully-connected token set S for the GAT neighborhood N_i, yielding attention weights of the same softmax form. For a single head with identity output projection this is a genuine, verifiable equality between two externally defined architectures, so the central claim has independent content and involves no fitted parameters renamed as predictions. The one flagged step is definitional in flavor: Eq. 19 is asserted, not derived, to equal Eq. 9 because ψ and φ were chosen to be the Transformer's own operations, while Eq. 8's Concat and output projection O^l are never mapped into the message-passing layer, so 'exactly the same set of update equations' overstates the demonstrated result for the multi-head case; this is an omitted-mapping/rigor gap, repairable by returning concatenated per-head weighted values from ψ and applying O^l in φ, and it is a correctness concern rather than a circular reduction. Separately, Eq. 18's ψ depends on the whole set S, departing from the pairwise message function of Eq. 11, so the paper silently adopts the set-dependent notion of message passing it already used for GAT in Eq. 14. The only self-citation, the footnote identifying this as a technical version of the author's The Gradient article [Joshi, 2020], is not load-bearing: neither the architecture definitions nor the equivalence depend on it, and no uniqueness theorem or unverified prior result is imported from the author's own work. Overall the paper sits in the 0–2 no-significant-circularity band; I assign 2 for the definitional instantiation caveat and the minor self-citation.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The paper introduces no free parameters and no new entities. It relies on two domain assumptions: the exactness of the Transformer update representation and the extension of GAT message passing to complete graphs. These are standard in the literature.

assumptions (2)
  • domain assumption The Transformer layer is exactly represented by equations 1-9 including the token-wise MLP and LayerNorm.
    The paper's equivalence claims that the Transformer update (equation 9) is the same as the GNN update (equation 19), which requires the MLP, LayerNorm, and residual connection to be compatible with the message passing formulation. This is plausible but not proven in the paper.
  • domain assumption The GAT message passing update in equations 14-16 can be applied to a complete graph without changing the aggregation semantics.
    The paper substitutes the neighbor set Ni with the full token set S, which is at the heart of the claimed equivalence.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Transformers are Graph Neural Networks." pith.science (2026). https://pith.science/paper/A67FZDI7

@misc{pith2026250622084,
  author       = {Pith},
  title        = {Pith review of: Transformers are Graph Neural Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A67FZDI7}},
  note         = {Machine review of arXiv:2506.22084}
}
read the original abstract

We establish connections between the Transformer architecture, originally introduced for natural language processing, and Graph Neural Networks (GNNs) for representation learning on graphs. We show how Transformers can be viewed as message passing GNNs operating on fully connected graphs of tokens, where the self-attention mechanism capture the relative importance of all tokens w.r.t. each-other, and positional encodings provide hints about sequential ordering or structure. Thus, Transformers are expressive set processing networks that learn relationships among input elements without being constrained by apriori graphs. Despite this mathematical connection to GNNs, Transformers are implemented via dense matrix operations that are significantly more efficient on modern hardware than sparse message passing. This leads to the perspective that Transformers are GNNs currently winning the hardware lottery.

Figures

Figures reproduced from arXiv: 2506.22084 by the authors.

Figure 1
Figure 1. Representation Learning for NLP. RNNs build representations one token at a time, which captures the sequential nature of language. Transformers build representations in parallel via attention mechanisms, which capture relative importance of words w.r.t. each-other. focus on different parts of the input sentence when building representations at each step, rather than just the representation of the last word processed… view at source ↗
Figure 2
Figure 2. A simple attention mechanism. Taking as input the representations of the token h ℓ i and the set of other tokens in the sentence {h ℓ j ∀j ∈ S}, we compute the attention weights wij denoting the relative importance for each pair (i, j) through the dot-product followed by a softmax normalization. Finally, we produce the updated token representation h ℓ+1 i by summing over the representations of tokens {h ℓ j } weight… view at source ↗
Figure 3
Figure 3. A Transformer layer. A multi-head attention sub-layer computes the relative importance of each token in a sentence w.r.t. each other token, and updates their representations accordingly. The updated representations are then processed by a token-wise multi-layer perceptron (MLP) sub-layer. Modern variants of the Transformer use SwiGLU [Shazeer, 2020] instead of ReLU [Glorot et al., 2011] as the MLP’s activation funct… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Representation learning on graphs with message passing. (left) Graphs model complex systems via a set of nodes connected by edges. (middle) GNNs build latent representations of graph data via message passing, where each node learns to aggregate representations from its…

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Dynamic Relational Priming Improves Transformer in Multivariate Time Series

    cs.LG 2025-09 conditional novelty 6.0 of 10

    Prime attention modulates attention keys and values per channel-pair and reports improved MTS forecasting accuracy across several benchmarks.

  2. Structured Spectral Graph Representation Learning for Multi-label Abnormality Analysis from 3D CT Scans

    cs.CV 2025-10 conditional novelty 4.0 of 10

    A graph-of-slice-triplets encoder with spectral convolution outperforms 3D CNN/Transformer baselines on multi-label chest CT abnormality classification and transfers to report generation and abdominal CT.

Reference graph

Works this paper leans on

39 extracted references · 28 canonical work pages · cited by 2 Pith papers

  1. [1]

    Achiam, S

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    J. L. Ba, J. R. Kiros, and G. E. Hinton. Layer normalization. arXiv preprint arXiv:1607.06450, 2016

    arXiv 2016

  3. [3]

    Bahdanau, K

    D. Bahdanau, K. Cho, and Y. Bengio. Neural machine translation by jointly learning to align and translate. In ICLR, 2015

    work page 2015

  4. [4]

    P. W. Battaglia, J. B. Hamrick, V. Bapst, A. Sanchez-Gonzalez, V. Zambaldi, et al. Relational inductive biases, deep learning, and graph networks. arXiv preprint, 2018

    work page 2018

  5. [5]

    Batzner, A

    S. Batzner, A. Musaelian, L. Sun, M. Geiger, J. P. Mailoa, M. Kornbluth, N. Molinari, T. E. Smidt, and B. Kozinsky. E (3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nature communications, 2022

    work page 2022

  6. [6]

    Bommasani, D

    R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, E. Brynjolfsson, et al. On the opportunities and risks of foundation models. ArXiv, 2021

    work page 2021

  7. [7]

    M. M. Bronstein, J. Bruna, T. Cohen, and P. Veli c kovi \'c . Geometric deep learning: Grids, groups, graphs, geodesics, and gauges. arXiv preprint, 2021

    work page 2021

  8. [8]

    Derrow-Pinion, J

    A. Derrow-Pinion, J. She, D. Wong, O. Lange, T. Hester, L. Perez, M. Nunkesser, S. Lee, X. Guo, B. Wiltshire, et al. Eta prediction with graph neural networks in google maps. In Proceedings of the 30th ACM international conference on information & knowledge management, 2021

    work page 2021

Show all 39 references
  1. [9]

    Di Giovanni, L

    F. Di Giovanni, L. Giusti, F. Barbero, G. Luise, P. Lio, and M. M. Bronstein. On over-squashing in message passing neural networks: The impact of width, depth, and topology. In International Conference on Machine Learning. PMLR, 2023

    2023

  2. [10]

    J. Dong, B. Feng, D. Guessous, Y. Liang, and H. He. Flex attention: A programming model for generating optimized attention kernels. arXiv preprint arXiv:2412.05496, 2024

    2024 arXiv

  3. [11]

    Dosovitskiy, L

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, ICLR , 2021

    2021

  4. [12]

    V. P. Dwivedi and X. Bresson. A generalization of transformer networks to graphs. arXiv preprint arXiv:2012.09699, 2020

    2012 arXiv

  5. [13]

    Fey and J

    M. Fey and J. E. Lenssen. Fast graph representation learning with PyTorch Geometric . In ICLR Workshop on Representation Learning on Graphs and Manifolds, 2019

    2019

  6. [14]

    Gilmer, S

    J. Gilmer, S. S. Schoenholz, P. F. Riley, O. Vinyals, and G. E. Dahl. Neural message passing for quantum chemistry. In ICML, 2017

    2017

  7. [15]

    Glorot, A

    X. Glorot, A. Bordes, and Y. Bengio. Deep sparse rectifier neural networks. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, 2011

    2011

  8. [16]

    A. Graves. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850, 2013

    2013 arXiv

  9. [17]

    K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770--778, 2016

    2016

  10. [18]

    Hochreiter and J

    S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 1997

    1997

  11. [19]

    S. Hooker. The hardware lottery. Communications of the ACM, 2021

    2021

  12. [20]

    Jaegle, S

    A. Jaegle, S. Borgeaud, J.-B. Alayrac, C. Doersch, C. Ionescu, D. Ding, S. Koppula, D. Zoran, A. Brock, E. Shelhamer, et al. Perceiver io: A general architecture for structured inputs & outputs. arXiv preprint arXiv:2107.14795, 2021

    2021 arXiv

  13. [21]

    C. Joshi. Transformers are graph neural networks. The Gradient, 2020. URL https://thegradient.pub/transformers-are-gaph-neural-networks/

    2020

  14. [22]

    Jumper, R

    J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov, O. Ronneberger, et al. Highly accurate protein structure prediction with alphafold. Nature, 2021

    2021

  15. [23]

    T. N. Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. In ICLR, 2017

    2017

  16. [24]

    Mikolov, I

    T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems, 2013

    2013

  17. [25]

    C. Olah. Deep learning, NLP , and representations. Christopher Olah's Blog, 2014. URL https://colah.github.io/posts/2014-07-NLP-RNNs-Representations/

    2014

  18. [26]

    Radford, J

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. Language models are unsupervised multitask learners. OpenAI, 2019

    2019

  19. [27]

    Radford, J

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. Mcleavey, and I. Sutskever. Robust speech recognition via large-scale weak supervision. In International Conference on Machine Learning, 2023

    2023

  20. [28]

    Raffel, N

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 2020

    2020

  21. [29]

    Ramp \'a s ek, M

    L. Ramp \'a s ek, M. Galkin, V. P. Dwivedi, A. T. Luu, G. Wolf, and D. Beaini. Recipe for a general, powerful, scalable graph transformer. Advances in Neural Information Processing Systems, 2022

    2022

  22. [30]

    N. Shazeer. Glu variants improve transformer. arXiv preprint arXiv:2002.05202, 2020

    2002 arXiv

  23. [31]

    J. M. Stokes, K. Yang, K. Swanson, W. Jin, A. Cubillos-Ruiz, N. M. Donghia, C. R. MacNair, S. French, L. A. Carfrae, Z. Bloom-Ackermann, et al. A deep learning approach to antibiotic discovery. Cell, 2020

    2020

  24. [32]

    Sutskever, O

    I. Sutskever, O. Vinyals, and Q. V. Le. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, 2014

    2014

  25. [33]

    R. Sutton. The bitter lesson. Incomplete Ideas (blog), 2019

    2019

  26. [34]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. In Advances in neural information processing systems, 2017

    2017

  27. [35]

    Veli c kovi \' c , G

    P. Veli c kovi \' c , G. Cucurull, A. Casanova, A. Romero, P. Li \` o , and Y. Bengio. Graph Attention Networks . ICLR, 2018

    2018

  28. [36]

    L. Weng. Attention? attention! Lil'Log, 2018. URL http://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html

    2018

  29. [37]

    Xiong, Y

    R. Xiong, Y. Yang, D. He, K. Zheng, S. Zheng, C. Xing, H. Zhang, Y. Lan, L. Wang, and T. Liu. On layer normalization in the transformer architecture. In International conference on machine learning, 2020

    2020

  30. [38]

    K. Xu, W. Hu, J. Leskovec, and S. Jegelka. How powerful are graph neural networks? In ICLR, 2019

    2019

  31. [39]

    R. Ying, R. He, K. Chen, P. Eksombatchai, W. L. Hamilton, and J. Leskovec. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, 2018

    2018

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.