REVIEW 8 cited by
TCL: Transformer-based Dynamic Graph Modelling via Contrastive Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Dynamic graph modeling has recently attracted much attention due to its extensive applications in many real-world scenarios, such as recommendation systems, financial transactions, and social networks. Although many works have been proposed for dynamic graph modeling in recent years, effective and scalable models are yet to be developed. In this paper, we propose a novel graph neural network approach, called TCL, which deals with the dynamically-evolving graph in a continuous-time fashion and enables effective dynamic node representation learning that captures both the temporal and topology information. Technically, our model contains three novel aspects. First, we generalize the vanilla Transformer to temporal graph learning scenarios and design a graph-topology-aware transformer. Secondly, on top of the proposed graph transformer, we introduce a two-stream encoder that separately extracts representations from temporal neighborhoods associated with the two interaction nodes and then utilizes a co-attentional transformer to model inter-dependencies at a semantic level. Lastly, we are inspired by the recently developed contrastive learning and propose to optimize our model by maximizing mutual information (MI) between the predictive representations of two future interaction nodes. Benefiting from this, our dynamic representations can preserve high-level (or global) semantics about interactions and thus is robust to noisy interactions. To the best of our knowledge, this is the first attempt to apply contrastive learning to representation learning on dynamic graphs. We evaluate our model on four benchmark datasets for interaction prediction and experiment results demonstrate the superiority of our model.
Forward citations
Cited by 8 Pith papers
-
DyGnROLE: Asymmetric Pretraining for Edge Classification on Dynamic Graphs
Role-disentangled Transformer plus temporal contrastive pretraining improves future edge classification with 10k labels on 7 of 8 DTGB datasets.
-
What Do Temporal Graph Learning Models Learn?
Temporal graph models consistently learn to favor popular nodes but fail to learn edge direction, density, and recency.
-
When Speed meets Accuracy: an Efficient and Effective Graph Model for Temporal Link Prediction
EAGLE predicts temporal links with top-k recent neighbors plus top-k shared temporal PageRank influencers, matching or beating transformer T-GNNs while running far faster.
-
Learnable Spatial-Temporal Positional Encoding for Link Prediction
L-STEP learns time-evolving positional encodings for graph nodes via a learnable spectral filter and predicts links with MLPs only, matching or beating attention-based baselines on 13 temporal datasets.
-
Future Link Prediction Without Memory or Aggregation
CRAFT replaces memory and aggregation with learnable node embeddings and destination-to-source-neighbor cross-attention, improving future link prediction on most of 17 temporal graph benchmarks.
-
Learning Discriminative and Generalizable Anomaly Detector for Dynamic Graph with Limited Supervision
SDGAD combines residual event representations, a two-hypersphere restriction loss, and a normalizing-flow boundary to detect dynamic-graph anomalies with little or no supervision.
-
Are We Really Measuring Progress? Transferring Insights from Evaluating Recommender Systems to Temporal Link Prediction
Temporal link prediction benchmarks can rank models differently depending on sampling and aggregation choices, so current progress measurements are not reliable.
-
Higher-order Structure Boosts Link Prediction on Temporal Graphs
HTGN adds hyperedge memory and hypergraph convolution to temporal GNNs, claiming better dynamic link prediction and lower memory cost, but the reported results are undermined by data inconsistencies and invalid proofs.
Discussion (0). Continue with ORCID to comment.