Article Source

• Temporal Graph Networks for Deep Learning on Dynamic Graphs

Temporal Graph Networks for Deep Learning on Dynamic Graphs

• Source code: github: tgn

Abstract

Graph Neural Networks (GNNs) have recently become increasingly popular due to their ability to learn complex systems of relations or interactions arising in a broad spectrum of problems ranging from biology and particle physics to social networks and recommendation systems. Despite the plethora of different models for deep learning on graphs, few approaches have been proposed thus far for dealing with graphs that present some sort of dynamic nature (e.g. evolving features or connectivity over time). In this paper, the authors present Temporal Graph Networks (TGNs), a generic, efficient framework for deep learning on dynamic graphs represented as sequences of timed events. Thanks to a novel combination of memory modules and graph-based operators, TGNs are able to significantly outperform previous approaches being at the same time more computationally efficient. The authors furthermore show that several previous models for learning on dynamic graphs can be cast as specific instances of the TGN framework. They perform a detailed ablation study of different components of the TGN framework and devise the best configuration that achieves state-of-the-art performance on several transductive and inductive prediction tasks for dynamic graphs.

What was discussed?

• Understanding the need for dynamic graphs and current limitations using static GNN approaches
• TGN memory modules and time dependent embedding modules
• TGN advanced training strategy with 19x speed up over TGAT and consistent improvement

Takeaways

• Temporal Graph Networks are a generalized form of GNN for dynamic time graphs
• Memory modules are introduced for each node and are used to store memory of the data dynamics
• Graph embeddings can be computed based on the memory states and downstream temporal based tasks show increased performance when using memory than without
• Node memory can also be updated on test time
• The authors introduce a general framework to compute memory but every memory function (message, aggregation and update) can be modified according to the problem at hand