Modeling Co-Evolution of Attributed and Structural Information in Graph Sequence

Most graph neural network models learn embeddings of nodes in static attributed graphs for predictive analysis. Recent attempts have been made to learn temporal proximity of the nodes. We find that real dynamic attributed graphs exhibit complex phenomenon of co-evolution between node attributes and graph structure. Learning node embeddings for forecasting change of node attributes and evolution of graph structure over time remains an open problem. In this work, we present a novel framework called CoEvoGNN for modeling dynamic attributed graph sequence. It preserves the impact of earlier graphs on the current graph by embedding generation through the sequence of attributed graphs. It has a temporal self-attention architecture to model long-range dependencies in the evolution. Moreover, CoEvoGNN optimizes model parameters jointly on two dynamic tasks, attribute inference and link prediction over time. So the model can capture the co-evolutionary patterns of attribute change and link formation. This framework can adapt to any graph neural algorithms so we implemented and investigated three methods based on it: CoEvoGCN, CoEvoGAT, and CoEvoSAGE. Experiments demonstrate the framework (and its methods) outperforms strong baseline methods on predicting an entire unseen graph snapshot of personal attributes and interpersonal links in dynamic social graphs and financial graphs.

1 Introduction

Graphs are ubiquitous in the world and real graphs evolve over time via individual behaviors. For example, social network users establish and/or remove links between each other via the behaviors of following, mentioning, replying, and etc. The user's attributes such as textual features from generated content are also changing. These two types of dynamics, social links and user attributes, have impact on each other. Specifically, on academic co-authorship networks, researchers are looking for collaborators (reflected as neighbor nodes) who have similar or complementary knowledge [1] (which may be reflected as published keywords, a type of node attributes). And their personal research topics may change according to new collaborations. The co-evolutionary patterns of node attributes and graph structure are complex yet valuable, and need to be effectively learned by the model for forecasting future attributes and structures in graph-based applications.