Variational Graph Auto-Encoders

Variational Graph Auto-Encoders

21 Nov 2016 | Thomas N. Kipf, Max Welling
The variational graph autoencoder (VGAE) is a framework for unsupervised learning on graph-structured data based on the variational autoencoder (VAE). It uses latent variables to learn interpretable representations for undirected graphs. The model employs a graph convolutional network (GCN) as an encoder and a simple inner product decoder. It achieves competitive results on link prediction tasks in citation networks. Unlike most existing models, the VGAE can naturally incorporate node features, which significantly improves predictive performance on benchmark datasets. The model is defined on an undirected, unweighted graph G with N nodes. It uses an adjacency matrix A and a degree matrix D. Latent variables z_i are summarized in a matrix Z, and node features are summarized in a matrix X. The inference model is a two-layer GCN that outputs mean vectors μ_i and log σ_i. The generative model uses an inner product between latent variables to predict the adjacency matrix A. The learning process optimizes the variational lower bound L with respect to the variational parameters W_i. A Gaussian prior is used for the latent variables. For sparse adjacency matrices, terms can be re-weighted or subsampled. The model can be adapted to a non-probabilistic version, GAE, which calculates embeddings Z and the reconstructed adjacency matrix Â. Experiments on link prediction tasks on citation networks show that both VGAE and GAE achieve competitive results. Adding input features significantly improves predictive performance. The VGAE model achieves higher performance on the Cora and Citeseer datasets. Future work will explore better prior distributions, more flexible generative models, and improved scalability through stochastic gradient descent. The models are compared against spectral clustering (SC) and DeepWalk (DW) baselines. Both SC and DW do not support input features. The VGAE and GAE models use input features and achieve better performance. The results are reported using area under the ROC curve (AUC) and average precision (AP) scores. The models are trained with Adam optimizer and a learning rate of 0.01. The experiments show that the VGAE model outperforms the baselines in most cases.The variational graph autoencoder (VGAE) is a framework for unsupervised learning on graph-structured data based on the variational autoencoder (VAE). It uses latent variables to learn interpretable representations for undirected graphs. The model employs a graph convolutional network (GCN) as an encoder and a simple inner product decoder. It achieves competitive results on link prediction tasks in citation networks. Unlike most existing models, the VGAE can naturally incorporate node features, which significantly improves predictive performance on benchmark datasets. The model is defined on an undirected, unweighted graph G with N nodes. It uses an adjacency matrix A and a degree matrix D. Latent variables z_i are summarized in a matrix Z, and node features are summarized in a matrix X. The inference model is a two-layer GCN that outputs mean vectors μ_i and log σ_i. The generative model uses an inner product between latent variables to predict the adjacency matrix A. The learning process optimizes the variational lower bound L with respect to the variational parameters W_i. A Gaussian prior is used for the latent variables. For sparse adjacency matrices, terms can be re-weighted or subsampled. The model can be adapted to a non-probabilistic version, GAE, which calculates embeddings Z and the reconstructed adjacency matrix Â. Experiments on link prediction tasks on citation networks show that both VGAE and GAE achieve competitive results. Adding input features significantly improves predictive performance. The VGAE model achieves higher performance on the Cora and Citeseer datasets. Future work will explore better prior distributions, more flexible generative models, and improved scalability through stochastic gradient descent. The models are compared against spectral clustering (SC) and DeepWalk (DW) baselines. Both SC and DW do not support input features. The VGAE and GAE models use input features and achieve better performance. The results are reported using area under the ROC curve (AUC) and average precision (AP) scores. The models are trained with Adam optimizer and a learning rate of 0.01. The experiments show that the VGAE model outperforms the baselines in most cases.
Reach us at info@study.space