LiGNN is a large-scale Graph Neural Networks (GNN) framework deployed at LinkedIn. The paper presents insights into developing and deploying GNNs at scale, including algorithmic improvements for GNN representation learning. These improvements include temporal graph architectures with long-term losses, effective cold start solutions via graph densification, ID embeddings, and multi-hop neighbor sampling. The authors describe how they built and sped up their large-scale training on LinkedIn graphs using adaptive sampling, grouping and slicing of training data batches, a specialized shared-memory queue, and local gradient optimization. Deployment lessons and A/B test experiments show that the techniques contributed to improvements in job application hearing back rate, Ads CTR, Feed engaged daily active users, session lift, and weekly active user lift.
The paper discusses challenges in developing GNNs at scale, including GNN training scalability, diverse entities, cold start, and dynamic systems. The authors address these challenges with advancements in GNNs, including scalable GNN training, handling diverse entities, cold start solutions, and dynamic systems. The paper also presents a heterogeneous graph for LinkedIn GNN models, which contains tens of node types and edge types. The graph includes engagement edges, affinity edges, and attribute edges.
The paper describes the GNN model architecture, including the encoder-decoder structure, and the use of GraphSAGE-style frameworks for inductive learning. The paper also discusses temporal graphs, which incorporate temporal sequence modeling within GNNs. The authors introduce a temporal model that includes a static SAGE-encoder and a transformer-based temporal sequence model. The model uses time-based node sampling to capture the last N activities of a member before a certain time.
The paper also discusses graph densification, which involves adding artificial edges based on auxiliary information to improve the performance of GNNs. The authors use approximate nearest neighbor search to identify similar high-out-degree nodes for low-out-degree nodes. The paper also presents multi-hop graph sampling techniques, including multi-hop random/weighted sampling and multi-hop Personalized PageRank (PPR) sampling.
The paper discusses training stability and speed, including techniques such as adaptive neighbor sampling, grouping and slicing, and shared-memory queue. The authors also discuss the use of mixed precision training and local gradient aggregation to improve training speed. The paper presents results from experiments in various applications, including Follow Feed, Out-of-Network Feed, Job Recommendations, People Recommendations, and Ads. The results show significant improvements in key metrics, including recall, AUC, and CTR. The paper concludes with deployment lessons, including the importance of impression discount before retrieval and the scalability of GNN training using a Graph Engine.LiGNN is a large-scale Graph Neural Networks (GNN) framework deployed at LinkedIn. The paper presents insights into developing and deploying GNNs at scale, including algorithmic improvements for GNN representation learning. These improvements include temporal graph architectures with long-term losses, effective cold start solutions via graph densification, ID embeddings, and multi-hop neighbor sampling. The authors describe how they built and sped up their large-scale training on LinkedIn graphs using adaptive sampling, grouping and slicing of training data batches, a specialized shared-memory queue, and local gradient optimization. Deployment lessons and A/B test experiments show that the techniques contributed to improvements in job application hearing back rate, Ads CTR, Feed engaged daily active users, session lift, and weekly active user lift.
The paper discusses challenges in developing GNNs at scale, including GNN training scalability, diverse entities, cold start, and dynamic systems. The authors address these challenges with advancements in GNNs, including scalable GNN training, handling diverse entities, cold start solutions, and dynamic systems. The paper also presents a heterogeneous graph for LinkedIn GNN models, which contains tens of node types and edge types. The graph includes engagement edges, affinity edges, and attribute edges.
The paper describes the GNN model architecture, including the encoder-decoder structure, and the use of GraphSAGE-style frameworks for inductive learning. The paper also discusses temporal graphs, which incorporate temporal sequence modeling within GNNs. The authors introduce a temporal model that includes a static SAGE-encoder and a transformer-based temporal sequence model. The model uses time-based node sampling to capture the last N activities of a member before a certain time.
The paper also discusses graph densification, which involves adding artificial edges based on auxiliary information to improve the performance of GNNs. The authors use approximate nearest neighbor search to identify similar high-out-degree nodes for low-out-degree nodes. The paper also presents multi-hop graph sampling techniques, including multi-hop random/weighted sampling and multi-hop Personalized PageRank (PPR) sampling.
The paper discusses training stability and speed, including techniques such as adaptive neighbor sampling, grouping and slicing, and shared-memory queue. The authors also discuss the use of mixed precision training and local gradient aggregation to improve training speed. The paper presents results from experiments in various applications, including Follow Feed, Out-of-Network Feed, Job Recommendations, People Recommendations, and Ads. The results show significant improvements in key metrics, including recall, AUC, and CTR. The paper concludes with deployment lessons, including the importance of impression discount before retrieval and the scalability of GNN training using a Graph Engine.