20 Jun 2016 | Théo Trouillon, Johannes Welbl, Sébastien Riedel, Éric Gaussier, Guillaume Bouchard
This paper introduces a method for link prediction in knowledge bases using complex-valued embeddings. The approach leverages complex embeddings to handle a wide range of binary relations, including symmetric and antisymmetric ones, using the Hermitian dot product. Unlike state-of-the-art models such as Neural Tensor Networks and Holographic Embeddings, this method is simpler, as it only uses the Hermitian dot product, which is the complex counterpart of the standard dot product between real vectors. The method is scalable and outperforms alternative approaches on standard link prediction benchmarks.
The paper discusses the use of complex embeddings for low-rank matrix factorization and illustrates this with a simplified link prediction task involving a single relation type. It shows that complex embeddings can effectively capture antisymmetric relations while retaining the efficiency benefits of the dot product, which ensures linearity in both space and time complexity.
The method is extended to handle multiple types of relations by allocating an embedding to each relation and sharing entity embeddings across all relations. The model uses complex vectors to represent embeddings, allowing for the decomposition of relation matrices into symmetric and antisymmetric components. This enables the model to accurately describe both symmetric and antisymmetric relations between entities.
Experiments on synthetic and real datasets show that the model performs well, outperforming other models such as DistMult, TransE, and RESCAL. On the FB15K and WN18 datasets, the model achieves higher accuracy and efficiency, particularly in handling antisymmetric relations. The model's performance is evaluated using metrics such as Mean Reciprocal Rank (MRR) and Hits at m, with the model showing significant improvements over existing approaches.
The paper also discusses the influence of negative samples on model performance and the importance of regularization in achieving good results. It concludes that the use of complex embeddings provides a simple and effective approach to link prediction in knowledge bases, with the potential for further improvements through extensions to tensor factorization and more intelligent negative sampling procedures.This paper introduces a method for link prediction in knowledge bases using complex-valued embeddings. The approach leverages complex embeddings to handle a wide range of binary relations, including symmetric and antisymmetric ones, using the Hermitian dot product. Unlike state-of-the-art models such as Neural Tensor Networks and Holographic Embeddings, this method is simpler, as it only uses the Hermitian dot product, which is the complex counterpart of the standard dot product between real vectors. The method is scalable and outperforms alternative approaches on standard link prediction benchmarks.
The paper discusses the use of complex embeddings for low-rank matrix factorization and illustrates this with a simplified link prediction task involving a single relation type. It shows that complex embeddings can effectively capture antisymmetric relations while retaining the efficiency benefits of the dot product, which ensures linearity in both space and time complexity.
The method is extended to handle multiple types of relations by allocating an embedding to each relation and sharing entity embeddings across all relations. The model uses complex vectors to represent embeddings, allowing for the decomposition of relation matrices into symmetric and antisymmetric components. This enables the model to accurately describe both symmetric and antisymmetric relations between entities.
Experiments on synthetic and real datasets show that the model performs well, outperforming other models such as DistMult, TransE, and RESCAL. On the FB15K and WN18 datasets, the model achieves higher accuracy and efficiency, particularly in handling antisymmetric relations. The model's performance is evaluated using metrics such as Mean Reciprocal Rank (MRR) and Hits at m, with the model showing significant improvements over existing approaches.
The paper also discusses the influence of negative samples on model performance and the importance of regularization in achieving good results. It concludes that the use of complex embeddings provides a simple and effective approach to link prediction in knowledge bases, with the potential for further improvements through extensions to tensor factorization and more intelligent negative sampling procedures.