Position: Topological Deep Learning is the New Frontier for Relational Learning

Position: Topological Deep Learning is the New Frontier for Relational Learning

6 Aug 2024 | Theodore Papamarkou, Tolga Birdal, Michael Bronstein, Gunnar Carlsson, Justin Curry, Yue Gao, Mustafa Hajij, Roland Kwitt, Pietro Liò, Paolo Di Lorenzo, Vasileios Maroulas, Nina Miolan, Farzana Nasrin, Karthikeyan Natesan Ramamurthy, Bastian Rieck, Simone Scardapane, Michael T. Schaub, Petar Veličković, Bei Wang, Yusu Wang, Guo-Wei Wei, Ghada Zamzmi
Topological deep learning (TDL) is a rapidly evolving field that uses topological features to understand and design deep learning models. This paper argues that TDL is the new frontier for relational learning. TDL can complement graph representation learning and geometric deep learning by incorporating topological concepts, providing a natural choice for various machine learning settings. The paper discusses open problems in TDL, ranging from practical benefits to theoretical foundations, and outlines potential solutions and future research opportunities. It invites the scientific community to actively participate in TDL research to unlock its potential. TDL is particularly useful for relational data, as it considers topological characteristics inherent to such data. Four practical advantages of TDL are presented: (1) the topology of the underlying data space determines the choice of neural network architecture; (2) topological domains enable modeling of data with multi-way interactions; (3) TDL captures regularities inherent to manifolds; and (4) TDL captures topological equivariances in the data. These advantages make TDL a natural choice for various machine learning problems. TDL has been applied in various domains, including attributed graphs, drug design, and 2D shape analysis. However, broader adaptation of TDL in real-world applications has not yet occurred. Several application areas are plausible candidates for TDL to shine, and the paper encourages collaboration with researchers from these fields to develop real-world applications. The paper also highlights the need for higher-order datasets and benchmarks to facilitate TDL research. Current datasets are limited, and there is a scarcity of higher-order data. Developing applications of TDL can produce higher-order datasets that naturally arise from the underlying domain. A systematic assessment and generalization of graph-lifting and rewiring algorithms is a plausible path towards synthetic higher-order datasets. Software development for TDL is a major challenge, as practical implementations are scarce due to the limited availability of easy-to-use software packages for deep learning on higher-order structures. Research experimentation and deployment of TDL models are often hindered by the limited availability of software. More human capital and financial investment are required to accelerate progress in TDL. The paper also discusses the complexity and scalability of TDL. TDL models increase space and time complexity compared to GNNs, and the benefits of TDL must be weighed against the resulting costs. Scalability is a major challenge in TDL, as the multitude of higher-order domains confines the interoperability of scalable TDL models and learning algorithms. The paper also addresses the problems of explainability, generalization, and fairness in TDL. These problems are common in machine learning and have not yet been fully explored in the context of TDL. The paper discusses how TDL can play an important role in solving these problems. Theoretical foundations of TDL are also discussed, focusing on open problems and associated research directions regarding the advantages of TDL, topological representation learning, and transformers in TDL. The paper highlights the need for more theoreticalTopological deep learning (TDL) is a rapidly evolving field that uses topological features to understand and design deep learning models. This paper argues that TDL is the new frontier for relational learning. TDL can complement graph representation learning and geometric deep learning by incorporating topological concepts, providing a natural choice for various machine learning settings. The paper discusses open problems in TDL, ranging from practical benefits to theoretical foundations, and outlines potential solutions and future research opportunities. It invites the scientific community to actively participate in TDL research to unlock its potential. TDL is particularly useful for relational data, as it considers topological characteristics inherent to such data. Four practical advantages of TDL are presented: (1) the topology of the underlying data space determines the choice of neural network architecture; (2) topological domains enable modeling of data with multi-way interactions; (3) TDL captures regularities inherent to manifolds; and (4) TDL captures topological equivariances in the data. These advantages make TDL a natural choice for various machine learning problems. TDL has been applied in various domains, including attributed graphs, drug design, and 2D shape analysis. However, broader adaptation of TDL in real-world applications has not yet occurred. Several application areas are plausible candidates for TDL to shine, and the paper encourages collaboration with researchers from these fields to develop real-world applications. The paper also highlights the need for higher-order datasets and benchmarks to facilitate TDL research. Current datasets are limited, and there is a scarcity of higher-order data. Developing applications of TDL can produce higher-order datasets that naturally arise from the underlying domain. A systematic assessment and generalization of graph-lifting and rewiring algorithms is a plausible path towards synthetic higher-order datasets. Software development for TDL is a major challenge, as practical implementations are scarce due to the limited availability of easy-to-use software packages for deep learning on higher-order structures. Research experimentation and deployment of TDL models are often hindered by the limited availability of software. More human capital and financial investment are required to accelerate progress in TDL. The paper also discusses the complexity and scalability of TDL. TDL models increase space and time complexity compared to GNNs, and the benefits of TDL must be weighed against the resulting costs. Scalability is a major challenge in TDL, as the multitude of higher-order domains confines the interoperability of scalable TDL models and learning algorithms. The paper also addresses the problems of explainability, generalization, and fairness in TDL. These problems are common in machine learning and have not yet been fully explored in the context of TDL. The paper discusses how TDL can play an important role in solving these problems. Theoretical foundations of TDL are also discussed, focusing on open problems and associated research directions regarding the advantages of TDL, topological representation learning, and transformers in TDL. The paper highlights the need for more theoretical
Reach us at info@study.space