Kolmogorov-Arnold Networks are Radial Basis Function Networks

Kolmogorov-Arnold Networks are Radial Basis Function Networks

May 14, 2024 | Ziyao Li
This paper presents a proof-of-concept that 3-order B-splines used in Kolmogorov-Arnold Networks (KANs) can be approximated by Gaussian radial basis functions (RBFs), leading to the development of FastKAN, a faster implementation of KANs. KANs are neural network architectures inspired by the work of Kolmogorov and Arnold, designed to efficiently approximate complex, high-dimensional functions by decomposing them into simpler components. The key innovation in FastKAN is the use of Gaussian RBFs to approximate the 3-order B-spline basis, along with layer normalization to prevent input values from shifting out of the RBF domain. The paper demonstrates that FastKAN significantly improves the speed of KANs, achieving a 3.33 times acceleration in forward calculations compared to an efficient KAN implementation. Additionally, FastKAN maintains or improves the accuracy of KANs, as shown on the MNIST dataset. The findings confirm that KANs are effectively RBF networks with fixed centers.This paper presents a proof-of-concept that 3-order B-splines used in Kolmogorov-Arnold Networks (KANs) can be approximated by Gaussian radial basis functions (RBFs), leading to the development of FastKAN, a faster implementation of KANs. KANs are neural network architectures inspired by the work of Kolmogorov and Arnold, designed to efficiently approximate complex, high-dimensional functions by decomposing them into simpler components. The key innovation in FastKAN is the use of Gaussian RBFs to approximate the 3-order B-spline basis, along with layer normalization to prevent input values from shifting out of the RBF domain. The paper demonstrates that FastKAN significantly improves the speed of KANs, achieving a 3.33 times acceleration in forward calculations compared to an efficient KAN implementation. Additionally, FastKAN maintains or improves the accuracy of KANs, as shown on the MNIST dataset. The findings confirm that KANs are effectively RBF networks with fixed centers.
Reach us at info@study.space