Kolmogorov-Arnold Networks are Radial Basis Function Networks

Kolmogorov-Arnold Networks are Radial Basis Function Networks

May 14, 2024 | Ziyao Li
This paper presents FastKAN, a faster implementation of Kolmogorov-Arnold Networks (KANs), which are neural networks inspired by the work of Kolmogorov and Arnold on representing continuous functions. KANs decompose complex functions into simpler components, enabling efficient learning. The original KANs use 3-order B-splines, which are theoretically capable of approximating smooth functions but can be computationally expensive due to the need for grid rescaling during training. FastKAN replaces the B-spline basis with Gaussian radial basis functions (RBFs), significantly improving computational efficiency. This substitution allows for a simpler implementation without loss of accuracy. Gaussian RBFs are functions whose value depends only on the distance from a center point, and they are widely used in function approximation. The Gaussian RBF is defined as φ(r) = exp(-r²/(2h²)), where r is the radial distance and h controls the function's width. The paper evaluates FastKAN's performance on NVIDIA V100 GPUs, showing that it accelerates the forward calculation of efficient_kan by 3.33 times. Accuracy tests on the MNIST dataset show that FastKAN performs as well as or better than KANs. The results indicate that KANs are indeed RBF networks with fixed centers. FastKAN's use of Gaussian RBFs and layer normalization ensures that inputs remain within the domain of the RBFs, preventing performance degradation. The paper concludes that replacing B-splines with Gaussian RBFs in KANs leads to a faster and more efficient implementation.This paper presents FastKAN, a faster implementation of Kolmogorov-Arnold Networks (KANs), which are neural networks inspired by the work of Kolmogorov and Arnold on representing continuous functions. KANs decompose complex functions into simpler components, enabling efficient learning. The original KANs use 3-order B-splines, which are theoretically capable of approximating smooth functions but can be computationally expensive due to the need for grid rescaling during training. FastKAN replaces the B-spline basis with Gaussian radial basis functions (RBFs), significantly improving computational efficiency. This substitution allows for a simpler implementation without loss of accuracy. Gaussian RBFs are functions whose value depends only on the distance from a center point, and they are widely used in function approximation. The Gaussian RBF is defined as φ(r) = exp(-r²/(2h²)), where r is the radial distance and h controls the function's width. The paper evaluates FastKAN's performance on NVIDIA V100 GPUs, showing that it accelerates the forward calculation of efficient_kan by 3.33 times. Accuracy tests on the MNIST dataset show that FastKAN performs as well as or better than KANs. The results indicate that KANs are indeed RBF networks with fixed centers. FastKAN's use of Gaussian RBFs and layer normalization ensures that inputs remain within the domain of the RBFs, preventing performance degradation. The paper concludes that replacing B-splines with Gaussian RBFs in KANs leads to a faster and more efficient implementation.
Reach us at info@study.space