Convolutional Kolmogorov-Arnold Networks

Convolutional Kolmogorov-Arnold Networks

June 21, 2024 | Alexander Dylan Bodner, Jack Natan Spolski, Antonio Santiago Tepsich, Santiago Pourteau
This paper introduces Convolutional Kolmogorov-Arnold Networks (Convolutional KANs), an innovative alternative to standard Convolutional Neural Networks (CNNs) that have transformed computer vision. By integrating non-linear activation functions from Kolmogorov-Arnold Networks (KANs) into convolutions, Convolutional KANs create a new layer that maintains similar accuracy to traditional architectures while using half the parameters. This reduction in parameters opens new avenues for optimizing neural network architectures. The paper explores the adaptation of KANs to convolutional layers, a common element in many CNN architectures. Traditional CNNs use fixed activation functions and linear transformations, but KANs offer flexibility and reduced parametric complexity through spline-based convolutional layers. The authors propose a KAN convolutional layer that uses learnable non-linear functions based on B-Splines, allowing the network to capture non-linear relationships more effectively. The paper evaluates the performance of Convolutional KANs against traditional models on MNIST and Fashion-MNIST datasets. Results show that Convolutional KANs achieve competitive accuracy with significantly fewer parameters. For example, the KKAN (Small) model with ~90k parameters achieves 0.22% less accuracy than the CNN (Medium) with 157k parameters on MNIST. Similarly, on Fashion MNIST, the KKAN (Small) model with ~95k parameters achieves 89.69% accuracy, slightly less than the CNN (Medium) with 160k parameters. The paper also discusses the limitations of KANs, including the need for further optimization and interpretation of the learned B-Splines. Future work includes analyzing the performance of KANs on more complex datasets like CIFAR-10 and ImageNet, improving grid extension methods, and optimizing KAN layers for faster inference and training times. The study highlights the potential of Convolutional KANs as an alternative to traditional CNNs in computer vision tasks.This paper introduces Convolutional Kolmogorov-Arnold Networks (Convolutional KANs), an innovative alternative to standard Convolutional Neural Networks (CNNs) that have transformed computer vision. By integrating non-linear activation functions from Kolmogorov-Arnold Networks (KANs) into convolutions, Convolutional KANs create a new layer that maintains similar accuracy to traditional architectures while using half the parameters. This reduction in parameters opens new avenues for optimizing neural network architectures. The paper explores the adaptation of KANs to convolutional layers, a common element in many CNN architectures. Traditional CNNs use fixed activation functions and linear transformations, but KANs offer flexibility and reduced parametric complexity through spline-based convolutional layers. The authors propose a KAN convolutional layer that uses learnable non-linear functions based on B-Splines, allowing the network to capture non-linear relationships more effectively. The paper evaluates the performance of Convolutional KANs against traditional models on MNIST and Fashion-MNIST datasets. Results show that Convolutional KANs achieve competitive accuracy with significantly fewer parameters. For example, the KKAN (Small) model with ~90k parameters achieves 0.22% less accuracy than the CNN (Medium) with 157k parameters on MNIST. Similarly, on Fashion MNIST, the KKAN (Small) model with ~95k parameters achieves 89.69% accuracy, slightly less than the CNN (Medium) with 160k parameters. The paper also discusses the limitations of KANs, including the need for further optimization and interpretation of the learned B-Splines. Future work includes analyzing the performance of KANs on more complex datasets like CIFAR-10 and ImageNet, improving grid extension methods, and optimizing KAN layers for faster inference and training times. The study highlights the potential of Convolutional KANs as an alternative to traditional CNNs in computer vision tasks.
Reach us at info@study.space
Understanding Convolutional Kolmogorov-Arnold Networks