21 May 2014 | Joan Bruna, Wojciech Zaremba, Arthur Szlam, Yann LeCun
This paper explores the generalization of Convolutional Neural Networks (CNNs) to signals defined on more general domains without the action of a translation group. The authors propose two constructions: one based on hierarchical clustering of the domain and another based on the spectrum of the graph Laplacian. These constructions aim to learn convolutional layers with a number of parameters independent of the input size, making them efficient for low-dimensional graphs.
The spatial construction extends properties of translation and multiscale clustering to general graphs, defining "locally" connected and pooling layers that require \(O(n)\) parameters. The spectral construction leverages the properties of convolutions in the Fourier domain, using the graph Laplacian to diagonalize convolutions, resulting in a construction with \(O(1)\) parameters per feature map.
The paper validates these constructions through experiments on low-dimensional graph datasets, demonstrating their efficiency and effectiveness. The spectral construction, in particular, shows improved performance by enforcing spatial localization through smooth spectral multipliers. The authors also discuss the relationship with previous work on wavelets and grid topologies, and propose future directions for improving the constructions and applying them to more complex problems.This paper explores the generalization of Convolutional Neural Networks (CNNs) to signals defined on more general domains without the action of a translation group. The authors propose two constructions: one based on hierarchical clustering of the domain and another based on the spectrum of the graph Laplacian. These constructions aim to learn convolutional layers with a number of parameters independent of the input size, making them efficient for low-dimensional graphs.
The spatial construction extends properties of translation and multiscale clustering to general graphs, defining "locally" connected and pooling layers that require \(O(n)\) parameters. The spectral construction leverages the properties of convolutions in the Fourier domain, using the graph Laplacian to diagonalize convolutions, resulting in a construction with \(O(1)\) parameters per feature map.
The paper validates these constructions through experiments on low-dimensional graph datasets, demonstrating their efficiency and effectiveness. The spectral construction, in particular, shows improved performance by enforcing spatial localization through smooth spectral multipliers. The authors also discuss the relationship with previous work on wavelets and grid topologies, and propose future directions for improving the constructions and applying them to more complex problems.