SO-Net is a novel architecture designed for deep learning with unordered point clouds, addressing the challenges of spatial distribution and permutation invariance. The network models the spatial distribution of point clouds using a Self-Organizing Map (SOM), which enables hierarchical feature extraction on individual points and SOM nodes. The SOM nodes are trained with unsupervised competitive learning, ensuring permutation invariance through fixed initial nodes and batch updates. The network's receptive field overlap is controlled by point-to-node k-nearest neighbor (kNN) search, allowing for efficient local feature aggregation. SO-Net demonstrates superior performance in various tasks such as point cloud reconstruction, classification, object part segmentation, and shape retrieval, outperforming or matching state-of-the-art approaches. Additionally, the training speed of SO-Net is significantly faster due to its parallelizability and simplicity. The code for SO-Net is available on the project website.SO-Net is a novel architecture designed for deep learning with unordered point clouds, addressing the challenges of spatial distribution and permutation invariance. The network models the spatial distribution of point clouds using a Self-Organizing Map (SOM), which enables hierarchical feature extraction on individual points and SOM nodes. The SOM nodes are trained with unsupervised competitive learning, ensuring permutation invariance through fixed initial nodes and batch updates. The network's receptive field overlap is controlled by point-to-node k-nearest neighbor (kNN) search, allowing for efficient local feature aggregation. SO-Net demonstrates superior performance in various tasks such as point cloud reconstruction, classification, object part segmentation, and shape retrieval, outperforming or matching state-of-the-art approaches. Additionally, the training speed of SO-Net is significantly faster due to its parallelizability and simplicity. The code for SO-Net is available on the project website.