The paper introduces Kernel Point Convolution (KPConv), a novel point convolution operator designed to operate directly on point clouds without intermediate representations. KPConv uses a set of kernel points in Euclidean space to define convolution weights, which are applied to input points close to these kernel points. This approach offers more flexibility compared to fixed grid convolutions, as the number of kernel points can vary. The kernel points are continuous and can be learned by the network, enabling the extension to deformable convolutions that adapt to local geometry. KPConv is efficient and robust to varying densities due to a regular subsampling strategy. The paper presents two network architectures, KP-CNN for classification and KP-FCNN for segmentation, which outperform state-of-the-art methods on various datasets. Ablation studies and visualizations are provided to understand the learned features and the descriptive power of deformable KPConv. The results show that deformable KPConv improves performance on diverse datasets, particularly in challenging tasks like object detection and point cloud completion.The paper introduces Kernel Point Convolution (KPConv), a novel point convolution operator designed to operate directly on point clouds without intermediate representations. KPConv uses a set of kernel points in Euclidean space to define convolution weights, which are applied to input points close to these kernel points. This approach offers more flexibility compared to fixed grid convolutions, as the number of kernel points can vary. The kernel points are continuous and can be learned by the network, enabling the extension to deformable convolutions that adapt to local geometry. KPConv is efficient and robust to varying densities due to a regular subsampling strategy. The paper presents two network architectures, KP-CNN for classification and KP-FCNN for segmentation, which outperform state-of-the-art methods on various datasets. Ablation studies and visualizations are provided to understand the learned features and the descriptive power of deformable KPConv. The results show that deformable KPConv improves performance on diverse datasets, particularly in challenging tasks like object detection and point cloud completion.