The paper introduces Point Transformer, a deep neural network designed to process unordered and unstructured point sets directly. The network aims to extract both local and global features and relate them using a local-global attention mechanism to capture spatial point relations and shape information. A key component, SortNet, is introduced to induce permutation invariance by selecting points based on learned scores, ensuring that the output is permutation invariant. The Point Transformer is evaluated on standard benchmarks for classification and part segmentation, demonstrating competitive results compared to prior work. The code for Point Transformer is publicly available. The contributions of the paper include the development of Point Transformer, the design of SortNet, and the evaluation of the network's performance on various tasks.The paper introduces Point Transformer, a deep neural network designed to process unordered and unstructured point sets directly. The network aims to extract both local and global features and relate them using a local-global attention mechanism to capture spatial point relations and shape information. A key component, SortNet, is introduced to induce permutation invariance by selecting points based on learned scores, ensuring that the output is permutation invariant. The Point Transformer is evaluated on standard benchmarks for classification and part segmentation, demonstrating competitive results compared to prior work. The code for Point Transformer is publicly available. The contributions of the paper include the development of Point Transformer, the design of SortNet, and the evaluation of the network's performance on various tasks.