This paper provides a comprehensive survey of permutation-invariant neural network architectures and their applications in approximating set functions. The authors highlight the shift in research towards handling set-based inputs, driven by the increasing demand for tasks involving set data. They introduce key concepts such as permutation invariance and equivariance, which are crucial for accurately modeling set functions. The paper reviews several neural network architectures, including Deep Sets, PointNet, Set Transformer, and their variants, detailing their architectures, properties, and performance. It also discusses theoretical analyses, datasets for performance assessment, and the generalization of Deep Sets in the sense of quasi-arithmetic mean. The authors propose a special class of Deep Sets called Hölder's Power Deep Sets and explore the limitations and future directions of set-based neural networks. The paper concludes with a taxonomy of tasks involving set functions and a discussion on explainable AI and federated learning.This paper provides a comprehensive survey of permutation-invariant neural network architectures and their applications in approximating set functions. The authors highlight the shift in research towards handling set-based inputs, driven by the increasing demand for tasks involving set data. They introduce key concepts such as permutation invariance and equivariance, which are crucial for accurately modeling set functions. The paper reviews several neural network architectures, including Deep Sets, PointNet, Set Transformer, and their variants, detailing their architectures, properties, and performance. It also discusses theoretical analyses, datasets for performance assessment, and the generalization of Deep Sets in the sense of quasi-arithmetic mean. The authors propose a special class of Deep Sets called Hölder's Power Deep Sets and explore the limitations and future directions of set-based neural networks. The paper concludes with a taxonomy of tasks involving set functions and a discussion on explainable AI and federated learning.