This survey presents a comprehensive overview of recent advancements in Visual Anomaly Detection (VAD), focusing on three main challenges: data scarcity, diversity of visual modalities, and complexity of hierarchical anomalies. The paper first provides background on VAD, including its conceptual definitions and general formulation. It then reviews existing methods from three perspectives: sample number, data modality, and anomaly hierarchy. The survey also discusses potential future research directions, including generic, multimodal, and holistic VAD. The survey highlights that VAD faces significant challenges, such as the scarcity of training data, the diversity of visual modalities, and the complexity of hierarchical anomalies. To address these challenges, various approaches have been proposed, including semi-supervised, unsupervised, few-shot, and zero-shot VAD methods. These methods aim to detect anomalies in different scenarios, such as industrial defect inspection, medical image analysis, and autonomous driving. The paper also discusses the importance of data modalities in VAD, including 2D RGB images, 3D point clouds, and multi-modal data. It reviews methods for detecting structural and semantic anomalies, emphasizing the need for models that can understand both local and global contexts. Additionally, the survey explores future directions for VAD, including the development of generic, multimodal, and holistic VAD frameworks that can handle diverse scenarios and improve the accuracy and efficiency of anomaly detection. The survey concludes that addressing the challenges in VAD requires a combination of advanced techniques, including foundation models, multimodal learning, and holistic approaches. These methods aim to enhance the performance of VAD in real-world applications, ensuring reliable and safe technological systems.This survey presents a comprehensive overview of recent advancements in Visual Anomaly Detection (VAD), focusing on three main challenges: data scarcity, diversity of visual modalities, and complexity of hierarchical anomalies. The paper first provides background on VAD, including its conceptual definitions and general formulation. It then reviews existing methods from three perspectives: sample number, data modality, and anomaly hierarchy. The survey also discusses potential future research directions, including generic, multimodal, and holistic VAD. The survey highlights that VAD faces significant challenges, such as the scarcity of training data, the diversity of visual modalities, and the complexity of hierarchical anomalies. To address these challenges, various approaches have been proposed, including semi-supervised, unsupervised, few-shot, and zero-shot VAD methods. These methods aim to detect anomalies in different scenarios, such as industrial defect inspection, medical image analysis, and autonomous driving. The paper also discusses the importance of data modalities in VAD, including 2D RGB images, 3D point clouds, and multi-modal data. It reviews methods for detecting structural and semantic anomalies, emphasizing the need for models that can understand both local and global contexts. Additionally, the survey explores future directions for VAD, including the development of generic, multimodal, and holistic VAD frameworks that can handle diverse scenarios and improve the accuracy and efficiency of anomaly detection. The survey concludes that addressing the challenges in VAD requires a combination of advanced techniques, including foundation models, multimodal learning, and holistic approaches. These methods aim to enhance the performance of VAD in real-world applications, ensuring reliable and safe technological systems.