This paper introduces MuSc, a novel zero-shot method for industrial anomaly classification (AC) and segmentation (AS). MuSc leverages the unlabeled test images to exploit both normal and abnormal cues, which are often overlooked in prior methods. The key observation is that normal image patches can find a large number of similar patches in other unlabeled images, while abnormal patches have few similar patches. MuSc consists of three main components: Local Neighborhood Aggregation with Multiple Degrees (LNAMD), Mutual Scoring Mechanism (MSM), and Re-scoring with Constrained Image-level Neighborhood (RsCIN). LNAMD extracts patch features using ViT, while MSM assigns anomaly scores to each patch based on the unlabeled images. RsCIN optimizes the anomaly classification by considering the relationship between image-level features. Experimental results on the MVtec AD and VisA datasets demonstrate that MuSc outperforms existing zero-shot and few-shot methods, achieving significant improvements in PRO, AP, and AUROC metrics. The method is also competitive with some full-shot methods and shows robustness to different aggregation degrees and image sizes.This paper introduces MuSc, a novel zero-shot method for industrial anomaly classification (AC) and segmentation (AS). MuSc leverages the unlabeled test images to exploit both normal and abnormal cues, which are often overlooked in prior methods. The key observation is that normal image patches can find a large number of similar patches in other unlabeled images, while abnormal patches have few similar patches. MuSc consists of three main components: Local Neighborhood Aggregation with Multiple Degrees (LNAMD), Mutual Scoring Mechanism (MSM), and Re-scoring with Constrained Image-level Neighborhood (RsCIN). LNAMD extracts patch features using ViT, while MSM assigns anomaly scores to each patch based on the unlabeled images. RsCIN optimizes the anomaly classification by considering the relationship between image-level features. Experimental results on the MVtec AD and VisA datasets demonstrate that MuSc outperforms existing zero-shot and few-shot methods, achieving significant improvements in PRO, AP, and AUROC metrics. The method is also competitive with some full-shot methods and shows robustness to different aggregation degrees and image sizes.