This paper presents an overview of 20 evaluation metrics for 3D medical image segmentation, selected based on a comprehensive literature review. It addresses challenges in evaluating medical segmentation, including metric selection, multiple definitions of metrics, inefficiency of metric calculations, and lack of support for fuzzy segmentation. The paper provides a discussion of metric properties to guide the selection of evaluation metrics and proposes an efficient evaluation tool that implements the 20 metrics. The tool is optimized for speed and memory usage, especially for large volumes like whole-body MRI or CT scans. It is implemented as an open-source project. The paper discusses different quality aspects in 3D medical image segmentation, including accuracy, precision, and efficiency. It defines metrics that can indicate various types of segmentation errors, such as added regions, added background, inside holes, and border holes. The paper also provides fuzzy definitions for all metrics, allowing for uncertainty in medical image segmentation to be considered in evaluation. It generalizes metrics for segmentation with multiple labels and provides an efficient open-source implementation of all 20 metrics, which outperforms state-of-the-art tools in common cases of medical image segmentation. The paper is organized into sections covering ethics approval, evaluation metrics for 3D image segmentation, metric definitions and algorithms, multiple definitions of metrics in the literature, implementation details, testing efficiency, metric selection, and availability and requirements. The paper emphasizes the need for a standard evaluation tool for medical image segmentation that standardizes metrics and their definitions. It highlights the importance of considering fuzzy segmentation and provides a comprehensive set of metrics for evaluating 3D medical image segmentation. The paper also discusses the properties of the metrics, their sensitivity to different factors, and guidelines for selecting a subset of metrics suitable for the data and segmentation task.This paper presents an overview of 20 evaluation metrics for 3D medical image segmentation, selected based on a comprehensive literature review. It addresses challenges in evaluating medical segmentation, including metric selection, multiple definitions of metrics, inefficiency of metric calculations, and lack of support for fuzzy segmentation. The paper provides a discussion of metric properties to guide the selection of evaluation metrics and proposes an efficient evaluation tool that implements the 20 metrics. The tool is optimized for speed and memory usage, especially for large volumes like whole-body MRI or CT scans. It is implemented as an open-source project. The paper discusses different quality aspects in 3D medical image segmentation, including accuracy, precision, and efficiency. It defines metrics that can indicate various types of segmentation errors, such as added regions, added background, inside holes, and border holes. The paper also provides fuzzy definitions for all metrics, allowing for uncertainty in medical image segmentation to be considered in evaluation. It generalizes metrics for segmentation with multiple labels and provides an efficient open-source implementation of all 20 metrics, which outperforms state-of-the-art tools in common cases of medical image segmentation. The paper is organized into sections covering ethics approval, evaluation metrics for 3D image segmentation, metric definitions and algorithms, multiple definitions of metrics in the literature, implementation details, testing efficiency, metric selection, and availability and requirements. The paper emphasizes the need for a standard evaluation tool for medical image segmentation that standardizes metrics and their definitions. It highlights the importance of considering fuzzy segmentation and provides a comprehensive set of metrics for evaluating 3D medical image segmentation. The paper also discusses the properties of the metrics, their sensitivity to different factors, and guidelines for selecting a subset of metrics suitable for the data and segmentation task.