Neural network pruning is the process of reducing the size of a neural network by removing parameters. This technique has been widely studied in recent years, but the field suffers from a lack of standardized benchmarks and metrics, making it difficult to compare pruning techniques or assess progress over the past three decades. To address this, the authors identify issues with current practices, suggest concrete remedies, and introduce ShrinkBench, an open-source framework for standardized evaluation of pruning methods. ShrinkBench allows for comprehensive evaluation of pruning techniques, helping to avoid common pitfalls when comparing methods. The paper provides a meta-analysis of the literature, including an overview of pruning approaches and consistent findings. It highlights that pruning parameters based on their magnitudes can significantly compress networks without reducing accuracy, and many pruning methods outperform random pruning. However, the lack of standard datasets, networks, metrics, and experimental practices hinders meaningful comparisons between methods. The authors discuss various pruning methods, including unstructured and structured pruning, scoring methods, scheduling, and fine-tuning. They also evaluate pruning based on different goals, such as reducing storage footprint, computational cost, and energy requirements. The paper emphasizes the trade-off between model efficiency and quality, noting that pruning increases efficiency while typically decreasing accuracy. The authors also highlight the importance of using standardized metrics and experimental practices to enable direct comparisons between pruning methods. They argue that current practices often lack consistency, leading to fragmented results and making it difficult to assess the relative efficacy of different methods. The paper concludes that standardized experiments are essential for evaluating neural network pruning methods and that ShrinkBench provides a framework to facilitate such evaluations.Neural network pruning is the process of reducing the size of a neural network by removing parameters. This technique has been widely studied in recent years, but the field suffers from a lack of standardized benchmarks and metrics, making it difficult to compare pruning techniques or assess progress over the past three decades. To address this, the authors identify issues with current practices, suggest concrete remedies, and introduce ShrinkBench, an open-source framework for standardized evaluation of pruning methods. ShrinkBench allows for comprehensive evaluation of pruning techniques, helping to avoid common pitfalls when comparing methods. The paper provides a meta-analysis of the literature, including an overview of pruning approaches and consistent findings. It highlights that pruning parameters based on their magnitudes can significantly compress networks without reducing accuracy, and many pruning methods outperform random pruning. However, the lack of standard datasets, networks, metrics, and experimental practices hinders meaningful comparisons between methods. The authors discuss various pruning methods, including unstructured and structured pruning, scoring methods, scheduling, and fine-tuning. They also evaluate pruning based on different goals, such as reducing storage footprint, computational cost, and energy requirements. The paper emphasizes the trade-off between model efficiency and quality, noting that pruning increases efficiency while typically decreasing accuracy. The authors also highlight the importance of using standardized metrics and experimental practices to enable direct comparisons between pruning methods. They argue that current practices often lack consistency, leading to fragmented results and making it difficult to assess the relative efficacy of different methods. The paper concludes that standardized experiments are essential for evaluating neural network pruning methods and that ShrinkBench provides a framework to facilitate such evaluations.