This paper surveys and categorizes methods for creating diverse ensembles in classification tasks. It begins by discussing the concept of error diversity in ensemble learning, noting that while ensemble methods often outperform single predictors, the exact meaning of error diversity remains unclear, especially in classification. The paper reviews existing explanations of diversity, including both regression and classification contexts, and highlights the importance of diversity in improving ensemble performance. In regression, the Ambiguity decomposition and bias-variance-covariance decomposition are discussed, showing how diversity contributes to reduced error. The paper then explores the connection between these decompositions and how they can be used to understand ensemble performance. In classification, the concept of diversity is more complex, as the outputs are non-ordinal, making traditional measures like covariance undefined. The paper discusses various approaches to quantifying classification error diversity, including heuristic metrics and entropy-based measures. The paper introduces a taxonomy of diversity creation methods, distinguishing between implicit and explicit methods. Implicit methods rely on randomness to generate diversity, while explicit methods directly manipulate the training data or model architecture to ensure diversity. The paper also discusses different dimensions along which diversity can be applied, such as starting points in hypothesis space, accessible hypotheses, and traversal of hypothesis space. The paper concludes that while there is a growing understanding of diversity in ensemble learning, the concept remains ill-defined, and further research is needed to develop a comprehensive framework for understanding and measuring classification error diversity. The paper proposes new directions for future research, emphasizing the need for a more formal and comprehensive understanding of diversity in classification tasks.This paper surveys and categorizes methods for creating diverse ensembles in classification tasks. It begins by discussing the concept of error diversity in ensemble learning, noting that while ensemble methods often outperform single predictors, the exact meaning of error diversity remains unclear, especially in classification. The paper reviews existing explanations of diversity, including both regression and classification contexts, and highlights the importance of diversity in improving ensemble performance. In regression, the Ambiguity decomposition and bias-variance-covariance decomposition are discussed, showing how diversity contributes to reduced error. The paper then explores the connection between these decompositions and how they can be used to understand ensemble performance. In classification, the concept of diversity is more complex, as the outputs are non-ordinal, making traditional measures like covariance undefined. The paper discusses various approaches to quantifying classification error diversity, including heuristic metrics and entropy-based measures. The paper introduces a taxonomy of diversity creation methods, distinguishing between implicit and explicit methods. Implicit methods rely on randomness to generate diversity, while explicit methods directly manipulate the training data or model architecture to ensure diversity. The paper also discusses different dimensions along which diversity can be applied, such as starting points in hypothesis space, accessible hypotheses, and traversal of hypothesis space. The paper concludes that while there is a growing understanding of diversity in ensemble learning, the concept remains ill-defined, and further research is needed to develop a comprehensive framework for understanding and measuring classification error diversity. The paper proposes new directions for future research, emphasizing the need for a more formal and comprehensive understanding of diversity in classification tasks.