April 4, 2018 | Ryan J. Urbanowicz, Melissa Meeker, William LaCava, Randal S. Olson, Jason H. Moore
This paper focuses on Relief-based algorithms (RBAs), a family of filter-style feature selection methods that balance computational efficiency with sensitivity to complex patterns of association. The authors provide an overview of feature selection, emphasizing the importance of selecting relevant features while discarding irrelevant ones. They introduce the original Relief algorithm and its key concepts, including how it works, how feature weights are interpreted, and why it can detect feature interactions without explicitly evaluating combinations of features. The paper also reviews the development of ReliefF and other derivative algorithms, highlighting their contributions, strategies, functionality, time complexity, and adaptability to various data characteristics. Additionally, the authors discuss the limitations of Relief, such as its inability to handle large numbers of irrelevant features and its lack of ability to detect higher-order interactions. The review concludes with a summary of software availability and a comprehensive table organizing key RBAs, detailing their differences, time complexities, and areas of focus (core algorithm, iterative approach, efficiency, and data type handling).This paper focuses on Relief-based algorithms (RBAs), a family of filter-style feature selection methods that balance computational efficiency with sensitivity to complex patterns of association. The authors provide an overview of feature selection, emphasizing the importance of selecting relevant features while discarding irrelevant ones. They introduce the original Relief algorithm and its key concepts, including how it works, how feature weights are interpreted, and why it can detect feature interactions without explicitly evaluating combinations of features. The paper also reviews the development of ReliefF and other derivative algorithms, highlighting their contributions, strategies, functionality, time complexity, and adaptability to various data characteristics. Additionally, the authors discuss the limitations of Relief, such as its inability to handle large numbers of irrelevant features and its lack of ability to detect higher-order interactions. The review concludes with a summary of software availability and a comprehensive table organizing key RBAs, detailing their differences, time complexities, and areas of focus (core algorithm, iterative approach, efficiency, and data type handling).