This paper provides a comprehensive survey of sparse representation methods, their algorithms, and applications. Sparse representation has gained significant attention in signal processing, image processing, computer vision, and pattern recognition due to its effectiveness in various tasks such as image denoising, deblurring, inpainting, super-resolution, visual tracking, and image classification. The paper categorizes sparse representation methods into five groups based on different norm minimizations: $l_0$-norm, $l_1$-norm, and $l_{2,1}$-norm minimizations. Additionally, these methods are empirically categorized into four groups: greedy strategy approximation, constrained optimization, proximity algorithm-based optimization, and homotopy algorithm-based sparse representation. The paper discusses the rationale behind each category and presents a wide range of applications to illustrate the potential of sparse representation theory. An experimental comparative study of these algorithms is also conducted, and the Matlab code used in the paper is available online. The paper aims to provide a foundation for researchers interested in sparse representation and its applications, offering insights into the latest developments and methodologies.This paper provides a comprehensive survey of sparse representation methods, their algorithms, and applications. Sparse representation has gained significant attention in signal processing, image processing, computer vision, and pattern recognition due to its effectiveness in various tasks such as image denoising, deblurring, inpainting, super-resolution, visual tracking, and image classification. The paper categorizes sparse representation methods into five groups based on different norm minimizations: $l_0$-norm, $l_1$-norm, and $l_{2,1}$-norm minimizations. Additionally, these methods are empirically categorized into four groups: greedy strategy approximation, constrained optimization, proximity algorithm-based optimization, and homotopy algorithm-based sparse representation. The paper discusses the rationale behind each category and presents a wide range of applications to illustrate the potential of sparse representation theory. An experimental comparative study of these algorithms is also conducted, and the Matlab code used in the paper is available online. The paper aims to provide a foundation for researchers interested in sparse representation and its applications, offering insights into the latest developments and methodologies.