This paper provides a comprehensive overview of optimization techniques for sparse estimation methods, which aim to use or obtain parsimonious representations of data or models. The authors cover various methods such as proximal methods, block-coordinate descent, reweighted $\ell_2$-penalized techniques, working-set and homotopy methods, and non-convex formulations. They also discuss the application of these techniques to structured sparsity and multiple kernel learning. The paper includes a detailed presentation of the theoretical foundations, including subgradient theory, Fenchel duality, and quadratic variational formulations. Additionally, it presents experimental results comparing the performance of different algorithms in terms of speed and convergence. The paper is organized into several sections, each focusing on specific aspects of sparse estimation, from notation and loss functions to specific optimization techniques and their applications.This paper provides a comprehensive overview of optimization techniques for sparse estimation methods, which aim to use or obtain parsimonious representations of data or models. The authors cover various methods such as proximal methods, block-coordinate descent, reweighted $\ell_2$-penalized techniques, working-set and homotopy methods, and non-convex formulations. They also discuss the application of these techniques to structured sparsity and multiple kernel learning. The paper includes a detailed presentation of the theoretical foundations, including subgradient theory, Fenchel duality, and quadratic variational formulations. Additionally, it presents experimental results comparing the performance of different algorithms in terms of speed and convergence. The paper is organized into several sections, each focusing on specific aspects of sparse estimation, from notation and loss functions to specific optimization techniques and their applications.