This paper reviews derivative-free optimization algorithms and compares 22 software implementations on a test set of 502 problems, including convex, nonconvex, smooth, and nonsmooth problems. The algorithms were tested under the same conditions and ranked based on their ability to find near-global solutions, improve starting points, and refine near-optimal solutions. A total of 112,448 problem instances were solved. The study found that the performance of solvers decreases with increasing problem size. TOMLAB/MULTIMIN, TOMLAB/GLCCLUSTER, MCS, and TOMLAB/LGO performed best in terms of solution quality within 2,500 function evaluations. These global solvers outperformed local solvers even for convex problems. TOMLAB/OQNLP, NEWUOA, and TOMLAB/MULTIMIN showed superior performance in refining near-optimal solutions. The paper also reviews local and global search methods, including direct search, model-based search, and stochastic methods. It discusses various derivative-free optimization software implementations, including ASA, BOBYQA, CMA-ES, DAKOTA, DFO, FMINSEARCH, GLOBAL, HOPSPACK, IMFIL, MCS, NEWUOA, NOMAD, PSWARM, SID-PSM, SNOBFIT, and TOMLAB solvers. The study highlights the importance of surrogate models and the need for further research in this area.This paper reviews derivative-free optimization algorithms and compares 22 software implementations on a test set of 502 problems, including convex, nonconvex, smooth, and nonsmooth problems. The algorithms were tested under the same conditions and ranked based on their ability to find near-global solutions, improve starting points, and refine near-optimal solutions. A total of 112,448 problem instances were solved. The study found that the performance of solvers decreases with increasing problem size. TOMLAB/MULTIMIN, TOMLAB/GLCCLUSTER, MCS, and TOMLAB/LGO performed best in terms of solution quality within 2,500 function evaluations. These global solvers outperformed local solvers even for convex problems. TOMLAB/OQNLP, NEWUOA, and TOMLAB/MULTIMIN showed superior performance in refining near-optimal solutions. The paper also reviews local and global search methods, including direct search, model-based search, and stochastic methods. It discusses various derivative-free optimization software implementations, including ASA, BOBYQA, CMA-ES, DAKOTA, DFO, FMINSEARCH, GLOBAL, HOPSPACK, IMFIL, MCS, NEWUOA, NOMAD, PSWARM, SID-PSM, SNOBFIT, and TOMLAB solvers. The study highlights the importance of surrogate models and the need for further research in this area.