This paper reviews derivative-free optimization algorithms and compares 22 software implementations using a test set of 502 problems. Derivative-free optimization is crucial when derivative information is unavailable, unreliable, or impractical to obtain. The review covers both direct and model-based algorithms, including local and global search methods. Direct algorithms compute search directions directly from function values, while model-based algorithms use surrogate models to guide the search. Local algorithms focus on finding near-optimal solutions within a bounded region, while global algorithms aim to find the global optimum. The paper also discusses deterministic and stochastic global search algorithms, such as Lipschitzian-based partitioning techniques, response surface methods, and genetic algorithms. The software implementations tested include TOMLAB/MULTI-MIN, TOMLAB/GLCCLUSTER, MCS, TOMLAB/LGO, TOMLAB/OQNLP, NEWUOA, and PSWARM. The study finds that the quality of solutions decreases with increasing problem size, and certain solvers like TOMLAB/MULTI-MIN, TOMLAB/GLCCLUSTER, MCS, and TOMLAB/LGO perform better in terms of solution quality within 2,500 function evaluations. Additionally, TOMLAB/OQNLP, NEWUOA, and TOMLAB/MULTIMIN excel in refining near-optimal solutions.This paper reviews derivative-free optimization algorithms and compares 22 software implementations using a test set of 502 problems. Derivative-free optimization is crucial when derivative information is unavailable, unreliable, or impractical to obtain. The review covers both direct and model-based algorithms, including local and global search methods. Direct algorithms compute search directions directly from function values, while model-based algorithms use surrogate models to guide the search. Local algorithms focus on finding near-optimal solutions within a bounded region, while global algorithms aim to find the global optimum. The paper also discusses deterministic and stochastic global search algorithms, such as Lipschitzian-based partitioning techniques, response surface methods, and genetic algorithms. The software implementations tested include TOMLAB/MULTI-MIN, TOMLAB/GLCCLUSTER, MCS, TOMLAB/LGO, TOMLAB/OQNLP, NEWUOA, and PSWARM. The study finds that the quality of solutions decreases with increasing problem size, and certain solvers like TOMLAB/MULTI-MIN, TOMLAB/GLCCLUSTER, MCS, and TOMLAB/LGO perform better in terms of solution quality within 2,500 function evaluations. Additionally, TOMLAB/OQNLP, NEWUOA, and TOMLAB/MULTIMIN excel in refining near-optimal solutions.