This paper presents an extensive comparison of Static Application Security Testing (SAST) tools, aiming to evaluate their effectiveness in identifying software vulnerabilities. The study uses a controlled, synthetic Java codebase with 1.5 million test executions to assess the performance of eight SAST tools. The evaluation focuses on two main research questions: (1) the extent of SAST tool coverage in identifying Common Weakness Enumeration (CWE) vulnerabilities, and (2) the accuracy of SAST tools in detecting these vulnerabilities. The results show that SAST tools have limited coverage of CWEs, with most tools failing to identify a significant portion of vulnerabilities. Despite this, SAST tools exhibit high precision, meaning they rarely produce false positives, but they suffer from low recall, indicating a high number of false negatives. This suggests that while SAST tools are effective at identifying known vulnerabilities, they are not as effective at detecting new or less common ones. The study also highlights the importance of using multiple SAST tools to cover a broader range of vulnerabilities. Additionally, it recommends that future improvements to SAST tools should focus on increasing recall, as this is a critical factor in effective vulnerability detection. The study also notes that the results should be interpreted with caution due to potential biases in the test data and the representativeness of the CWEs used. Overall, the findings suggest that while SAST tools are valuable for identifying certain types of vulnerabilities, they have limitations in detecting others. The study provides recommendations for improving the effectiveness of SAST tools and highlights the need for further research into alternative approaches to vulnerability detection, such as machine learning-based solutions.This paper presents an extensive comparison of Static Application Security Testing (SAST) tools, aiming to evaluate their effectiveness in identifying software vulnerabilities. The study uses a controlled, synthetic Java codebase with 1.5 million test executions to assess the performance of eight SAST tools. The evaluation focuses on two main research questions: (1) the extent of SAST tool coverage in identifying Common Weakness Enumeration (CWE) vulnerabilities, and (2) the accuracy of SAST tools in detecting these vulnerabilities. The results show that SAST tools have limited coverage of CWEs, with most tools failing to identify a significant portion of vulnerabilities. Despite this, SAST tools exhibit high precision, meaning they rarely produce false positives, but they suffer from low recall, indicating a high number of false negatives. This suggests that while SAST tools are effective at identifying known vulnerabilities, they are not as effective at detecting new or less common ones. The study also highlights the importance of using multiple SAST tools to cover a broader range of vulnerabilities. Additionally, it recommends that future improvements to SAST tools should focus on increasing recall, as this is a critical factor in effective vulnerability detection. The study also notes that the results should be interpreted with caution due to potential biases in the test data and the representativeness of the CWEs used. Overall, the findings suggest that while SAST tools are valuable for identifying certain types of vulnerabilities, they have limitations in detecting others. The study provides recommendations for improving the effectiveness of SAST tools and highlights the need for further research into alternative approaches to vulnerability detection, such as machine learning-based solutions.