This study evaluates the validity of altmetrics as indicators of article impact and utility by comparing 11 altmetrics with Web of Science citations for 76 to 208,739 PubMed articles. The research finds statistically significant associations between higher altmetric scores and higher citations for articles with positive altmetric scores, except for Google+ posts. However, the evidence is insufficient for LinkedIn, Pinterest, question and answer sites, and Reddit. The study also introduces a sign test to account for biases caused by different citation and usage windows. The results suggest that altmetrics like tweets, Facebook wall posts, research highlights, blogs, mainstream media mentions, and forum posts are associated with citation counts, particularly in medical and biological sciences. However, the magnitude of the correlation is unknown. The coverage of altmetrics, except for Twitter, is low, and the study highlights the need to consider the time from publication when using altmetrics to rank articles. The study also notes that altmetrics may capture a broader aspect of research visibility and impact compared to citations, which only reflect the behavior of publishing authors. The findings suggest that altmetrics can be useful for identifying exceptional or above-average articles but may not be reliable for general use. The study concludes that while altmetrics show some association with citations, their validity as indicators of article impact remains uncertain.This study evaluates the validity of altmetrics as indicators of article impact and utility by comparing 11 altmetrics with Web of Science citations for 76 to 208,739 PubMed articles. The research finds statistically significant associations between higher altmetric scores and higher citations for articles with positive altmetric scores, except for Google+ posts. However, the evidence is insufficient for LinkedIn, Pinterest, question and answer sites, and Reddit. The study also introduces a sign test to account for biases caused by different citation and usage windows. The results suggest that altmetrics like tweets, Facebook wall posts, research highlights, blogs, mainstream media mentions, and forum posts are associated with citation counts, particularly in medical and biological sciences. However, the magnitude of the correlation is unknown. The coverage of altmetrics, except for Twitter, is low, and the study highlights the need to consider the time from publication when using altmetrics to rank articles. The study also notes that altmetrics may capture a broader aspect of research visibility and impact compared to citations, which only reflect the behavior of publishing authors. The findings suggest that altmetrics can be useful for identifying exceptional or above-average articles but may not be reliable for general use. The study concludes that while altmetrics show some association with citations, their validity as indicators of article impact remains uncertain.