This scoping review evaluates randomized controlled trials (RCTs) of artificial intelligence (AI) in clinical practice, focusing on the USA and China, which lead in the number of trials. The review highlights a growing interest in AI across various medical specialties, particularly in gastroenterology and radiology, with a majority of trials reporting positive primary endpoints related to diagnostic yield or performance. However, concerns arise from the predominance of single-center trials, limited demographic reporting, and varying operational efficiency reports, raising questions about the generalizability and practicality of the results. Despite promising outcomes, the review emphasizes the need for more comprehensive research, including multicenter trials, diverse outcome measures, and improved reporting standards. Future AI trials should prioritize patient-relevant outcomes to fully understand AI's true effects and limitations in healthcare. The review also notes the importance of addressing publication bias and the need for more international collaboration to ensure the generalizability of AI systems across different populations and healthcare systems.This scoping review evaluates randomized controlled trials (RCTs) of artificial intelligence (AI) in clinical practice, focusing on the USA and China, which lead in the number of trials. The review highlights a growing interest in AI across various medical specialties, particularly in gastroenterology and radiology, with a majority of trials reporting positive primary endpoints related to diagnostic yield or performance. However, concerns arise from the predominance of single-center trials, limited demographic reporting, and varying operational efficiency reports, raising questions about the generalizability and practicality of the results. Despite promising outcomes, the review emphasizes the need for more comprehensive research, including multicenter trials, diverse outcome measures, and improved reporting standards. Future AI trials should prioritize patient-relevant outcomes to fully understand AI's true effects and limitations in healthcare. The review also notes the importance of addressing publication bias and the need for more international collaboration to ensure the generalizability of AI systems across different populations and healthcare systems.