Artificial intelligence (AI) has the potential to significantly improve patient safety in clinical care but requires careful implementation to avoid harm. Examples include detecting sepsis, pressure ulcers, and adverse drug events. However, poorly designed AI systems can lead to missed diagnoses and delayed treatments, as seen in a study where an AI system only identified 7% of sepsis cases. To mitigate risks, patient safety safeguards must be developed. The Biden administration has issued an executive order to establish an AI safety program, including frameworks for identifying AI-related errors, data analysis for improvement, and guidelines for healthcare organizations. Three key recommendations are proposed: (1) develop guidelines for safe AI implementation, drawing from human-systems integration principles used in aviation and defense; (2) frequently monitor AI systems for safety risks through testing and red-teaming; and (3) ensure traceability of AI contributions to patient safety events by capturing metadata. The Office of the National Coordinator for Health Information Technology (ONC) has developed EHR implementation guides, which can serve as a model for AI guidelines. The FDA should also consider guidelines for AI-enabled medical devices. Healthcare organizations should establish governance structures to manage AI tools, monitor safety issues, and develop solutions. The Joint Commission should promote accreditation standards for AI safety monitoring. Metadata should be standardized to enable traceability and address AI bias. A public-private effort involving federal agencies, industry, and stakeholders is needed to standardize metadata capture. AI safety efforts should align with existing frameworks from the FDA, NIST, and other agencies. Federal and state programs may be necessary to support AI safety initiatives, especially for organizations lacking resources. Some recommendations can be implemented within 365 days, while others will require more time. The executive order provides a foundation for rapid development of patient-focused safeguards.Artificial intelligence (AI) has the potential to significantly improve patient safety in clinical care but requires careful implementation to avoid harm. Examples include detecting sepsis, pressure ulcers, and adverse drug events. However, poorly designed AI systems can lead to missed diagnoses and delayed treatments, as seen in a study where an AI system only identified 7% of sepsis cases. To mitigate risks, patient safety safeguards must be developed. The Biden administration has issued an executive order to establish an AI safety program, including frameworks for identifying AI-related errors, data analysis for improvement, and guidelines for healthcare organizations. Three key recommendations are proposed: (1) develop guidelines for safe AI implementation, drawing from human-systems integration principles used in aviation and defense; (2) frequently monitor AI systems for safety risks through testing and red-teaming; and (3) ensure traceability of AI contributions to patient safety events by capturing metadata. The Office of the National Coordinator for Health Information Technology (ONC) has developed EHR implementation guides, which can serve as a model for AI guidelines. The FDA should also consider guidelines for AI-enabled medical devices. Healthcare organizations should establish governance structures to manage AI tools, monitor safety issues, and develop solutions. The Joint Commission should promote accreditation standards for AI safety monitoring. Metadata should be standardized to enable traceability and address AI bias. A public-private effort involving federal agencies, industry, and stakeholders is needed to standardize metadata capture. AI safety efforts should align with existing frameworks from the FDA, NIST, and other agencies. Federal and state programs may be necessary to support AI safety initiatives, especially for organizations lacking resources. Some recommendations can be implemented within 365 days, while others will require more time. The executive order provides a foundation for rapid development of patient-focused safeguards.