FDA Strengthens AI Regulation for Enhanced Patient Safety

A Special Communication published in the Journal of the American Medical Association (JAMA) explored how the US Food and Drug Administration (FDA) regulates artificial intelligence (AI) in healthcare, emphasizing the critical role of patient safety. The investigation highlighted AI’s potential in clinical research, medical product development, and patient care, and underscored key areas that need attention as regulations evolve to address the unique challenges AI presents in biomedicine and healthcare.

Background

AI advancements have the potential to completely revolutionize biomedicine and healthcare. Expectations for AI frequently exceed those for prior medical technologies such as telemedicine, digital health tools, and electronic health records. While many of these technologies were groundbreaking, the ability of AI tools to analyze data, diagnose problems, and provide individualized care is revolutionary.

However, the application of AI in medicine and healthcare poses serious questions about supervision and regulation. The US FDA has long been developing standards for the use of AI in the creation of medical products and healthcare. However, the dynamic nature of AI technology creates certain unique regulatory issues, particularly in terms of effectiveness, safety, post-market performance, and accountability. Furthermore, the rapid evolution of AI technology necessitates adaptable regulatory frameworks.

FDA rules for artificial intelligence in medicine
According to the analysis, FDA oversight of AI-enabled medical goods began in 1995, with the approval of PAPNET, an AI-based tool used by pathologists to diagnose cervical cancer. Although PAPNET was not extensively used due to its exorbitant cost, the FDA has subsequently approved nearly 1,000 AI-based medical equipment and technologies, primarily for radiology and cardiology.

AI is also utilized extensively in drug research, including drug discovery, clinical trials, and dosage optimization. Furthermore, while AI-based applications have grown increasingly popular in cancer, there is growing interest in applying AI to mental health, where digital technologies have the potential to make a big difference.

The number of regulatory submissions for the use of AI in drug development received by the FDA has increased tenfold in one year, and given AI’s wide range of applications and complexities, the FDA has also adapted the regulatory framework to be risk-based while also taking into account AI’s evolution in real-world clinical settings.

The FDA’s five-point plan of action for regulating machine learning and AI-based medical devices, proposed in 2021, aims to stimulate innovation while assuring the products’ effectiveness and safety. This plan of action is also consistent with Congressional advice, which wants the FDA to adopt laws that are flexible enough to allow developers to improve AI technologies without constantly seeking FDA approval.

However, the author emphasizes that these rules must take into account the necessity to manage AI products throughout their entire life cycle, particularly by ongoing monitoring of their performance after deployment in clinical contexts.

The FDA’s medical products center has also established four areas of priority for AI development: improving public health safety, supporting regulatory innovation, promoting best practices and harmonized standards, and furthering research into AI performance evaluation.

Key Concepts for FDA Regulation of AI
The FDA intends to shape the regulation of AI-enabled medical products based on both US legislation and worldwide norms. Collaborations with organizations like the International Medical Device Regulators Forum enable the FDA to promote global AI standards, such as controlling AI’s role in drug development and upgrading clinical trials, through international cooperation.

With the rapid advancement of AI technology, one of the FDA’s primary difficulties is successfully processing huge amounts of AI submissions while ensuring that innovation and safety are not hampered. Furthermore, continual postmarket supervision of AI systems is required to verify that they work as intended throughout time, particularly in diverse and changing clinical situations. This necessitates a flexible, science-based regulatory structure, such as the Software Precertification Pilot, which enables ongoing evaluation of AI technologies.

The risk-based approach to regulating AI-enabled medical devices also provides for adaptable approaches to a variety of AI models. Simple AI models employed for administrative purposes, for example, are less regulated, but complicated AI models implanted in cardiac defibrillators face tougher rules.

Another example cited by the reviewers was Sepsis ImmunoScore, an AI-based tool for sepsis identification that was designated as a Class II device and required extra safety precautions to account for potential bias or algorithm failure concerns.

The assessment underlines the need for specialized regulatory tools to examine the increasing number of AI models, such as generative AI and huge language models. This is especially crucial given the hazards posed by unanticipated outputs, such as inaccurate diagnoses, which will require extensive testing both before and after implementation in clinical procedures.

Conclusions
To summarize, the review found that flexible regulatory approaches and coordinated efforts across industries, international organizations, and governments, as well as strict FDA regulation, are critical for keeping up with AI technology’s rapid advancement in medicine and ensuring the efficacy and safety of AI tools.

The authors propose that extensive postmarket monitoring and the complete life cycle of AI technologies are required to ensure their continued safety and effectiveness in clinical practice. They argue that a focus on the health outcomes of patients rather than financial optimization should govern the incorporation of AI into healthcare. They also warn that balancing innovation and patient care must remain a priority in order to keep AI from being driven solely by financial incentives.

For more information: FDA Perspective on the Regulation of Artificial Intelligence in Health Care and Biomedicine, JAMA Network, doi:10.1001/jama.2024.21451