FDA Releases Action Plan for Artificial Intelligence/Machine Learning-Enabled Software as a Medical Device

Key Points

• FDA will publish draft guidance on its Predetermined Change Control Plan in 2021 and host a public workshop on transparency requirements around AI/ML-based devices.
• FDA plans to pilot Real-World Performance Monitoring of AI/ML-based SaMD and establish Good Machine Learning Practices (GMLPs).
• We expect FDA’s new approach, as applied to SaMD, ultimately to be extended to AI/ML for software embedded in a physical medical device (Software in a Medical Device, or SiMD).

  ---

After much anticipation, on January 12, 2021, the U.S. Food and Drug Administration (FDA) released its first “Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan” (“Action Plan”). The Action Plan builds on, and addresses stakeholder feedback from, a proposed AI/ML regulatory framework published in 2019. While this Action Plan sets in motion a series of concrete steps to build the fundamental regulatory structure for this technology, it remains only an early step towards the agency’s vision of a new regulatory approach to AI/ML.

The particular regulatory question the agency is contending with is rather discrete and quite technical: for a medical device that is stand-alone software developed through AI/ML, what type of validation is required to support the software’s safety and effectiveness, particularly as it is subject to modification and adaptation? Ultimately, FDA, and likely Congress, will need to grapple with how the agency can obtain an appropriate assurance of safety and effectiveness for a method of development that is so complex and stretches the agency’s traditional methods of review. This new approach, as applied to SaMD, will then need to be extended to the growing number of applications of AI/ML for SiMD embedded within a physical medical device. Many of the concepts that FDA is evaluating in this Action Plan as means of helping to assure safety and effectiveness for AI/ML also have resonance for other advanced and fast evolving technologies other than software.

Background

In April 2019, FDA published the “Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) - Discussion Paper and Request for Feedback.” The discussion paper described a potential framework for premarket review of AI/ML-driven modifications to SaMD. This framework centers on a contemplated Predetermined Change Control Plan comprised of two elements: (1) Software as a Medical Device Pre-Specifications (SaMD SPS) and (2) an Algorithm Change Protocol (ACP). The first element, SaMD SPS, identifies the types of modifications the software will undergo. The second element, the ACP, refers to how the sponsor will assess and control the potential risks relating to the modifications made by the algorithm via AI/ML.

In January, FDA published the Action Plan based on feedback to the discussion paper and in recognition of the public health need to facilitate and oversee AI/ML-based medical device software. The Action Plan sets forth five next steps, discussed in more detail below, that the agency plans to take as AI/ML-based SaMD evolves:

• Continuing to develop its own proposed regulatory framework for draft guidance on a predetermined change control plan for software learning.
• Supporting GMLPs for evaluation of ML algorithms.
• Enabling a more transparent patient-centered approach.
• Developing new methods to evaluate and improve ML algorithms.
• Creating new pilots to enable real-world performance (RWP) monitoring.

FDA Will Publish Draft Guidance on the Predetermined Change Control Plan in 2021

FDA notes that it received general support for its Predetermined Change Control Plan model. A change control plan would allow manufacturers to specify in advance certain modifications to the AI/ML-based SaMD as it learns and changes over time. Based on “strong community interest,” FDA plans to publish draft guidance in 2021. The draft guidance will include discussion of how to ensure the SaMD algorithms are safe over time, a concern that many commenters raised, and will detail which elements to include in the change control plan pre-specifications and ACP to ensure continued safety and effectiveness. FDA will also focus on additional areas of interest to stakeholders, such as refining the appropriate types of modifications under the framework and narrowing in on specifics for “focused review” of the Predetermined Change Control Plan.

FDA Will Hold a Public Workshop on How Device Labeling Supports Transparency and Enhances Trust in AI/ML-based Devices

Many comments to the discussion paper described the challenge of labeling and the need for FDA to clarify transparency requirements around AI/ML-based devices. In particular, stakeholders expressed the need for a manufacturer to describe “data that w[as] used to train an algorithm, the relevance of its inputs, the logic it employs (when possible), the role intended to be served by its output, and the evidence of the device’s performance.”

To safeguard transparency in AI/ML software, in October 2020, FDA held a Patient Engagement Advisory Committee (PEAC) devoted to AI/ML to gather information about what factors impact patient trust in these technologies. FDA plans to hold a public workshop to share what it learned and elicit broad input on how device labeling increases transparency, and will consider input in making decisions about information to include in labeling requirements.

FDA Will Support Piloting of Real-World Performance Monitoring of AI/ML-based SaMD

In the discussion paper, the agency noted that in order to oversee AI/ML-based SaMD across its total product lifecycle, it would be necessary to collect and monitor real-world data. Real-world data collection and monitoring allows manufacturers to understand how products are used, identify areas of improvement and respond to safety and usability concerns, thereby mitigating risk. Comments to the paper sought more specificity and direction in the area of RWP monitoring.

As part of the Action Plan, FDA plans to support the piloting of RWP monitoring on a voluntary basis, in collaboration with other FDA programs focused on real-world data. FDA foresees that evaluations performed during RWP monitoring will allow for the development of thresholds and performance evaluations for AI/ML-based SaMD, particularly in regards to safety and usability. In the future, we expect a legislative ask to support a pre-certification concept for AI/ML, similar to the voluntary program piloted for certain digital health software. Assuming it would resemble the current pre-certification pilot, such a program would be intended to leverage real-world performance to provide streamlined regulatory oversight of AI/ML-based SaMD.

FDA Will Harmonize GMLPs Development

In the discussion paper, FDA used the term “Good Machine Learning Practices” to describe a list of AI/ML best practices, including data management, feature extraction, training, interpretability, evaluation and documentation. In comments, stakeholders supported the development of best practices and requested that FDA work with other groups to harmonize the efforts to develop GMLPs. FDA has participated in a number of domestic and international working groups, associations and consortia, and the agency intends to continue to deepen these relationships while ensuring that the FDA Medical Device Cybersecurity Program is part of these collaborations. We expect FDA to continue to flesh out the framework and ultimately develop industry standards, based on third party recognized standards, to support GMLPs, which under current law would need to be established as an aspect of existing Quality Systems requirements under 21 C.F.R. Part 820.

FDA Will Continue to Support Scientific Efforts to Evaluate and Address Algorithmic Bias and Improve Algorithmic Robustness

FDA recognizes that AI/ML systems, which are developed and trained from historical datasets, are very vulnerable to bias. Factors such as race, ethnicity and socioeconomic status impact health care delivery and may therefore be reflected in algorithms. As such, FDA is supporting scientific efforts to develop ways to evaluate AI/ML-based medical software across the country, including at FDA Centers for Excellence in Regulatory Science and Innovation (CERSIs) at the University of California San Francisco, Stanford University and Johns Hopkins University. FDA intends to continue to develop and expand these efforts.

Takeaways and Next Steps

Although the Action Plan proposes many comprehensive actions to develop the regulatory framework for AI/ML, it is largely a roadmap to future policy changes that will be articulated through multiple guidances. It remains to be seen to what degree this Action Plan can continue to be implemented in a SaMD silo. An increasing array of SiMD also relies on AI/ML, and the regulatory requirements for software updates to these devices would also benefit from clarity.

FDA’s plan to issue draft guidances suggests the agency has determined it has sufficient authority to implement the core elements of its Action Plan, such as predetermined change protocol plans. Indeed, it has included change protocol concepts in certain device clearances and approvals already. At the same time, however, FDA has informally indicated a desire for additional statutory authority to supports its overall regulatory approach for AI/ML. The Verifying Accurate and Leading-edge IVCT Development (VALID) Act legislation, which would revamp the regulation of in vitro clinical tests, including many types of diagnostic software, features similar concepts of approved change protocols and technology-based approvals. If enacted, this framework would potentially pull a significant portion of SaMD—that is used for diagnostic purposes—into a new statutorily authorized framework.

FDA remains interested in feedback as it works to implement the Action Plan and will continue to engage with industry stakeholders and agency actors in a harmonious approach. The public docket established for the Proposed Regulatory Framework for Modifications to AI/ML remains open (FDA-2019-N-1185) for comment.

Contact Information

If you have any questions concerning this alert, please contact: