On September 26, 2024, FDA published a compilation of commonly used terms in the digital health, artificial intelligence (AI), and machine learning space and their definitions, available here.1 FDA intends the glossary to be used for general education purposes. The definitions are either directly from, or adapted from, various public sources, including consensus standard organizations and published literature, and FDA includes references to those sources in the glossary.
FDA publishes this glossary amid growing discussion of a need for a set of national standards applicable to AI. The agency has already made notable progress in adapting its regulatory framework for AI’s novel attributes. As part of the glossary, the agency defines one of issues that continues to concern FDA: data drift. FDA defines “data drift” as “the change in the input data distribution a deployed model receives over time, which can cause the model’s performance to degrade. This occurs when the properties of the underlying data change. Data drift can affect the accuracy and reliability of predictive models.” FDA explains that AI-enabled medical products can experience data drift due to statistical differences between the data used for model development and data used in clinical operation due to variations between medical practices or context of use between training and clinical use, and changes in patient demographics, disease trends and data collection methods over time.
One way to avoid data drift is to conduct what FDA terms “AI performance monitoring.” FDA defines AI performance monitoring as the process of regularly collecting and analyzing data on the use of a deployed AI system to evaluate its performance in accomplishing its intended tasks in real-world settings.” Although this concept of AI performance monitoring is already incorporated into the total lifecycle management for AI-enabled medical devices, FDA has indicated that the agency needs expanded authority to adequately regulate the use of AI, in particular for AI-enabled medical devices.2
FDA plans routine updates to the glossary. FDA welcomes stakeholders to submit feedback by emailing digitalhealth@fda.hhs.gov.
1 FDA, Digital Health and Artificial Intelligence Glossary – Educational Resource (Sept. 26, 2024), https://www.fda.gov/science-research/artificial-intelligence-and-medical-products/fda-digital-health-and-artificial-intelligence-glossary-educational-resource.
2 Government Accountability Office (GAO), Federal Regulation: Selected Emerging Technologies Highlight the Need for Legislative Analysis and Enhanced Coordination, GAO-24-106122 (Jan. 25, 2024), https://www.gao.gov/products/gao-24-106122.