HL7 Informative Document: AI/ML Data Lifecycle, Edition 1 - US Realm

DESCRIPTION

The AI Data Lifecycle (Phase I: Data Used for Machine Learning Models), Edition 1, provides an Informative Document to developers to promote the use of standards to improve the trust and quality of interoperable data used in AI models. Standards are needed for the development and implementation of AI systems in healthcare to ensure that the data used to train and receive output from these systems are of consistently high quality, interoperable (uses data that involve standard terminologies such as (e.g. FHIR, SNOMED, CPT), transparent, and ethically sound, and used for the purpose intended ( e.g. "answers the question").

ALTERNATIVE NAMES

HL7 Informative Document: AI/ML Data Lifecycle, Edition 1 - US Realm may also go by the following names or acronyms:

BENEFITS

Promote the use of standards that improve the trust and quality of interoperable data used in AI models. Standards are needed for the development and implementation of AI systems in healthcare to ensure that the data used to train and receive output from these systems are of consistently high quality, interoperable (uses data that involve standard terminologies such as (e.g. FHIR, SNOMED, CPT), transparent, and ethically sound, and used for the purpose intended ( e.g. "answers the question"). The rationale for needing standards regarding these attributes is expanded below: (br)

1. Ensures Consistency that data are collected, annotated, and processed in a consistent and standardized manner. This consistency is essential for training machine learning models, as it ensures that the data is of high quality and can be used to produce reliable and consistent results.
2. Enables Interoperability between different systems and technologies, allowing different AI systems to share data and work together seamlessly. This is particularly important in healthcare, where different organizations and systems often need to share patient data to provide the best possible care.
3. Promotes Transparency in the development and implementation of AI systems by ensuring that data collection and processing methods are well-defined and clearly documented. This transparency is essential for building trust in AI systems, particularly in healthcare where the stakes are high, and the consequences of errors can be severe.
4. Ensures ethical considerations in AI systems are developed and implemented in an ethical and responsible manner. For example, standards can help prevent bias in training data sets by ensuring that data is collected from diverse populations and that bias is actively mitigated during the development process. Within the construct of these rationales, the Life Cycle of Data used in AI for healthcare involves several critical stages, each of which requires specific standards.

IMPLEMENTATIONS/CASE STUDIES

Twelve (12) Case Studies are provided in a supplementary document for the first Specification. See: HL7_AI_DATA_LIFECYCLE_E1_I1_2024JAN_Supporting_Publication_Document_Case_Studies.docx.

These case studies were developed by clinicians who participated in the initial project team during the period February 2023 through July 2024, coordinated during over 30 meetings.

One Case Study references recent regulatory guidance promulgated by the US Department of Health and Humans Services (HHS) Office of the National Coordinator (ONC), specifically a set of Certifying Requirements for AI Decision Support Interventions (DSI) and Predictive Models (Algorithmic Transparency)

DEVELOPMENT BACKGROUND

AI Data Lifecycle is the initial priority and project for the HL7 AI Focus Team under the aegis of the HL7 Electronic Health Record (EHR) WG. It is the first such project in HL7 to address use of AI in healthcare systems.

Project leadership and resources for the initial Specification (AI Data Lifecycle (Phase I: Data Used for Machine Learning Models), Edition 1) were provided by the US Department of Veterans Affairs, supported by volunteers with expertise in clinical healthcare, clinical informatics, AI, and software development.

The first Specification provides Informative Guidance for developers of predictive ML models in healthcare, and lists standards, guidelines, regulations and best practices to enhance quality and trust in predictive AI by emphasizing data quality and interoperability used in training ML models. This first product is a White Paper (Informative Guidance) for Developers of Machine Learning Models in Predictive AI, approximately 40 pages long plus Appendices and supporting documents such as Case Studies.

RELATED DOCUMENTS

HL7 Informative Document: AI/ML Data Lifecycle, Edition 1 - US Realm

(Download)

ADDITIONAL DETAILS

Artificial Intelligence (AI), to include Machine Learning (ML), depends on data quality. This project considers how to capture, render and share the attributes of provenance, accountability [e.g. audit trails], trustworthiness, context, structure, patterns, annotation and annotation history at each step in the life cycle of the data. The goals are to provide implementation guidance for AI/ML projects to leverage discoverable patterns and annotations provided by standards-based interoperable datasets, provide a roadmap for AI/ML experts to take advantage of interoperability standards to combine data from multiple, disparate data sources; and, in turn, articulate the return on investment of using interoperable, HL7-conformant data sets to create AI/ML solutions that are trusted by clinicians.

TOPICS

• Care Provision
• Clinical Quality
• Decision Support
• Medical Records
• Patient Care
• Public Health

STATUS DATE

2024-08-27

RESPONSIBLE WORK GROUP

PRODUCT TYPES

• Document
• Implementation Guide
• White Paper

STAKEHOLDERS

• Clinical and Public Health Laboratories
• Clinical Decision Support Systems Vendors
• EHR, PHR Vendors
• Emergency Services Providers
• Equipment Vendors
• Health Care IT Vendors
• Healthcare Institutions
• HIS Vendors
• Immunization Registries
• Lab Vendors
• Local and State Departments of Health
• Payors
• Pharmaceutical Vendors
• Quality Reporting Agencies
• Regulatory Agency
• Standards Development Organizations (SDOs)