AI Executive Order: Healthcare Industry Implications

This client alert is one in a series of alerts on the various aspects of the executive order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence that was signed by President Biden on October 30, 2023.

The Biden administration issued its widely anticipated Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (EO) on October 30, 2023, which sets forth several directives aimed at the responsible use and development of AI in the healthcare industry. The portions of the EO that implicate the healthcare sector focus on three areas of concern: protecting patients, encouraging innovation, and working closely with state and local governments to implement policies.

Protecting Patients

The range of tasks that the U.S. Department of Health and Human Services (HHS) and collaborating agencies will be undertaking for patient protection foreshadow potentially broad regulation of AI-enabled healthcare technologies, covering both pre- and post-market disclosures and monitoring, incorporation of safety, privacy, and security standards into the technology, and constant, ongoing assessment of technologies by HHS.

HHS AI Task Force

HHS, in collaboration with the Departments of Defense and Veteran’s Affairs and the nation’s largest healthcare systems, is charged with establishing an HHS AI Task Force within 90 days of the EO. In the year following its creation, the Task Force will develop policies, frameworks, and regulatory actions to address the following issues:

• Development, maintenance, and use of AI-enabled technologies in healthcare delivery and financing, including quality measurement, performance improvement, program integrity, benefits administration, and patient experience.
• Long-term safety and performance monitoring of AI-enabled technologies, including a means to communicate product updates to regulators, developers, and users.
• Incorporation of safety, privacy, and security standards into the software development life cycle (see discussion below on privacy and security).
• Identification of uses of AI to promote workplace efficiency and reduce administrative burden in the healthcare sector.
• Development, maintenance, and availability of documentation to help users determine appropriate and safe use of AI in local settings.
• Incorporation of equity principles in AI-enabled technologies (see discussion below on equity).

Quality Assessment

The EO requires HHS to develop a strategy to determine whether AI-enabled technologies maintain appropriate levels of quality. The strategy will include development of an “AI assurance policy” to evaluate the performance of healthcare tools and the infrastructure needs for enabling pre-market assessment and post-market oversight of AI-enabled healthcare technology performance against real world data.

Safety

The EO also orders HHS (in consultation with the Departments of Defense and Veterans Affairs) to establish a safety program that includes:

• A framework to identify and capture clinical errors resulting from AI deployed in healthcare settings;
• Specifications for a central tracking repository for incidents that cause harm;
• Analysis of captured data to develop best practices and informal guidelines to avoid identified harms; and
• Dissemination of the guidelines to stakeholders, including healthcare providers.

Drug Development

HHS has one year from the date of the EO to develop a strategy to regulate the use of AI in drug development processes. At a minimum, the strategy will:

• Define the objectives, goals, and high-level principles required for appropriate regulation in each drug development phase;
• Identify areas where future rulemaking, guidance, or additional statutory authority may be necessary;
• Identify the existing budget, resources, personnel, and potential for new public/private partnerships necessary for such a regulatory system; and
• Consider risks identified by any such regulatory actions.

HHS is not required to publish this strategy, but we expect that it will build on a discussion paper issued by the U.S. Food and Drug Administration (FDA) earlier this year.

Health Equity

The EO notes AI’s potential to exacerbate discrimination and bias in healthcare. HHS is tasked with considering appropriate actions to advance compliance with federal nondiscrimination laws by health and human services providers that receive federal funding, as well as how those laws relate to AI. Specifically, this may include HHS providing technical assistance to health and human services providers and payers about their obligations under federal nondiscrimination and privacy laws that relate to AI and issuing guidance or taking appropriate action in response to any complaints or reports of noncompliance with federal nondiscrimination and privacy laws that relate to AI.

Eliminating and mitigating bias in health systems is also a notable piece of the regulatory framework that HHS has been instructed to develop. In establishing policies and regulations for the healthcare sector, HHS is expected to develop policies that promote:

• Incorporation of equity principles in AI technologies;
• Use of disaggregated data and representative population data sets when developing new models;
• Monitoring performance against discrimination and bias in existing models; and
• Identifying and mitigating discrimination and bias in current systems.

Privacy and Data Security

The EO generally emphasizes the need for AI to be safe and secure, addressing AI systems’ most pressing security risks, such as biotechnology, cybersecurity, critical infrastructure, and other national security dangers. Consistent with this, HHS is tasked with incorporating safety, privacy, and security standards into the software-development life cycle to protect personally identifiable information, including measures to address AI-enhanced cybersecurity threats in the health and human services sector.

In addition to measures that will follow specifically from HHS’s directive, the EO also directs the secretary of commerce, through the Director of the National Institute of Standards and Technology (NIST) and in coordination with the Departments of Homeland Security and Energy, to develop guidelines and best practices for developing, deploying, and red-team testing AI systems to be safe, secure, and trustworthy.

Encouraging Innovation

Encouraging the development of AI-enabled technology in the healthcare sector is prominently spotlighted by the EO. The EO explicitly instructs HHS to advance healthcare technology developers’ responsible AI innovations by prioritizing grantmaking and other awards to support responsible AI development and use. HHS will, at a minimum, collaborate with the private sector through HHS programs that may support the advancement of AI tools that develop personalized immune-response profiles for patients, prioritize the allocation of certain funds to initiatives that explore ways to improve healthcare data quality, and accelerate certain grants awarded through the National Institutes of Health.

State and Federal Coordination

The EO is clear in its intent to flow down federal guidance to state and local entities. Within 180 days of the EO, HHS shall publish a plan addressing the use of automated or algorithmic systems in the implementation by states and localities of public benefits and services administered by HHS. The plan will promote assessment of recipients’ access to benefits, require notice to recipients about the presence of such systems, use regular evaluations to detect unjust denials, and add processes to retain appropriate discretion of expert (human) staff, processes to appeal denials to human reviewers, and analysis of whether algorithmic systems used by benefit programs achieve equitable and just outcomes.

Additionally, as part of the work of the HHS AI Task Force, HHS is instructed to work with state, local, tribal, and territorial agencies to advance best practices for use of AI.

The EO also notes that the federal government will enforce existing consumer protection laws and principles and enact appropriate safeguards against fraud, unintended bias, discrimination, infringements on privacy, and other harms from AI and emphasizes that such protections are important in critical fields like healthcare where mistakes by or misuse of AI could harm patients, cost consumers, and jeopardize safety and rights.

Key Take Aways

The EO applies a broad-brush approach to addressing the sometimes conflicting goals of protecting individuals and patients from the potential harms of AI technology in healthcare, and encouraging development and innovation, opening the door to comprehensive regulation that could impose onerous disclosure and monitoring requirements on healthcare technology developers, at all stages of the development process.

Notably, while the EO often acknowledges that HHS will need to collaborate with appropriate agencies in its work, it does not reference other agencies’ established guidelines that will be implicated by HHS’s actions. For example, the FDA is charged with reviewing AI- and machine learning-enabled medical devices and issued several white papers and most recently guidance on predetermined change control plans to address the special issues that arise in medical device premarket notifications and applications that use AI/ML. Earlier this month, the FDA announced the creation of its Digital Health Advisory Committee to explore scientific and technical issues related to digital health technologies that include AI. Additionally, the Office of the National Coordinator for Health IT (ONC) released a proposed rule in April 2023 that would create technical transparency and risk-management requirements for a wide range of healthcare software systems including generative AI.

We will be closely following how HHS navigates the developing patchwork of AI-related federal guidance to create a comprehensive policy for the healthcare industry, as directed by the EO, within a relatively short time frame, and the broader impacts of the EO on the healthcare industry.