Our most recent blog featured Dr. Reid Blackman discussing the main ethical issues associated with AI development, including its use in research, and ways to mitigate these. Because of the outstanding response we had to this blog, we are following it up with a curated list of resources on AI research ethics. Our list includes background material, relevant US and international regulations, policies and guidance, tools for IRB organizations working on AI ethics, and select peer-reviewed articles on AI Ethics.

The use of AI in human subjects research falls into two main categories:

– Studies that make use of an established AI model or tool. In this case, the characteristics of the model are well-established, including possible risks and limitations, and these should be described in an IRB application, in consent documents, and in other materials for potential participants.

– Studies to develop or evaluate a new AI tool. This scenario involves research to create or evaluate a new AI model; for example, the development of an algorithm to predict the likelihood of progression to type II diabetes in adults. In this case, the investigators must consider and be able to describe relevant ethical issues related to the development and deployment of their models, including the choice of training data. This scenario is the one for which the resources in this blog are most applicable.

General AI Background:

McCarthy, J. (2007). What Is Artificial Intelligence? Stanford University, Stanford, CA. Retrieved from http://jmc.stanford.edu/articles/whatisai/whatisai.pdf

Bryson, J. J. (2019). The Past Decade and Future of AI’s Impact on Society. In Towards a New Enlightenment? A Transcendent Decade (Vol. 11). Turner. https://www.bbvaopenmind.com/wp-content/uploads/2019/02/BBVA-OpenMind-Joanna-J-Bryson-The-Past-Decade-and-Future-of-AI-Impact-on-Society.pdf

Keans, M., Roth, A. (2020) The Ethical Algorithm: The Science of Socially Aware Algorithm Design. Oxford University Press

Leading AI ethicist, Dr. Reid Blackman, has articles and other useful resources on his website.

Janelle Shane on The Realities of Artificial Intelligence (online course). This course is based on her book “You Look Like a Thing and I Love You: How Artificial Intelligence Works and Why It’s Making the World a Weirder Place” and her blog AI Weirdness and is included in the free soft skills library for all Learn eCORE subscribers.

U.S. Regulations and Associated Guidance:

Studies that meet the regulatory definition of human subjects sesearch, must meet the applicable regulations in the federal human subjects protection regulations (the Common Rule and its associated subparts); those that are clinical investigators or otherwise fall under the purview of the FDA, must meet the FDA regulations.

– Federal Human Subjects Research Protection Regulations – While the Common Rule does not specifically address AI, the relevant requirements for approval of research must be met, such as: equitable participant selection, appropriate risk-benefit balance, an informed consent process that includes discussion of the AI specific risks and mitigation strategies, appropriate privacy and confidentiality protections, and suitable plans for monitoring the resulting data. These issues would apply to both biomedical and SBER research taking advantage of AI methods.

• The Secretary’s Advisory Committee on Human Research Protections (SAHRP) recently published “IRB Considerations on the Use of AI in Human Subjects Research” (Oct. 2022): https://www.hhs.gov/ohrp/sachrp-committee/recommendations/irb-considerations-use-artificial-intelligence-human-subjects-research/index.html.

This guidance considers how to apply the Common Rule definitions and regulatory requirements to AI human subjects research. Key areas considered in the document include:

• Whether AI activities meet the regulatory definition of research
• Does the AI research involve the use of private, identifiable information (PII)?
• When might the human subjects exemption categories apply to AI research?
• What protections are likely needed in AI research?
• Informed consent issues include waivers and documentation
• Unique features of AI research to consider, like group harm like stigmatization or loss of benefits for a population subgroup due to AI model results

– The FDA regulations would apply if the AI study were a clinical investigation or the AI to be developed qualifies as a medical device.

• Good Machine Learning Practice for Medical Device Development: Guiding Principles October 2021. https://www.fda.gov/medical-devices/software-medical-device-samd/good-machine-learning-practice-medical-device-development-guiding-principles
• Proposed Regulatory Framework for Modification of AI/ML based Software as a Medical Device. Discussion paper; 9/22/21. https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-and-machine-learning-software-medical-device
• Clinical Decision Support Software -Guidance for Industry and FDA Staff; Sept 2022. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/clinical-decision-support-software

– In October 2022, the White House Office on Science and Technology published an AI Bill of Rights that identified potential harms and appropriate protections While primarily focused on commercial AI, the five basic principles described in this document are also relevant to AI research, including:

• Safe and efficient systems
• Protection from algorithmic discrimination
• Data privacy
• Provision of notification and explanation
• Availability of human alternatives and way to resolve problems

International Governance:

Currently, countries from almost every continent have passed laws and regulations related to AI, including Australia, Brazil, Canada, China, Egypt, India, Japan, and South Africa. Depending on where the research will be conducted, investigators need to be aware of any country-specific AI laws.

– The European Union has published a proposed EU Artificial Intelligence Act that will likely be enacted in the coming year: https://artificialintelligenceact.eu/

– The United Nations Educational, Scientific, and Cultural Organization (UNESCO) as published Recommendations on the Ethics of Artificial Intelligence which were adopted on Nov. 23, 2021. https://unesdoc.unesco.org/ark:/48223/pf0000381137

– Fjeld, J., Achten, N., Hilligoss, H., Nagy, A. & Srikumar, M. Principled artificial intelligence: Mapping consensus in ethical and rights-based approaches to principles for AI. Berkman Klein Center Research Publication (2020). This article presents results of an international survey of AI governance documents.

Organizations working on AI Ethics:

Several leading organizations that are active in the AI Ethics arena are listed below and have useful resources on their websites.

– Berkman Klein Center on Ethics and Governance of AI

– AI Now Institute

– Human-Centered Artificial Intelligence at Stanford

– AIethicist.org

Tools for IRB Review OF AI Research:

Tamiko Eto, MS, CIP, the IRB manager at Kaiser Permanente Research Division, has written extensively on AI in human subjects research. She has developed a unique set of Checklists and decision trees for use in IRB review of AI research (ai-hsr-irb-reviewer-checklist-with-decision-tree). Note that her materials are licensed in the Creative Commons; to view this license, visit http://creativecommons.org/licenses/by-nc-sa/4.0/

Selected Publications on AI ethics:

Keans, M., Roth, A. (2020) The Ethical Algorithm: The Science of Socially Aware Algorithm Design. Oxford University Press

Blackman, R (2022). Ethical Machines: Your Concise Guide to Totally Unbiased, Transparent, and Respectful AI. Harvard Business Review Press.

Bernstein, M. S. et al. Ethics and society review: Ethics reflection as a precondition to research funding. Proc Natl Acad Sci U S A 118, e2117261118 (2021).

Considerations for IRB Review: Artificial Intelligence & Machine Learning. Post by Julie Ozier, MHL, CIP, CHRC, Senior Vice President for IRB Review, Advarra. March 11, 2022

Friesen, P., R. Douglas-Jones, M. Marks, R. et al. Governing AI-Driven Health Research: Are IRBs Up to the Task?, Ethics & Human Research 43, no. 2 (2021): 35-42. DOI: 10.1002/eahr.500085

Murphy, K., Di Ruggiero, E., Upshur, R. et al. Artificial intelligence for good health: a scoping review of the ethics literature. BMC Med Ethics 22, 14 (2021). https://doi.org/10.1186/s12910-021-00577-8

McCradden, M., Anderson, J.A., Stephenson, E.A., et al. A Research Ethics Framework for the Clinical Translation of Healthcare Machine Learning. Amer J of Bioethics, 22:5, 8-22, (2022). DOI: 10.1080/15265161.2021.2013977