Thew Dhanat's Blog

AI in Medicine Checklists

October 29, 2022

This is an in progress post. More updates will be coming (soon).

Artificial Intelligence / Machine learning application to other disciplines is an evolving area, including medicine. Many researchers are rushing to use machine learning without a comprehensive understanding. There are many pitfalls and errors in machine learning that even AI practitioners sometimes are not invulnerable. Many reporting guidelines and checklists were developed in an attempt to fix those problems, yet I expect many more to come as more issues appear. Some of them are for general AI research and some of them are specific to a particular field. In this blog post, I compile general information, reporting guidelines and checklists for AI-in-medicine researchers. They might not directly solve the problems, but they can help us keep the issues in concern and give more crucial information to readers. Besides researchers, healthcare stakeholders can use these resources to evaluate AI-in-medicine research. If you find any errors or missing resources, please kindly contact me.

General

State of AI Report 2022

2021 AI Index Report

Research

Model Cards for Model Reporting

Datasheets for Datasets

Government

Europe

EU AI Act

The State of State AI Policy (2021-22 Legislative Session)

Ethics guidelines for trustworthy AI

US

National Security Commission on Artificial Intelligence’s (NSCAI)

NIST AI Risk Management Framework (AI RMF)

An Accountability Framework for Federal Agencies and Other Entities

Reproducibility

Leakage and the Reproducibility Crisis in ML-based Science

Medicine

Minimum information about clinical artificial intelligence modeling: the MI-CLAIM checklist

Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence

Developing a reporting guideline for artificial intelligence-centred diagnostic test accuracy studies: the STARD-AI protocol

Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI

Specialty

Proposed Requirements for Cardiovascular Imaging-Related Machine Learning Evaluation (PRIME): A Checklist: Reviewed by the American College of Cardiology Healthcare Innovation Council

Standardized Reporting of Machine Learning Applications in Urology: The STREAM-URO Framework

Government

Artificial Intelligence and Machine Learning (AI/ML)-Enabled Medical Devices

FDA-approved A.I.-based algorithms

European Health Data Space

Technical Performance Assessment of Quantitative Imaging in Radiological Device Premarket Submissions

Clinical trials

Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension

Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension