Assessment of Radiology Artificial Intelligence Software: A Validation and Evaluation Framework

Autor: William Tanguay, Philippe Acar, Benjamin Fine, Mohamed Abdolell, Bo Gong, Alexandre Cadrin-Chênevert, Carl Chartrand-Lefebvre, Jean Chalaoui, Andrei Gorgos, Anne Shu-Lei Chin, Julie Prénovault, François Guilbert, Laurent Létourneau-Guillon, Jaron Chong, An Tang
Rok vydání: 2022
Předmět:
Zdroj: Canadian Association of Radiologists journal = Journal l'Association canadienne des radiologistes.
ISSN: 1488-2361
Popis: Artificial intelligence (AI) software in radiology is becoming increasingly prevalent and performance is improving rapidly with new applications for given use cases being developed continuously, oftentimes with development and validation occurring in parallel. Several guidelines have provided reporting standards for publications of AI-based research in medicine and radiology. Yet, there is an unmet need for recommendations on the assessment of AI software before adoption and after commercialization. As the radiology AI ecosystem continues to grow and mature, a formalization of system assessment and evaluation is paramount to ensure patient safety, relevance and support to clinical workflows, and optimal allocation of limited AI development and validation resources before broader implementation into clinical practice. To fulfil these needs, we provide a glossary for AI software types, use cases and roles within the clinical workflow; list healthcare needs, key performance indicators and required information about software prior to assessment; and lay out examples of software performance metrics per software category. This conceptual framework is intended to streamline communication with the AI software industry and provide healthcare decision makers and radiologists with tools to assess the potential use of these software. The proposed software evaluation framework lays the foundation for a radiologist-led prospective validation network of radiology AI software. Learning Points: The rapid expansion of AI applications in radiology requires standardization of AI software specification, classification, and evaluation. The Canadian Association of Radiologists' AI Tech & Apps Working Group Proposes an AI Specification document format and supports the implementation of a clinical expert evaluation process for Radiology AI software.
Databáze: OpenAIRE