Artificial intelligence (AI) is a relatively new area for clinical study. And although there is lots of excitement about the potential of AI for improving health and care, its application in clinical practice is still unpredictable, with few interventions making it from published paper to real-world implementation. Part of the problem is that there are no specific, universally agreed minimum standards for clinical trials of AI health interventions, which are crucial for making sure that studies follow good scientific practice and that reporting is complete, comprehensive and transparent.
This makes it hard for researchers to properly design and report their studies, and to address potential sources of bias. It also makes it difficult for the health community to assess the quality of the findings and how (or even whether) an AI intervention might work in a particular setting. The risk is that we adopt new health technologies and products whose safety and efficacy aren’t adequately supported by evidence.
This project brings together experts from around the globe to develop the world’s first reporting standards for clinical trials of health interventions involving AI. This collaborative, international effort is building consensus among clinicians, data scientists, healthcare regulators and national bodies, journal editors, patients and others about the AI-specific considerations for clinical trials, using as a starting point the existing SPIRIT and CONSORT guidelines for clinical trials (of drugs and other therapies). Based on this, it has already developed two checklists – SPIRIT-AI and CONSORT-AI – that can be used by researchers, peer reviewers and journal editors conducting and appraising clinical trials for AI systems.
The SPIRIT-AI and CONSORT-AI guidelines are the first internationally recognised standards for reporting AI clinical trials and their protocols. They will help to put authors, peer reviewers and journal editors on a level playing field to make sure AI clinical trials are supported by the highest level of evidence.
The impact and outcomes
The creation of the SPIRIT-AI and CONSORT-AI standards will be a crucial first step in helping to bridge the gap between clinical AI trial data and real-world care. Not only will the standards help researchers by providing clear guidance to follow, but they will also provide a benchmark against which the health and care community can assess future studies to make sure that we adopt safe and effective technologies and approaches to deliver the best possible care.
Like the existing guidelines, the hope is that the criteria will be endorsed and adopted by the world’s most influential journals, researchers and experts, to ultimately become a global standard for clinical AI trials everywhere.
Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, and the EQUATOR Network
This work was funded by a Wellcome Trust Institutional Strategic Support Fund: Digital Health Pilot Grant, Research England (part of UK Research and Innovation), Health Data Research UK and the Alan Turing Institute
and Melanie Calvert and Samantha Cruz Rivera.
Can we accurately forecast non-elective bed occupancy and admissions in the NHS? A time-series MSARIMA analysis of longitudinal data from an NHS Trust
1 July 2022
Hospitals need to be able to predict their capacity for admitting patients when planning elective surgeries. Researchers funded by HDR UK developed a new model for making forecasts that were more...
A population-based cohort study of obesity, ethnicity and COVID-19 mortality in 12.6 million adults in England
21 June 2022
Obesity dramatically increases the risk of death from COVID-19 but, the extent of this risk across different body weights and ethnic groups was not clear. Researchers using health and Census...