AI goes toe-to-toe with experts at diagnosing tumour metastasis

Artificial intelligence (AI) algorithms can be used to diagnose tumour metastasis through medical radiology imaging, achieving comparable or even better performance than healthcare professions, according to a recent review and meta-analysis.

However, the body of research currently available has several notable weaknesses.

“The methodology and reporting of studies on the AI application in the medical field is often flawed,” the researchers said. “Normative and rigorous reporting standards should be established to enable the results to be more credible.”

From the online databases of PubMed and Web of Science, 69 studies met the inclusion criteria, of which 34 were qualified for a meta-analysis. Only papers that applied an AI model to medical image for detecting tumour metastasis were considered; those that instead used waveform data or histopathology images were left out.

Pooled receiver operating characteristic curve analysis of the 34 studies showed that AI algorithms were highly accurate in detecting tumour metastasis, with an area under the curve (AUC) of 0.90. The resulting sensitivity and specificity values were 82 percent and 84 percent, respectively. [EClinicalMedicine 2020;31:100669]

Notably, several of the studies included in the quantitative analysis used more than one AI algorithm, each yielding different accuracy estimates. When the most accurate algorithms in each study were considered, the AUC value jumped to 0.93, sensitivity to 87 percent, and specificity to 88 percent.

Eight studies compared the AI algorithms with the clinical assessment of healthcare professionals. Pooled analysis of these showed that AI performed slightly better, both in terms of sensitivity (89 percent vs 72 percent) and specificity (85 percent vs 72 percent) for diagnosing tumour metastasis.

“Diagnosis of tumour metastasis using AI algorithms has great potential. From this meta-analysis, we conservatively draw a conclusion that the AI algorithms may be used for the diagnosis of tumour metastasis from medical radiology imaging with equivalent or even better performance to healthcare professionals, in terms of sensitivity and specificity, providing a basis for its clinical application,” the researchers said.

However, they also noted that “although the outcome of our research seems to bring light to the application of AI in detecting tumour metastasis from medical radiology imaging, several common methodological defects should be noted.”

Most notably, only a small number of studies compared the performance of AI against that of a trained healthcare professional. “Assessing the performance of AI in insolation instead of comparing with the most common way in clinical practice makes the outcomes unreliable when applied in the clinical setting,” the researchers explained.

Moreover, of the few studies that did this comparison, even fewer used the same dataset or had lax criteria for professional recruitment. There was also a lack of external validation. Of the 69 studies included, only four validated their findings in an outside cohort, making it difficult to generalize their findings to other populations.

“High-quality studies are always the cornerstone of evaluation for diagnostic performance by various algorithms, which will finally benefit patients and the healthcare system,” the researchers said. More studies with better designs are needed to establish a stronger case for the clinical use of AI in diagnosing tumour metastases.