Algorithm-based smartphone apps not reliable in detecting skin cancer

Current algorithm-based smartphone apps show poor and variable performance in detecting all cases of melanoma or other skin cancer, a systematic review of diagnostic accuracy studies has found. Moreover, the existing assessment processes for awarding the Conformit Europenne (CE) marking for these apps are inadequate for protecting the public.

“Smartphone algorithm-based apps for skin cancer all include disclaimers that the results should only be used as a guide and cannot replace healthcare advice,” the researchers said. “Therefore, these apps attempt to evade any responsibility for negative outcomes experienced by users.”

The following databases were searched from inception to 10 April 2019 for studies of any design evaluating algorithm-based smartphone apps to assess images of skin lesions suspicious for skin cancer: Medline, Embase, Cochrane Central Register of Controlled Trials, Cinahl, CPCI, Zetoc, Science Citation Index and online trial registers.

Reference standards were as follows: histological diagnosis or follow-up and expert recommendation for further investigation or intervention. Two investigators independently extracted data and assessed validity using the Quality Assessment of Diagnostic Accuracy Studies 2 tool. They also reported estimates of sensitivity and specificity for each app.

Six different identifiable smartphone apps were evaluated in nine studies, of which six verified results via histology or follow-up (n=725 lesions) and three via expert recommendations (n=407 lesions). Studies had small sample sizes and poor methodological quality, with selective recruitment, high rates of unevaluable images and differential verification. In addition, clinicians performed lesion selection and image acquisition instead of smartphone users. [BMJ 2020;368:m127]

Two apps with CE marking were available for download. There was no published peer-reviewed study found assessing the TeleSkin skinScan app, whereas three studies (n=267, 66 malignant or premalignant lesions) evaluated the SkinVision app, which achieved a sensitivity of 80 percent (95 percent confidence interval [CI], 63–92 percent) and a specificity of 78 percent (95 percent CI, 67–87 percent) for the detection of malignant or premalignant lesions.

However, when verified against expert recommendations, accuracy of the SkinVision app turned out to be poor in all three studies.

“Manufacturers can apply CE marking to [smartphone apps] as long as they have shown compliance with the ‘essential requirements’ as outlined in the EU Medical Device Directive, and without necessarily being subject to independent inspection by notified bodies such as the Medicines and Healthcare products Regulatory Agency in the UK,” the investigators said.

Across all studies identified, smartphone apps provided inconsistent management advice for the same lesions, and such recommendations usually conflicted with histopathological results or clinical assessment. In addition, some apps were unable to identify any melanoma cases. [JAMA Dermatol 2013;149:422-426; HICSS 2014;2675-2684]

“[H]ealthcare professionals who work in primary and secondary care need to be aware of the limitations of algorithm-based apps to reliably identify melanomas and should inform potential smartphone app users about these limitations,” the investigators said.