AI as good as experts at detecting melanomas

Convolutional neural networks (CNN) can outperform less-experienced pathologists at discriminating melanomas from nevi, reports a recent study. For experienced pathologists, CNNs may still be helpful as a form of triage.

Eighteen international expert pathologists across eight different countries were invited to participate and were asked to assess 50 melanoma and 50 nevi images, as labelled by two expert dermatopathologists to reflect the ground truth. Meanwhile, CNNs were trained and tested using the same panel of 100 images.

Overall, the 18 experts achieved a mean accuracy of 90.33 percent, with average sensitivity and specificity values of 88.88 percent and 91.77 percent, respectively. When trained using unannotated images, the CNN model performed slightly worse, with accuracy, sensitivity, and specificity all being 88 percent. The model area under the curve (AUC) was 0.95.

However, when CNN was trained using images with the tumour region annotated as a region of interest, the model performed just as well as the experts, with an accuracy of 92 percent, sensitivity of 94 percent, and specificity of 90 percent. The AUC for the annotated model was 0.97.

In terms of diagnosis, CNNs trained using unannotated and annotated images differed from ground truth in 12 and eight cases, respectively. Often, these discrepancies occurred for pathologically unequivocal cases.

“Although such classifiers may not yield similar performances on slides from another institution, the practical application of environment-specific assistance tools may be more realistic than an attempt to achieve broad generalization across all environments,” the researchers said. “Further studies are required to investigate CNN-based classifiers in a real-life setting.”