Track

Basic Science

Abstract

Accessing high-quality open-access dermatopathology image databases for learning and cross-referencing poses a common challenge for clinicians and dermatopathology trainees. While existing platforms like VisualDx offer collections, they often require subscriptions which limit accessibility.

Here, we employed an artificial intelligence (AI)-enabled workflow to curate and categorize images from the PubMed Central (PMC) database, covering 174 benign and malignant cutaneous neoplasms. We aimed to establish a comprehensive dermatopathology database for educational, cross-referencing, and machine-learning purposes.

Our workflow involved retrieving full-length articles from PMC using specific keywords, extracting relevant images, and classifying them using a novel hybrid method. This approach combined deep learning-based image modality classification with figure caption analyses. Validation on 651 manually annotated images demonstrated the robustness of our workflow, with precision rates of 87.02% for the deep learning approach, 84.44% for the keyword-based retrieval method, and 92.64% for the hybrid approach.

To enhance accessibility, we developed a new website featuring a fully annotated image database of over 7,000 images. The website organizes images based on their ontological relations and offers a user-friendly search bar for rapid retrieval of specific diagnoses.

Finally, we conducted an open-ended challenge study to assess the performance of AI algorithms, including GPT-4v, on the retrieved image dataset. Our analysis revealed limitations in current AI image analysis algorithms, with a zero F1-score in the open-ended setting. Additionally, existing AI algorithms may rely on non-image features to arrive at inaccurate diagnoses. These findings underscore the current challenges in AI-assisted image analysis and need for future development.