Accuracy of AI Tools in the Diagnosis of Benign, Potentially Malignant and Malignant Oral Lesions: A Pilot Study

Background: Artificial intelligence (AI) is expected to play an increasingly important role in medicine and dentistry. While its diagnostic potential has been tested in various medical fields, limited research exists on its applications within oral medicine diagnoses using clinical images. Objective: This pilot study aimed to evaluate the diagnostic accuracy of ChatGPT, Gemini, and Copilot in identifying benign, potentially malignant, and malignant oral lesions. Methods: A cross-sectional study was conducted using clinical images from three categories: benign oromucosal conditions, oral potentially malignant disorders, and malignant oral lesions. Results: ChatGPT evaluated all images and consistently outperformed Copilot—and in some cases Gemini—across multiple diagnostic questions, with statistically significant advantages particularly in the cancer subgroup. Copilot showed the weakest performance, with high rates of missing evaluations and significantly lower proportions of correct responses in several analyses. Across both full-dataset and adjusted analyses, ChatGPT demonstrated the highest diagnostic performance overall. Diagnostic accuracy metrics for malignancy suspicion was similar for ChatGPT and Gemini. Several limitations such as sample size, lack of reproducibility testing and inability of some AI models to process images must be taken into account when interpreting the results. Conclusions: AI tools show promise but cannot yet replace clinical expertise. Further research and development are needed to improve the accuracy and applicability of AI diagnostic tools.