Computer Vision in Spiritual Seeing: Recognition of Christian Saints in Orthodox Iconography

Christian Orthodox iconography is a fundamental element of the religious cultural heritage of many countries. Iconoclasm, vandalism, and the passage of time ruined the appearance of icons, making it difficult to recognize the depicted saints. This work aims to test the performance of 13 state-of-the-art deep learning models for the task of Christian Orthodox saints’ recognition from images of preserved wooden hand-painted icons, which has never before been reported in the literature. Additionally, this work introduces the first public image dataset (ICONSAINT—ICONographic SAINT Recognition Dataset) of saint icons for classification tasks, including 2730 annotated images of 546 icons of 123 classes. All models were tested in three experimental setups, involving a balanced part of the dataset of six classes, an imbalanced part of the dataset of 12 classes and a medium-imbalanced part of the dataset of eight classes, reporting accuracy of up to 89% with VGG19 for the balanced data, of up to 78% for MobileNet with the imbalanced data, and of up to 87% with DenseNet201 for the medium-imbalanced data. Moreover, Class Activation Maps (CAMs) were considered to highlight the regions of the input image that mostly influenced the decision of the models towards adding valuable explainability to the results through visual explanations.