2D-3D Facial Image Analysis for Identification of Facial Features Using Machine Learning Algorithms With Hyper-Parameter Optimization for Forensics Applications

Recognizing a face is a remarkable process that humans naturally use. Computer vision has tried to resemble this ability of human vision as a biometric tool to identify humans. Commercial and law enforcement applications are increasingly using face recognition technology to identify people. Currently, it is one of the most sought-after detection methods used in forensics for criminal identification purposes. Owing to similarities in the appearance of certain faces, especially in criminal cases, this problem poses a great challenge in forensic investigation and detection. The novelty of this work lies in the development of a framework for face recognition using 2D facial images gathered from various sources to generate a 3D face mesh using 468 MediaPipe landmarks which detects multiple faces in real-time. This leads to the generation of input feature vectors being formulated utilizing Euclidean/Geodesic distances and their ratios between the landmarks. These feature vectors are then trained into various classifiers that can provide the correct matching decision in an unrestricted environment such as large pose, expression, and occlusion variations. These quantitative similarity measures can then be presented as statistical evidence to identify criminals in forensic investigations. This two-dimensional to three-dimensional annotation provides a higher quality of three-dimensional reconstructed face models without the need for any additional three-dimensional morphable models. The proposed methods were validated and tested to achieve comparable recognition performance using hyperparameter optimization. Regarding accuracy, the statistical results show that Extreme gradient boosting is the best classification model that provides the best accuracy (78%) for predicting facial images compared with Adaptive Boosting (77%), Random Forest (75%), Bernoulli Naive Bayes (68%), Decision Tree (57%), Logistic Regression (71%), Light Gradient Boosting Model (58%), Extra Tree Classifier (57%), Support Vector Machine (58%), and Nearest Centroid (62%) classifiers which can be further increased by considering a greater number of images in the dataset implying at the potential of further research for scale-up implementation.