Mathematical Modeling and Statistical Evaluation of Hybrid Deep Learning Architectures for Multiclass Classification of Cervical Cells in Digital Papanicolaou Images

Cervical cytology screening remains dependent on manual analysis, which is time-consuming and subject to variability. This study proposes a leakage-free hybrid deep learning framework for multiclass classification of cervical cells extracted from whole-slide Papanicolaou images. A fine-tuned DenseNet121 feature extractor was combined with three classifiers: Support Vector Machine (SVM), Stacked Extreme Learning Machine (SELM), and Cascaded Deep Forest (CDF). Experiments were conducted on the CRIC Cervix Collection dataset using slide-level data partitioning and group-aware stratified 7-fold cross-validation. Model comparison followed a paired non-parametric protocol (Friedman test with Wilcoxon post hoc and Holm correction). DenseNet121 + CDF achieved the highest cross-validation Accuracy (0.7370 ± 0.0357), significantly outperforming SVM (0.6644 ± 0.0287) and SELM (0.6431 ± 0.0471) (χ2(2) = 11.14, p = 0.0038; Kendall’s W = 0.79). Independent testing showed competitive generalization across models. These results support the statistical robustness of the Cascaded Deep Forest-based hybrid architecture for multiclass cervical cytology classification under realistic slide-level conditions.