Experimental Evaluation of Public Retinal Vessel Segmentation Datasets (2020-2025) with Deep Learning: An Empirical Study

Retinal vessel segmentation is a cornerstone of computer-aided diagnosis systems for ophthalmic and systemic diseases. However, segmentation performance varies substantially across studies due to differences in dataset quality, annotation consistency, and imaging conditions. This study presents a comprehensive experimental evaluation of eleven publicly available retinal vessel segmentation datasets commonly used in research between 2020 and 2025, with eight of them fully assessed under identical conditions using the U-Net++ architecture and three included for reference and contextual comparison. The results reveal notable variability in segmentation accuracy driven by disparities in image resolution, sample size, and annotation precision. Among the evaluated datasets, the Retinal Vascular Tree Analysis (RETA) Benchmark achieved the best overall performance (Dice <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$=0.80 \pm 0.015$</tex>; Accuracy <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$=0.97 \pm 0.016)$</tex>, outperforming all other datasets. CHASE-DB1, AV-DRIVE, HRF, and STARE also achieved Dice <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\geq \mathbf{7 0 \%}$</tex> and Accuracy <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\gt \mathbf{9 5 \%}$</tex>, confirming their robustness and suitability for retinal vessel segmentation benchmarking. In contrast, smaller or older datasets such as Digital Retinal Images for Vessel Extraction (DRIVE), LES-AV from the LES-AV database, and IOSTAR vessel segmentation dataset produced moderate results but still captured major vessel structures effectively. These findings underscore the critical importance of dataset quality and standardization in achieving reliable segmentation outcomes and highlight the need to develop harmonized, multimodal, and large-scale retinal image datasets to advance reproducibility, generalization, and clinical applicability of deep learning-based retinal analysis.