Robust Balinese Script Detection for Transliteration Using YOLOv8

YOLOv8-based detection system was developed to support automatic transliteration of Balinese script by addressing challenges related to small character size, visual similarity, and varying image quality. The study evaluates detection performance using a Balinese script dataset consisting of four dataset variations, Original, Grayscale, Augmented, and book-excluded dataset, designed to analyze the impact of preprocessing and data variation on model accuracy. The Original and Grayscale datasets demonstrating superior performance by achieving mAP@50 values above 0.93. Hyperparameter tuning experiments involving optimizer selection, epoch variation, and model scale indicated that the YOLOv8x variant with the auto optimizer and a fine-tuning strategy delivered the highest performance, reaching an mAP@50 of 0.9405 while maintaining balanced precision and recall. Confusion matrix analysis showed that most detection errors occurred between visually similar characters. Robustness evaluation further revealed that detection performance remains reliable when image clarity moderate, rotation does not exceed ±3°, shear remains within ±2°, and character bounding boxes occupy approximately at least 1% of the total image dimension. These findings contribute the importance of dataset preparation, model configuration, and robustness constraints to achieve accurate Balinese script detection for real-world transliteration applications.