Real-Time Lip Reading and Speech Synthesis Using CTC-CNNBILSTM Networks with Flask Deployment

This paper presents a novel approach to real-time lip reading to speech using a combination of Connectionist Temporal Classification (CTC), Convolutional Neural Networks (CNN), media pipe and Bidirectional Long Short-Term Memory (Bi- LSTM) networks, followed by a deployment strategy using Flask. The proposed system aims to transcribe spoken language from silent video sequences by leveraging the spatial and temporal features of lip movements. Extensive experiments on the GRID dataset demonstrate the effectiveness of the model, achieving 8.15% as Character Error Rate (CER) and 91.85% as Character Accuracy (CA). Additionally, we outline the deployment process, which enables realtime lip reading through a web application.procedures during medical education, simulation-based routines and periodic NR training are required to improve neonatal outcome.