Search for a command to run...
Even though the area of assistive technologies has just improved significantly, and there have been efforts to achieve the digital inclusiveness of the deaf and the hard-of-hearing community. The deaf and hard-of-hearing population in India have met enormous communication barriers due to lack of awareness and resources. Sign language is one of the necessary tools of communication, that are vital to deaf and hearing-impaired people, however, communication between sign language speakers and none-signers is one of the greatest problems. In this paper, a real-time and multilingual translator and sign language recognition system will be proposed to overcome the communication barrier between deaf and hearing communities using a browser-based system. The suggested system makes use of MediaPipe to extract landmarks of the hand efficiently and a Convolutional Neural Network (CNN) model to recognize gestures. The identified signs have been converted to various languages, with the help of external translation APIs, which allows them to be accessible to all of the language territories like Indian regional languages i.e. Hindi, Marathi, Bengali, Tamil and Punjabi, etc. The first priority has been given to DeepL. The architecture is also designed to use commodity hardware and client-side processing reducing computational overhead and making it scalable to deploy. The system also incorporates a user dashboard on activity tracking and learning analytics where the users are able to track the frequency of sign practice and history of interaction. The experimental outcomes prove that the suggested method is moderately accurate with low latency, which means that it is possible to recognize sign language in real-time on Internet-based platforms without special equipment. The platform inspires the hearing and deaf students to acquire the skills of learning the sign language at a long-term level with the assistance of learning modules that are flexible in the sense of difficulty and analytic-based feedback, recognition, and translation.