Applying Hand Gesture Recognition for User Guide Application Using MediaPipe

Hand gesture recognition is considered important with development technology in industry 4.0 in Human-Computer-Interactions (HCI) which gives computers the competence to capture and interpret hand gestures the executing command without touching devices physically. The MediaPipe is present as a framework built-in machine learning that has a solution for a hand gesture recognition system. In this research, we develop a simple user guide application using the MediaPipe framework. The user guide is commonly known as documentation about technical communication or a manual in a certain system to assist people. The user guide has step-by-step descriptions about handling a particular system and helps the user deal with user frustration by giving them the means to be identified, understand, and disentangle technical problems that frequently occurred by themselves. In our experiment, we captured a real-time image using Kinect, then trained a variety of hand gesture data, identified each hand gesture, and recognized hand gestures to convey information based on hand gestures in the system user guide application. The user can archive information user guide based on hand gestures that have been recognized. We proposed using hand gesture recognition using MediaPipe in our application to improve the convenience of utilization the user guide application and change user guide application that is still manual become a more interactive application.