Development of Augmented Reality Application for Mute and Deaf Children Based on Sign Language

Abstract

There are deaf and mute disabilities not only in adults, but also in children between the ages of 3 and 5 years. Most of them have hearing issues. Hearing loss in children can be divided into two groups, where it occurs before language is developed and post-lingual after language is developed, which are prelingual. The aim of sign language is to provide a brief and thorough understanding of sign language linguistics. The existence of sign language has made it possible for deaf and mute individuals to use their environment in spoken languages. To provide some context, it uses a hand gesture with facial expression and body language. Augmented Reality (AR) is a technology which makes it possible to combine digital content with real-world information in real time. With the help of Augmented Reality application for Mute and Deaf Children with Virtual Reality Learning Experience is used as a teaching aid to enhance communication between deaf and mute children. It will view 3D animation from the sign language pattern and become a more interesting interactive learning device for mute and deaf children to learn sign language. It also improves the understanding of the natural world. The goal of this project was to build an AR framework based on sign language in mobile applications. The pattern of the hand gesture will be spotted by the application when the camera is fixed on a sign language pattern flashcard and an animated 3D of animals will be shown in real time. This project is also intended to test the usability of AR in hand gesture learning. The approach used in this project is to use not only a psychomotor but also a cognitive domain. The outcome shows that the best ways for mute and deaf children to learn and understand the hand gesture better are through the combination of picture and movement.