Sign to Speech Language Converter Using Smart Data Glove

Authors

  • Wei Jie Lim , Yvette Shaan-Li Susiapan , Subhashini Gopalkrishnan

Abstract

This research work details a smart data glove integrated with sensors for sign language to speech
conversion. Sign languages developed as a means of communication wherever there are people with hearing
disabilities. Signing is not only used by those who are deaf but also by individuals who are physically unable
to speak due to disability. This data glove is able to receive hand movements which represent different
alphabets or words in American Sign Language (ASL) via sensors located on the glove. The data collected
from the glove is transmitted wirelessly to the processing unit which then processes the data to produce the
output in the form of speech. The output provided is also displayed onscreen in real time. Supervised
learning artificial intelligence was implemented in this project to obtain higher accuracy in detecting hand
gestures. The results from the flex sensor showed an accuracy of 38.46% for a single sensor but improves to
99% when the sensor is used in conjunction with a gyroscope sensor for sign language recognition while the
response time of the system to recognize an alphabet upon receipt of data from the data glove is less than 1
second. A fast response time is necessary as this project is intended for real time application. 16000 training
datasets were used in supervised learning technique resulting in an average accuracy of 90.77% of ASL
alphabet recognition when the glove was worn by different users. This glove can be used by people with
speech or hearing disabilities during meetings or presentations attended by differently abled persons as they
will be able to deliver the message in ASL to people who are not well versed in ASL, in real time, therefore
removing the communication barrier between the them

Published

2020-01-31

Issue

Section

Articles