Development of sign language motion recognition system for hearing-impaired people using electromyography signal

Shigeyuki Tateno*, Hongbin Liu, Junhong Ou

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Sign languages are developed around the world for hearing-impaired people to communicate with others who understand them. Different grammar and alphabets limit the usage of sign languages between different sign language users. Furthermore, training is required for hearing-intact people to communicate with them. Therefore, in this paper, a real-time motion recognition system based on an electromyography signal is proposed for recognizing actual American Sign Language (ASL) hand motions for helping hearing-impaired people communicate with others and training normal people to understand the sign languages. A bilinear model is applied to deal with the on electromyography (EMG) data for decreasing the individual difference among different people. A long short-term memory neural network is used in this paper as the classifier. Twenty sign language motions in the ASL library are selected for recognition in order to increase the practicability of the system. The results indicate that this system can recognize these twenty motions with high accuracy among twenty participants. Therefore, this system has the potential to be widely applied to help hearing-impaired people for daily communication and normal people to understand the sign languages.

Original languageEnglish
Article number5807
Pages (from-to)1-22
Number of pages22
JournalSensors (Switzerland)
Volume20
Issue number20
DOIs
Publication statusPublished - 2020 Oct 2

Keywords

  • Bilinear model
  • Electromyography
  • Long short-term memory neural network
  • Motion recognition
  • Sign language

ASJC Scopus subject areas

  • Analytical Chemistry
  • Biochemistry
  • Atomic and Molecular Physics, and Optics
  • Instrumentation
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Development of sign language motion recognition system for hearing-impaired people using electromyography signal'. Together they form a unique fingerprint.

Cite this