Enhancing the Recognition of Arabic Sign Language by Using Deep Learning and Leap Motion Controller

  • 13 Mar 2021
  • Recently published Research - Informatics & Communication


Ammar Alnahhas, Bassel Alkhatib, Nazeer Al-Boukaee, Noor Alhakim, Ola Alzabibi, Noor Ajalyakeen

Published in

International Journal of Scientific & Technology Research, Volume 9, Issue 4, April 2020.


Because of the need to build assistive systems for people with special needs, many computer systems have emerged to help understand sign language. In this paper we present an innovative method for recognizing words in Arabic sign language using the Leap Motion device which helps building a 3D model of the human hand using infrared. Our methodology focuses on analyzing the mathematical features derived from the Leap Motion controller, where we take advantage of the chronology of successive frames that can be represented using the extracted features, we process these features using a Recurrent neural network. We present our results which we have performed on real data, the results show superiority of our method Which has achieved outstanding results that outperform previous research. The experiment result shows that the highest average classification rate reached 89% for one-hand gestures, 96% for two hands gestures.

Keywords:  Sign language recognition, Arabic sign language, Deep learning, LSTM, RNN, Leap Motion controller, neural networks.

Link to read full paper