Ethiopian sign language recognition using deep convolutional neural network

Abstract

In recent years, several technologies have been utilized to bridge the communication gap between persons who have hearing or speaking impairments and those who don't. This paper presents the development of a novel sign language recognition system which translates Ethiopian sign language (ETHSL) to Amharic alphabets using computer vision technology and Deep Convolutional Neural Network (CNN). The system accepts sign language images as input and gives Amharic text as the desired output. The proposed system comprises of three main stages which are: preprocessing, feature extraction, and recognition. The methodology employed involves data acquisition, preprocessing the acquired data, background normalization, image resizing, region of interest (ROI) identification, noise removal, brightness adjustment, and feature extraction, while Deep Convolutional Neural Network (CNN) was used for end-to-end classification. The data used in this study was acquired from students with hearing impairments at the Debre Markos Teaching College with an iPhone 6s phone which has a resolution of 3024 × 4020. The images are in JPEG file format and were collected in a controlled environment. The proposed system was implemented using Kera’s (Tensorflow2.3.0 as backend) in python and tested using the image dataset collected from Debre Markos Teaching College graduating students of 2012. The results show that the running time was minimized by adjusting the images to a suitable size and color. In addition, the results show an improved recognition accuracy compared to previous works. The proposed model achieves 98.5% training, 95.59% validation, and 98.3% testing accuracy of recognition.