Deep-Learning-Based Cognitive Assistance Embedded Systems for People with Visual Impairment

Abstract

For people with vision impairment, various daily tasks, such as independent navigation, information access, and context awareness, may be challenging. Although several smart devices have been developed to assist blind people, most of these devices focus exclusively on navigation assistance and obstacle avoidance. In this study, we developed a portable system for not only obstacle avoidance but also identifying people and their emotions. The core of the developed system is a powerful and portable edge computing device that implements various deep learning algorithms for images captured from a webcam. The user can easily select a function by using a remote control device, and the system vocally reports the results to the user. The developed system has three primary functions: detecting the names and emotions of known people; detecting the age, gender, and emotion of unknown people; and detecting objects. To validate the performance of the developed system, a prototype was constructed and tested. The results reveal that the developed system has high accuracy and responsiveness and is therefore suitable for practical applications as a navigation and social assistive device for people with visual impairment.