Gait classification and trajectory generation for different daily activities using deep learning on an embedded system

Abstract

Designing an active prosthesis that reproduces natural human gait remains challenging because of the inherently complex, non-linear biomechanics of locomotion. Activity classification and gait trajectory regression could help to reach a better control system. Three-step deep learning models were developed to work in sequence based on the kinematic readings (linear acceleration and angular velocity) of the shank. For each step, two deep learning configurations were developed and optimized for accuracy and inference time on the ESP32/ESP32S3 microcontrollers to achieve real-time prediction for the prosthesis’s control. The first step is to classify the type of activity the subject is performing whether static (three activities) or dynamic (five activities) activity. The next step to classify the current phase out of eight phases for certain activities such as walking, running, and bicycling. Lastly, the trajectory for the ankle’s angular velocity is generated on the basis of the output of the previous models. In general, MLP could achieve very close accuracy in comparison to CNN while having much faster performance on the embedded microcontroller. Both achieved almost 98% in differentiating between static and dynamic activities. However, MLP achieved a faster inference time of 281 μs, which is approximately 124 times faster than CNN. Both models also achieved a close accuracy of around 93% in the classification of static activities and dynamic activities. MLP predicted 3.154 ms for static activities, making it approximately 86.2% faster than CNN. In dynamic activities, MLP achieved 4.247 ms and CNN achieved 83.032 ms. Finally, for gait trajectory analysis, the MLP was approximately 97% faster than CNN for walking/running, with an RMSE 8.5% lower 0.549 deg/s to 0.6 deg/s. For bicycling, MLP was 74% faster, but RMSE was 26% higher 0.516 deg/s to 0.409 deg/s. In summary, when sufficient phase information is available, the MLP can achieve accuracy comparable to the CNN while offering significantly faster prediction times. However, in cases where such information is lacking, such as in bicycling activity, the CNN delivers a more accurate trajectory with an acceptable inference time.

Article Highlights

Single shank-mounted sensor classifies daily activities and gait phases in real time on a microcontroller.

A lightweight MLP predicts ankle motion 200 ms ahead with accuracy close to a CNN.

ESP32-S3 cuts inference time below 10 ms without any drop in accuracy.