Gesture recognition method integrating multimodal inter-frame motion and shared attention weights

Abstract

To improve the accuracy and precision of gesture recognition, this study improves YOLOv5 by incorporating a coordinate attention mechanism and a bidirectional feature pyramid network. Based on the improved YOLOv5, a static gesture recognition model is constructed. In addition, this study introduces a multimodal inter-frame motion attention weight module to enhance the model’s ability to recognize dynamic gestures. In the performance evaluation experiments, the proposed model achieves an area under the receiver operating characteristic curve of 0.94, a harmonic mean of 96.4%, and an intersection over union of 0.9. The accuracy of static gesture recognition reaches 100%, while the average accuracy of dynamic gesture recognition achieves 95.7%, which significantly outperforms the comparison models. These results demonstrate that the proposed gesture recognition model offers high accuracy for static gestures and reliable recognition performance for dynamic gestures. This approach provides a potential method and perspective for improving human–computer interaction in virtual reality and intelligent assistance scenarios.