Motor Skill Training in Virtual Reality and Skill Transfer via Effective Feedback Modalities

Abstract

Virtual Reality (VR) is an effective tool for learning and training in a variety of fields including rehabilitation, behavioral analysis, and fine motor skills. The training process and outcome of sports and other physical activities rely heavily on the realness of the environment and the interactive feedback loop. Yet, there is still a gap in existing work regarding using VR for these fields and the effect of different types of feedback on learning and skill transfer. There are many unknowns regarding designing effective VR systems and the impact of feedback types and their presentation on motor learning. In this work, three aims are identified in examining VR's capabilities for (a) training and skill transfer in skill-based sports from the simulated-to-real world and using effective feedback to expedite gaining skills, (b) behavioral analysis in sports, and (c) simulating physical activities. We investigated each aim and validated it by developing (1) a VR table tennis training system to investigate the impact of different feedback modalities on learning and skill transfer, (2) A baseball catching simulation for analyzing users' behaviors regarding their attention and reaction to the flying ball, and (3) a rehabilitation system to encourage stroke patients to perform physical activities. The outcome of human experiments demonstrates the ability of VR to accommodate skill learning, behavioral analysis, and simulated physical activities. Additionally, the results of these experiments suggest the type and quantity of feedback significantly affect how subjects learn, and that a combination of visual and haptic feedback can accelerate learning.