Pronunciation trainer for second language learning using generative AI

Abstract

Generative AI models have demonstrated great promise in a variety of fields, including language learning and translation tasks. This research aims to develop a web-based pronunciation training system using Generative AI techniques to provide real-time feedback and multilingual support. The system leverages advanced AI models including pre-trained Automatic Speech Recognition (ASR) and Text-To-Speech (TTS) models, to analyse and synthesize speech. Machine learning algorithms are additionally used for real-time evaluation. The key features of the system include diverse sample texts for pronunciation, immediate pronunciation feedback, audio of the sample text using the TTS model, audio playback of the user input, support for both English and German languages and finally, an interactive user-interface. To assess the system’s effectiveness, evaluation techniques such as Mean Opinion Score (MOS), response time evaluation and Task Completion Rate (TCR) are employed. The Mean Opinion Score obtained was 3.72 and the Task Completion Rate was 80% showing that this novel system can significantly enhance language learning by providing users with pronunciation training, making it a valuable tool for both educators and learners. Even though AI tools help learners reduce their speaking anxiety, they may have difficulties with interpreting feedback and detecting small pronunciation differences. By creating a comprehensive system that uses generative AI to improve pronunciation training, this novel research aims to overcome existing issues in second-language learning.