Data Augmentation for Voiceprint Recognition Using Generative Adversarial Networks

Abstract

Voiceprint recognition systems often face challenges related to limited and diverse datasets, which hinder their performance and generalization capabilities. This study proposes a novel approach that integrates generative adversarial networks (GANs) for data augmentation and convolutional neural networks (CNNs) with mel-frequency cepstral coefficients (MFCCs) for voiceprint classification. Experimental results demonstrate that the proposed methodology improves recognition accuracy by up to 15% in low-resource scenarios. The optimal ratio of real-to-GAN-generated samples was determined to be 3:2, which balanced dataset diversity and model performance. In specific cases, the model achieved an accuracy of 96.6%, showcasing its effectiveness in capturing unique voice characteristics while mitigating overfitting. These results highlight the potential of combining GAN-augmented data and CNN-based classification to enhance voiceprint recognition in diverse and resource-constrained environments.

1. Introduction

In this era of rapid technological advancement, artificial intelligence has become an essential part of daily life, offering various conveniences. A notable application of Artificial Intelligence (AI) is voice recognition, which has become ubiquitous in many real-world scenarios. Voiceprint recognition, in particular, is a key technology used for identity verification and has been extensively integrated into access control systems and activation devices.

Voiceprint recognition systems extract distinct features from speech signals to identify speakers. This technology is widely employed in security authentication and customer service. However, traditional voiceprint recognition systems often face challenges associated with small-sample learning. In many practical applications, the availability of sound samples is inconsistent or insufficient due to various constraints, which can result in suboptimal model performance. Furthermore, a lack of diverse training samples would limit the model’s ability to recognize different patterns.

In real-world situations, collecting sound samples is difficult due to varying conditions, leading to an uneven and inadequate sample distribution. Furthermore, accurate sound sample collection is critical for effectively operating applications like access control systems.

To address these challenges, this study aims to improve the reliability, precision, and flexibility of voiceprint recognition systems, even when dealing with limited sample sizes. By employing generative adversarial networks (GANs)—a type of neural network capable of generating data similar to the original dataset—the study seeks to create synthetic sound samples. This approach aims to expand the dataset, particularly when certain samples are scarce. By supplementing the training data with high-quality, diverse samples, the study anticipates improving the accuracy and dependability of the model.

Users can customize the GAN model to generate new samples tailored to specific requirements, allowing the model to train alongside the original dataset. This process enhances the system’s ability to identify sounds in various real-world conditions, thereby reducing the cost and effort of collecting sound samples for companies developing similar systems. Utilizing GANs to generate new samples creates data that closely resemble real-world sound conditions. GANs effectively address challenges posed by diverse accents and languages, enabling voiceprint recognition systems to perform reliably across a wide range of linguistic and contextual variations. This approach not only reduces development costs but also eliminates the need for significant investments in collecting a wide variety of sound samples, making voiceprint recognition systems more accessible and reliable.

This study proposes an innovative approach to voiceprint recognition by integrating GANs and convolutional neural networks (CNNs) with mel-frequency cepstral coefficients (MFCCs) for enhanced data augmentation and feature extraction. We introduced a dynamic data augmentation strategy using GANs to generate synthetic voice samples, addressing the challenge of data scarcity. Second, the methodology incorporates an innovative iterative reduction technique to balance the ratio of real and GAN-generated samples, optimizing model performance and mitigating overfitting. Third, a custom-designed convolutional neural network (CNN) leverages MFCC features to effectively capture unique voice characteristics, ensuring high accuracy in resource-constrained scenarios. Finally, the approach is validated through comprehensive experiments on diverse datasets, demonstrating significant improvements in recognition accuracy and robustness compared to traditional methods.

By leveraging the power of GANs, we aim to create diverse and realistic voice samples that can enhance the performance of voiceprint recognition models, even in the face of limited training data. Generative adversarial networks have emerged as a powerful class of deep learning models that can generate novel and realistic synthetic data. These models consist of two neural networks, a generator, and a discriminator, that are trained in an adversarial fashion. The methodology enhances both data augmentation and feature extraction processes. The use of GANs allows for the generation of synthetic voice samples, effectively expanding the dataset and improving model training. A key innovation of this approach lies in the iterative reduction of synthetic samples, which balances the contribution of real and GAN-generated data to mitigate overfitting and maintain robust performance.

The CNN model is specifically tailored for voiceprint recognition tasks, employing MFCC features to capture critical speech characteristics. Designed with advanced techniques such as dropout layers and adaptive optimizers, the architecture is optimized to achieve high accuracy in resource-constrained environments. The methodology’s effectiveness is demonstrated through comprehensive experiments involving diverse datasets, varied sample sizes, and different ratios of synthetic and real samples, showcasing its adaptability and reliability.

This approach also highlights its practical applications, including multilingual speaker verification, biometric authentication, and forensic analysis. By reducing data collection costs while maintaining high performance, the methodology offers a resource-efficient solution that is well-suited for real-world implementations. These contributions distinguish the proposed method from existing approaches, addressing the common limitations of data scarcity and overfitting in voiceprint recognition systems.

2. Literature Review

In the field of sound processing, insufficient samples are a major problem affecting model performance. The collection and annotation of sound data usually requires a lot of manpower and material resources, and GANs provide an effective method to generate more sound samples, so as to improve the combination of CNNs and MFCCs in sound-processing tasks. In this article, we will discuss the research on the use of GANs to generate samples to solve the problem of insufficient sound samples processed by CNNs, and introduce the functions and operating principles of various tools.

The following section delves into the background and existing research related to the development of voice recognition systems, including CNNs, MFCCs, and the application of GANs in sound processing. These technologies are instrumental in overcoming challenges in voice recognition, particularly those posed by limited and diverse datasets.

2.1. Convolutional Neural Network for Voiceprint Recognition

Convolutional neural networks are a class of deep learning models primarily designed for tasks involving structured grid-like data, such as images and videos. CNNs have shown remarkable performance in tasks like image classification, object detection, and even speech recognition, when combined with certain preprocessing techniques like MFCCs. The distinguishing feature of CNNs is their ability to automatically and adaptively learn spatial hierarchies of features from input data. They consist of multiple layers, including convolutional layers, activation layers, pooling layers, and fully connected layers, each of which plays a specific role in feature extraction and classification.

A CNN is a powerful deep learning framework that excels in identifying structured patterns within data. While originally designed for image processing, CNNs have proven highly effective in speech and voice recognition tasks by leveraging feature representations such as spectrograms and MFCCs.

In this study, CNNs are utilized to process MFCC features extracted from voice samples. The CNN architecture comprises convolutional layers that detect intricate patterns within the MFCCs, pooling layers that reduce spatial dimensions while retaining critical features, and fully connected layers that perform final classification. The inclusion of dropout layers mitigates overfitting, ensuring the model’s robustness. This design allows the network to capture unique voice characteristics, enabling precise speaker identification and verification.

By focusing on extracting and classifying voice features, the CNNs in this methodology are optimized for performance in low-resource and diverse voiceprint recognition scenarios. The model’s parameters, such as the number of convolutional filters and kernel sizes, were selected based on empirical testing to balance computational efficiency and recognition accuracy.

2.2. Mel-Frequency Cepstral Coefficients in Voiceprint Recognition

Mel-frequency cepstral coefficients are widely used in speech and voice recognition for their ability to capture the essential characteristics of speech signals in a compact and computationally efficient form. By simulating the human auditory perception of sound frequencies, MFCCs provide robust features that enhance voiceprint recognition accuracy.

In this study, MFCCs are extracted from audio data to serve as input features for the CNN model. The preprocessing involves transforming raw audio signals into MFCC features using techniques such as frame segmentation, spectral analysis, and cepstral transformation. This ensures that critical information such as phonetic patterns and vocal characteristics is preserved for subsequent classification.

Delta and delta-delta cepstrum features, which capture the rate of change and temporal dynamics of the MFCCs, are also included to enhance the model’s ability to generalize across diverse speech patterns. By combining static and dynamic features, the MFCC representation allows the CNN to learn both the fundamental and evolving attributes of voice samples, improving speaker recognition performance even with limited data.

2.3. Generative Adversarial Networks

GANs are a type of neural network architecture designed for unsupervised learning, consisting of two main components: a generator and a discriminator. These networks are trained in a competitive setting, where the generator tries to create data similar to the real dataset, and the discriminator evaluates whether the data are real or generated. Through this adversarial process, GANs can generate realistic data that mimic the statistical distribution of the training dataset, making them highly effective in tasks like image generation, speech synthesis, and data augmentation.

2.3.1. Generator

The generator in a GAN is responsible for creating synthetic data that resemble real data. It starts with random noise as input, typically sampled from a uniform or Gaussian distribution, and transforms this noise into structured data through a series of layers. The generator’s goal is to produce outputs that are indistinguishable from real data.

The process begins by sampling from a latent space, which is a lower-dimensional representation of the input space. The latent space captures the underlying factors that define the data distribution. By feeding noise into the latent space, the generator produces output through a series of transformations, such as fully connected layers, convolutional layers, and transposed convolutions (for images). These transformations allow the generator to map random noise to structured data, like images, audio, or text.

The generator’s objective is to “trick” the discriminator into classifying its output as real. This goal is achieved by iteratively updating the weights of the generator through backpropagation, based on feedback from the discriminator. The better the generator becomes at fooling the discriminator, the closer the generated data becomes to the real data distribution.

Formally, the generator’s objective is to minimize the following loss function, as follows:

$L_G=-{\mathrm{E}}_{z~p_z\left(z\right)}\left[\log \left(D\right(G\left(z\right)\left)\right)\right]$

where $G\left(z\right)$ is the output of the generator given input noise z, and $D\left(G\right(z\left)\right)$ is the discriminator’s prediction for the generated data.

The goal is to minimize the discriminator’s ability to distinguish between real and fake data.

2.3.2. Discriminator

The discriminator is a binary classifier that distinguishes between real data (from the training set) and fake data (produced by the generator). Its job is to correctly label real data as real and generated data as fake. The discriminator receives inputs from both the real dataset and the generator’s output and must decide whether each input is genuine or artificially created.

The discriminator is typically implemented as a neural network, which can consist of fully connected layers or convolutional layers, depending on the task (e.g., image, text, or audio recognition). It assigns a probability score to each input, indicating whether it believes the input is real (closer to 1) or generated (closer to 0). The discriminator’s training process involves minimizing its classification error by correctly identifying real versus fake data.

The discriminator’s objective function is to maximize the likelihood of correctly classifying real and fake samples. This can be expressed as follows:

$L_D=-{\mathrm{E}}_{x~p_{data}\left(x\right)}\left[\log \left(D\left(x\right)\right)\right]-{\mathrm{E}}_{z~p_z\left(z\right)}\left[\log (1-D(G\left(z\right)\left)\right)\right]$

where $D\left(x\right)$ is the discriminator’s prediction for a real sample x, and $D\left(G\right(z\left)\right)$ is its prediction for a generated sample $G\left(z\right)$.

The discriminator aims to maximize its ability to distinguish between real and fake data while minimizing the generator’s chances of fooling it. As the discriminator improves, it becomes more challenging for the generator to produce convincing fake data.

2.3.3. Objective Function

GANs use an adversarial training process, where the generator and discriminator are both trying to optimize opposing objectives. The discriminator seeks to maximize the probability of correctly classifying real versus fake data, while the generator tries to minimize the discriminator’s ability to tell the difference. This competition is modeled using a minimax objective function, as follows:

$\underset{G}{\min }\underset{D}{\max }V(D,G)={\mathrm{E}}_{x~p_{data}\left(x\right)}\left[\log \left(D\left(x\right)\right)\right]+{\mathrm{E}}_{z~p_z\left(z\right)}\left[\log (1-D(G\left(z\right)\left)\right)\right]$

This equation represents the minimax game between the generator and the discriminator, as set out below:

• -. The generator ($G$) aims to minimize this function, by improving its ability to produce data that the discriminator cannot distinguish from real data.

• -. The discriminator ($D$) aims to maximize this function, by improving its ability to correctly identify real and generated data.

The objective function is based on the cross-entropy loss, commonly used in classification tasks. As the GAN trains, the generator’s ability to produce realistic data improves, while the discriminator becomes more adept at distinguishing real data from fake data. The ideal state, or Nash equilibrium, is reached when the generator can produce data so realistic that the discriminator cannot tell the difference with better than random guessing (i.e., 50% accuracy).

GANs consist of two neural networks—a generator and a discriminator—that are trained simultaneously in an adversarial setup. The generator creates synthetic samples from random noise, while the discriminator evaluates whether the samples are real or generated. This iterative process improves the generator’s ability to produce realistic data while enhancing the discriminator’s ability to distinguish between real and synthetic samples.

The generator aims to minimize the discriminator’s ability to differentiate between real and generated samples, while the discriminator strives to maximize its classification accuracy. This dynamic adversarial training is modeled as a minimax optimization problem, where the generator and discriminator iteratively improve by competing against each other. Effective training requires carefully tuned hyperparameters, such as learning rates and batch sizes, to ensure convergence and avoid instability or pattern collapse.

In this study, the GAN is utilized to augment the voiceprint dataset by generating synthetic audio samples. The training process incorporates mechanisms such as regularization and adaptive learning rates to stabilize the adversarial training and maintain the diversity of generated samples. By combining the synthetic samples with real data, the GAN enhances the training dataset, addressing the challenges of data scarcity and improving the performance of the voiceprint recognition system.

2.4. Application of Deep Learning in Speaker Recognition

GANs have seen extensive application in various fields, especially in tasks involving data generation and enhancement. Research in deep learning and speech processing has demonstrated that GANs can address significant challenges in areas like speaker recognition, speech enhancement, and data augmentation. In this section, we explore the role of GANs in conjunction with other deep learning techniques and spectral feature extraction methods, as well as their application in generating and enhancing speech data.

Wan et al. proposed a method for extracting speaker features using deep neural networks, called Deep Speaker Embeddings. These embeddings effectively distinguish between different speakers, significantly enhancing the accuracy and stability of speech recognition [1]. Kang et al. introduced a CNN-based end-to-end speaker verification method for text-independent vocal verification by learning speech feature representations. This approach demonstrates good adaptability and robustness when processing different speech datasets, while also showcasing the application of CNNs in speech feature extraction, which could serve as a reference for combining MFCC and a GAN to generate samples [2]. Snyder et al. presented a method for speaker recognition using deep neural network (DNN) embeddings. This method extracts features through a DNN for vocal recognition, leading to a significant performance boost [3].

Chung and Nagrani introduced the VoxCeleb2 dataset to demonstrate its use in deep voice recognition, showing a notable improvement in voice recognition performance. VoxCeleb2 contains a large variety of speech data, which contribute to advancing deep learning research in this field [4]. Lei et al. proposed a voice recognition model based on a phoneme perception deep network. By incorporating phonemic information, this model improves its ability to differentiate vocal features, thus enhancing recognition accuracy [5]. Ravanelli and Bengio introduced SincNet, a deep network model for speaker recognition directly from raw waveforms. SincNet shows strong performance by extracting features directly from the original audio waveform, avoiding complex pre-processing steps [6]. Heigold et al. developed an end-to-end text-dependent speaker verification method that extracts features from speech data using a DNN. This approach simplifies traditional feature extraction and matching steps, improving verification accuracy through end-to-end learning [7].

Snyder et al. also proposed another text-independent speaker verification method using DNN embeddings, significantly enhancing verification performance. This approach emphasizes the robustness and consistency of DNN embeddings in handling different textual content [3].

2.5. Spectral Feature Extraction Methods

Spectral feature extraction is a widely used technique in speaker recognition, with MFCCs being one of the most common features. Hinton et al. summarized common views on the role of DNNs in speech modeling, emphasizing the importance of spectral features in deep learning [8]. Davis and Mermelstein conducted a detailed comparison of multiple parametric representations and proposed the use of MFCCs in continuous speech recognition. Their method converts speech signals into feature vectors, making the subsequent recognition process more efficient and accurate [9].

Rabiner and Juang extensively covered the basics of speech recognition, including spectral feature extraction methods, in their work. They delved into techniques like linear predictive cepstral coefficients (LPCCs) and MFCCs, comparing their performance in various speech recognition tasks [10].

Muda et al. introduced an algorithm combining MFCC and dynamic time warping (DTW) for speech recognition, effectively improving recognition efficiency by leveraging MFCC’s feature extraction capabilities and DTW’s time alignment [11].

Logan discussed the application of MFCCs in music modeling, showing that MFCCs are effective across a range of audio processing tasks, improving the accuracy and stability of feature extraction [12].

Kim and Stern introduced power normalized cepstral coefficients (PNCCs), a new spectral feature extraction method that offers better robustness in noisy environments [13]. Wang et al. proposed a line spectrum frequency distance measurement method based on one-way inner product for speech feature extraction. This new method improves the accuracy and stability of feature extraction while effectively handling background noise [14]. Alkhatib et al. researched the principles of MFCC and LPCC parameter extraction, and the methods for extracting differential cepstral parameters. Their study demonstrated that using MFCC, LPCC, and their first- and second-order differences in combination with the K-means algorithm and a three-layer backpropagation neural network can significantly improve recognition rates. They also confirmed that MFCC parameters offer better robustness than LPCC [15].

2.6. Application of Generative Adversarial Networks in Speech Synthesis and Enhancement

GANs have shown great potential in speech synthesis and enhancement. Saito et al. explored GAN-based methods for speech enhancement and noise robustness in vocal verification [16]. Wang et al. discussed the use of GANs for data augmentation in speaker recognition. Their approach significantly improves recognition performance by generating new training samples, particularly when data are insufficient [17]. Pascual et al. proposed the speech enhancement generative adversarial network (SEGAN), which reduces noise and enhances speech quality through adversarial training, generating clearer and more natural speech signals [18].

Sisman et al. summarized the development and challenges in speech conversion research, exploring various GAN applications and outlining the technical challenges faced in this field [19]. In Kaneko et al.’s study, an innovative speech conversion method was proposed using a sequence-to-sequence (Seq2Seq) model with GAN to learn similarity measures, improving the naturalness of generated speech [20]. Chandna et al. discussed CNN-based techniques for mono audio source separation, which was particularly useful in speech enhancement tasks as it effectively separated speech from background noise [21].

Skerry-Ryan et al. introduced a GAN-based Tacotron model for intonation transfer, achieving more natural speech synthesis effects by adapting intonation characteristics to different contexts and emotional expressions [22]. Pan et al. proposed a GAN with dual discriminators for speech enhancement, improving the quality and stability of generated speech, particularly in noisy environments [23].

In addition to deep learning and spectral feature extraction, traditional machine learning methods like support vector machines (SVMs) and Gaussian mixture models (GMMs) are still widely applied in speaker recognition. Campbell et al. combined SVM and GMM hypervectors for speaker verification, demonstrating improved performance in discriminative recognition [24].

Reynolds et al. detailed the use of adaptive GMMs in speaker verification, showcasing the model’s effectiveness and stability in this field [25]. Bimbot et al. provided a thorough review of text-independent speaker recognition techniques, serving as a valuable resource for researchers [26]. Kinnunen and Li summarized recent research advances in speaker recognition, covering topics from feature extraction to hypervector modeling and identifying key challenges [27].

Mak and Yu analyzed various voice activity detection techniques for the National Institute of Standards and Technology (NIST) used in NIST’s speaker recognition evaluations, providing experimental data on their performance in noisy environments [28]. Burget et al. examined the role of feature extraction and channel compensation in GMM-based speaker recognition systems, proposing several methods for improving performance [29].

While the reviewed studies demonstrate significant advancements in speaker recognition and speech processing, there are notable limitations. Many of the approaches are heavily reliant on deep learning models, which can require large amounts of computational resources and data for training. For instance, the work by Wan et al. on Deep Speaker Embeddings [1] and Kang et al. on CNN-based speaker verification [2] showcase the effectiveness of DNNs but highlight the issue of data scarcity, which GANs partially address but do not completely resolve. Additionally, Snyder et al.’s [3] DNN embedding method significantly boosts performance but raises concerns about the model’s robustness in diverse real-world conditions. Chung and Nagrani’s work on the VoxCeleb2 dataset [4] emphasizes the importance of large datasets, but such resources are not always available, limiting the scalability of these solutions.

Moreover, traditional methods like SVM and GMM, though augmented by GAN-generated data [16,24,25], may lag behind deep learning models in performance, particularly in more complex environments. Lastly, although GANs have proven effective in data augmentation and noise reduction, as seen in SEGAN [18], their adversarial training process is sensitive to hyperparameters and can lead to instability in training, as noted by Pan et al. [23]. Therefore, while GANs and deep learning have driven significant progress, challenges such as model scalability, data dependency, and training stability remain.

3. GAN-Enhanced CNN for Voiceprint Recognition

The methodology described here proposes an algorithm combining both CNN and GAN in an innovative way to address the challenge of limited data in voiceprint recognition, named GAN-enhanced CNN for voiceprint recognition (GEC-VR). The CNN is used for classification, while the GAN model augments the dataset by generating new audio samples, thus enhancing the training process, as Figure 1 shows. The detailed description is as follows:

• Input:

• Audio samples (e.g., from Common Voice dataset);

• Number of original samples $N$;

• Number of GAN-generated samples $G$;

• CNN model initialized.

• Output:

• Trained CNN model;

• Accuracy metrics for voiceprint recognition;

• Algorithm Steps:

• 1.. Data Collection

  • Collect $N$ audio samples, ensuring diversity in gender, language, and voice characteristics.

  • Split the data into training and test sets, where $N=N_{train}+N_{test}$:

    •  . $N_{train}=50$;

    •  . $N_{test}=10$.

• 2.. Preprocessing

  • Convert all audio files into Waveform Audio File (WAV) format.

  • Extract MFCC from audio:

    •  . Use librosa.feature.mfcc to compute MFCC features.

    •  . Set $n_{\mathrm{m}\mathrm{f}\mathrm{c}\mathrm{c}}=13$, i.e., extract 13 feature variables.

  • Format the MFCC features to ensure consistent input length for CNN.

• 3.. CNN Model Construction

  • Initialize a CNN model with the following layers:

    •  . Convolutional layers: Use four convolutional layers with ReLU activation.

    •  . Pooling layers: Apply max-pooling after each convolutional layer.

    •  . Fully connected layers: Include three fully connected layers for classification.

    •  . Dropout layers: Introduce dropout between fully connected layers to prevent overfitting.

  • Compile the model using:

    •  . Loss function: sparse_categorical_crossentropy;

    •  . Optimizer: Adam.

• 4.. GAN Model Construction

  • Generator:

    •  . Generate new samples based on random noise (e.g., 600-dimensional noise).

  • Discriminator:

    •  . Classify the generated samples as real or fake.

  • Use a GAN training cycle to improve the quality of the generated audio samples.

• 5.. Training and Evaluation

  • Combine the original samples $N$ with the GAN-generated samples GGG.

  • Train the CNN model:

    •  . Perform forward propagation and backpropagation.

    •  . Train for a predefined number of epochs (e.g., 30 epochs).

  • Evaluate model accuracy using the test set.

  • Repeat the training cycle multiple times (e.g., 50 rounds) to ensure stable performance.

• 6.. Iterative GAN Reduction and Re-training

  • Reduce the number of GAN-generated samples by $∆$. ($∆$ can be determined by user, e.g., five).

  • Re-train the model and evaluate accuracy.

  • Repeat the reduction and re-training process for 10 cycles.

  • Track accuracy across different GAN sample sizes to determine the optimal balance between real and GAN-generated data.

• 7.. Final Evaluation

  • Record the final model’s accuracy metrics for voiceprint recognition.

  • Analyze the impact of GAN sample generation on model performance.

3.1. Data Collection and Audio Preprocessing

The diversity of voiceprint data is essential for training a model with strong general capabilities. This includes characteristics such as language, dialect, gender, age, and intonation. A diverse dataset helps the model maintain a high level of recognizability across various real-world application scenarios. By learning from different datasets, the model can extract detailed features and improve identification accuracy. In this study, data were collected from the Common Voice database [30], which provides a wide variety of voices in different languages and genders for research use. Users can also contribute by uploading their own voice samples. During the modeling process, 10 cases were collected, with each case containing 60 sound clips ranging from 3 to 4 s.

The dataset used in this study consists of voice samples collected from 10 unique cases, with 60 original samples per case. To address the challenge of limited data diversity, GANs were employed to augment the dataset. For each case, GANs generated an additional 40 synthetic samples, resulting in a total of 100 samples per case after augmentation. The final dataset size across all cases is therefore 1000 samples, comprising 600 original and 400 GAN-generated samples. This augmentation ensured a balanced representation of diverse voice characteristics, enhancing the robustness of the model training process.

The pre-processing phase involves manually tagging the voice data into distinct categories, as the downloaded audio is untagged. Once tagged, the data are stored in separate folders for later use. Before building the model, the collected audio must be categorized and stored in specific folders where the code will be executed. The next step involves converting all audio files into WAV format to ensure consistent data input for the model. The modeling was carried out using Google Colab, which provided a cloud-based runtime environment. Zero-padding was applied to standardize the length of all audio samples. This ensures uniform input dimensions for the CNN while preserving the original data integrity, avoiding truncation or distortion.

3.2. Feature Extraction—MFCC

Voiceprint recognition relies on extracting meaningful features from speech signals to identify individuals. In this process, feature extraction is crucial to the model’s performance. These features include attributes like fundamental frequency, intensity, timbre, and tempo of speech. Such features help the model capture the unique voice characteristics, enhancing the accuracy of recognition. Effective feature extraction not only improves the model’s ability to identify learned cases but also helps it generalize to new, unseen data, making the model more robust.

3.2.1. Audio Data Loading and the MFCC Characteristics Calculations

The ‘librosa.load’ function is used to load audio data, generating two primary outputs: an audio time series and the sample rate (the number of samples per second). The parameter ‘sr = 22,050 Hz’ was chosen for this study to determine the sample rate. This function decodes the audio file, making it ready for processing.

The ‘librosa.feature.mfcc’ function extracts the MFCC from the audio data. MFCC is a common feature used in speech recognition. For this study, the ‘n_mfcc’ parameter was set to 13, meaning that 13 MFCC feature variables were extracted from each audio sample.

3.2.2. Formatting

All extracted audio features were padded to the same length to prepare the data for CNN model training. Padding, rather than cropping, was used to ensure that no important features were lost. After padding, the data were reshaped into a four-dimensional array to meet the input requirements of the CNN. This process ensures that the features are complete and ready for training.

MFCCs were selected as the primary features for voiceprint recognition due to their established effectiveness in capturing speech characteristics. MFCCs simulate the human auditory system’s perception of sound frequencies, making them particularly well-suited for distinguishing vocal patterns. They are widely used in speech and speaker recognition tasks because of their ability to represent both static and dynamic speech information when combined with delta and delta-delta coefficients.

While MFCCs offer robust feature extraction, alternative features like linear predictive cepstral coefficients (LPCCs) also have potential for voiceprint recognition. LPCCs emphasize speech production characteristics and could complement MFCCs to capture additional vocal nuances. However, this study focused exclusively on MFCCs to maintain a targeted scope and streamline the experimental design. Future work will explore the integration of LPCCs and other advanced acoustic features to further enhance model performance and generalizability.

3.3. CNN Model Construction

The voiceprint recognition model was built using pre-processed audio data in a graphics processing unit (GPU) environment on Google Colab, which significantly accelerated the training process compared to using tensor processing units (TPUs) or central processing units (CPUs). Pre-processed audio data refer to audio signals that have undergone format conversion, resampling, segmentation, MFCC extraction, and padding to ensure their consistency and suitability for CNN-based classification. The model’s architecture relies on several key components to effectively process the audio features and make predictions.

The CNN used in this study is designed specifically for voiceprint recognition, leveraging MFCCs as input features. The model incorporates multiple convolutional and pooling layers to effectively extract and reduce the sampling rate of critical features from the MFCC representations. This architecture ensures the preservation of important voiceprint characteristics while reducing computational complexity.

The CNN consists of four convolutional layers, each followed by max-pooling layers to reduce feature dimensions and improve generalization. The convolutional layers use ReLU (rectified linear unit) as the activation function, which introduces non-linearity and ensures efficient gradient propagation during training. The number of filters in the convolutional layers progressively increases from 32 to 256, with a kernel size of 3 × 3 in all layers. These choices were determined through empirical testing to balance model complexity and accuracy.

The fully connected layers, placed after the convolutional and pooling stages, include three dense layers with 512, 256, and 7 neurons, respectively. The CNN architecture was originally designed with seven output nodes to accommodate a broader range of classes. For this study, the output layer was adjusted to five nodes to align with the dataset, which includes five speaker classes. This modification ensures consistency between the model and the dataset. Dropout layers with a rate of 0.5 are incorporated between the fully connected layers to prevent overfitting and ensure the model’s robustness across diverse datasets. The output layer uses a softmax activation function to classify voiceprints into distinct categories.

The model is optimized using the Adam optimizer, chosen for its adaptive learning rate capabilities, which help stabilize training and enhance convergence. The sparse categorical crossentropy loss function is used, as it is well-suited for multi-class classification tasks. Training is conducted over 30 epochs with a batch size of 32, parameters chosen based on performance tuning to balance efficiency and model accuracy.

This architecture and hyperparameter design ensure that the CNN is effective for voiceprint recognition, particularly in scenarios with limited and diverse datasets. By combining efficient feature extraction and robust classification capabilities, the model achieves high accuracy in recognizing distinct voiceprints.

3.4. Model Training and Evaluation

Once the model construction was complete, it was trained to learn patterns from the data. Afterward, its adaptability and accuracy were tested on a separate test dataset. Accuracy, defined as the ratio of correct predictions to total predictions, was evaluated over 10 rounds of training, and the best-performing model was selected for further use. A standard CNN architecture was chosen to ensure the reliability of the classification results. This allowed us to isolate and evaluate the contributions of the iterative reduction technique without introducing confounding variables from complex or proprietary CNN designs.

The model’s performance was optimized using the sparse_categorical_crossentropy loss function, which is particularly suited for multi-class classification tasks with integer labels. This loss function measures the difference between predicted and true labels, with the goal of minimizing this discrepancy to enhance the model’s accuracy. To achieve this, the Adam optimizer was employed, which combines the strengths of AdaGrad and RMSProp. By calculating the first and second moment estimates of the gradient, Adam provides greater stability and requires less tuning compared to traditional optimization algorithms like the stochastic gradient descent (SGD). The CNN used in this study is a well-established architecture for classification tasks. It serves as a robust tool for processing the combined dataset of real and GAN-generated samples, supporting the implementation of the iterative reduction technique.

The process of training the model involves two main phases: forward propagation and backpropagation. In forward propagation, data flow through the network from the input layer to the output layer, with each neuron calculating output values based on the current weights. After this step, backpropagation is used to adjust the weights and biases by computing the gradient of the loss function with respect to each weight. The model then uses this information to update the weights, aiming to minimize the loss by applying the gradient descent algorithm or the Adam optimizer.

Training is conducted over multiple cycles, called epochs, where the entire dataset passes through the model. Each epoch includes a complete forward and backpropagation cycle. In this study, 30 epochs were performed, meaning the model iterated over the data 30 times, refining its predictions and learning from the patterns in the data.

3.5. GAN Model Construction

GAN produces various samples through the trained generator and lets the discriminator determine whether the generated samples are real samples or fake samples; subsequently, the discriminator produces feedback. The audio quality generated by the adversarial generator gradually increases, and the discriminant ability of the discriminator gradually improves; this mechanism allows for samples generated by the GAN to have high-quality output. It can also help us improve the problem of low accuracy caused by insufficient samples when building voiceprint recognition models. The GAN model runs in the TPU environment of Colab, and the TPU is more efficient than the GPU and CPU for such a small but very large amount of data processing. To mitigate pattern collapse and instability, the Wasserstein GAN with gradient penalty (WGAN-GP) architecture was utilized. Additionally, hyperparameters such as learning rates and batch sizes were optimized, and diversity was monitored by visualizing and evaluating the statistical distribution of generated samples.

3.5.1. GAN Audio Preprocessing

Prior to training the GAN, audio samples are preprocessed to ensure consistency and compatibility with the model. The preprocessing steps are as follows:

1. Audio loading: the audio files are loaded using the torchaudio.load function, and the first channel is selected in case of multi-channel audio.

2. Standardized length: audio lengths are padded to match the longest sample in the dataset, ensuring uniformity.

3. Sample rate: a sample rate of 48,000 Hz is used to capture audio details effectively, as higher rates (e.g., 96,000 Hz) significantly increase computational costs with marginal gains in quality.

3.5.2. Generator Design

The GAN generator transforms random noise into synthetic audio samples through a series of layers, as follows:

1. Noise input: a 600-dimensional random noise vector serves as the input.

2. Layer expansion: the generator progressively increases the feature dimensions through fully connected layers, from 256 up to 4096 dimensions.

3. Activation functions:

• LeakyReLU is introduced after each intermediate layer to prevent vanishing gradients and ensure nonlinearity.

• Tanh is applied in the final layer to generate outputs in the range of [−1, 1], simulating real audio signals.

The generator’s design ensures high-quality audio generation while balancing computational efficiency.

3.5.3. Discriminator Design

The discriminator evaluates the authenticity of audio samples. Its structure mirrors the generator but operates in reverse, as follows:

1. Input dimension: takes a flattened input of 4096 dimensions (or reshaped as (64, 64, 1)).

2. Layer reduction: successively reduces the feature dimensions from 4096 to 256.

3. Activation functions:

• LeakyReLU maintains gradient flow for efficient learning.

• Sigmoid outputs a probability indicating whether a sample is real (close to 1) or synthetic (close to 0).

The discriminator and generator jointly refine their performance during adversarial training.

The generator’s goal is to trick the discriminator into judging the generated audio samples as real. Therefore, the loss of the generator is the cross-entropy between the output of the discriminator on the generated sample and the target label 1 (true). For the transformation process, the generator in the GAN model performs the following transformations. First, the initial noise vector (latent_dim = 600) serves as a random input to the generator. The generator applies multiple fully connected and convolutional layers to increase the dimensionality of the data, producing a flattened vector of size 4096. This output vector is reshaped into a matrix of dimensions (64, 64, 1), simulating a single-channel feature map. The GAN produces a flattened vector of size 4096. This output vector is reshaped into a matrix of dimensions (64, 64, 1) to match the input requirements of the CNN model, as Figure 2 shows.

This transformation process bridges the gap between the generator’s latent space and the input requirements of the CNN, enabling the end-to-end integration of GAN-augmented data into the classification pipeline.

3.5.4. GAN Reliability Test

After the GAN model is built, the next step is to generate audio, and use the voiceprint recognition model built earlier to test the accuracy. After 10 constructions and trainings, the models with the highest accuracy were compared one-by-one for further use.

One key aspect of GEC-VR is the iterative reduction of GAN-generated samples to find the optimal number of synthetic data points that improve the model’s performance without causing overfitting. By adjusting the number of GAN samples, the algorithm can be expected to ensure that the model generalizes well to unseen data.

Adaptive optimizers like Adam, combined with MFCC feature extraction, improve the efficiency and accuracy of training voiceprint recognition models by leveraging advanced signal processing and optimization techniques. However, careful monitoring of GAN over-reliance is necessary, as generating too many synthetic samples can degrade model performance. This algorithm provides a balanced and systematic approach that addresses real-world challenges in voiceprint recognition with limited and diverse datasets.

Traditional GAN evaluation metrics, such as Fréchet inception distance (FID), are designed for visual domains and lack standardization for evaluating synthetic audio. Consequently, this study assesses the GAN indirectly by analyzing its contribution to the voiceprint recognition system’s performance, measured through accuracy, precision, recall, and F1 score.

4. Experimental Studies

4.1. Data Processing

To ensure consistency during the evaluation, all original voice data were divided into a training set and a test set. This step was crucial to guarantee that accuracy changes in different cases could be evaluated without the randomness of the test set affecting the results. Each case dataset comprised 60 samples, with 50 samples allocated for training and 10 for testing. The analysis began with five cases: four male voices and one female voice. Table 1 shows the training and testing sample distribution for each case while Table 2 shows the confusion matrix of the texting set.

4.2. GAN Generation and Training

Taking Case 1 as an example, the 50 training samples were reduced to 10 samples. Using the remaining 10 samples, 60 new samples were generated by the GAN. After combining these 60 new samples with the old ones, the model was trained using the previously built CNN model, and the accuracy was tested after 50 rounds of repeated training. Table 3 presents the new sample count for Case 1 (10 original + 60 GAN-generated samples) compared with other cases, which retained 50 original samples. While 100% training accuracy is theoretically achievable, particularly in small or well-separated datasets, it warrants careful examination to rule out overfitting or bias. Metrics such as precision, recall, and F1 score are calculated per class and can show variations even when the overall accuracy is 100%. This occurs due to minor discrepancies in class-level performance. In this study, 100% accuracy was observed in specific cases due to several factors, as described below.

Dataset characteristics: The dataset size per case was relatively small (100 samples, including 60 real and 40 GAN-generated), and the samples within each case exhibited distinct voiceprint patterns. This led to highly separable classes in the training data.

Regularization techniques: To mitigate overfitting, dropout layers were incorporated into the CNN model, ensuring robust generalization. Additionally, GAN-generated samples added diversity to the training set, improving the model’s capacity to learn generalizable patterns.

Balanced training: by carefully tuning the ratio of real to synthetic samples, the model avoided overfitting to either data type, resulting in a stable training performance.

Confusion matrices were generated for each training step, showing the prediction results for each case. For instance, Table 2 shows that Case 1 had two misidentifications as Case 5, while Cases 2, 3, and 4 had no errors. Based on this, the accuracy for Case 1 was 80%, as outlined in Table 3, which summarizes the accuracy per training session. Table 4 shows the averaged accuracies of training on hybrid samples.

4.3. Repetitive Training Analysis

To more clearly observe the effects of different GAN sample sizes on accuracy, the number of GAN-generated samples was progressively reduced from 60 to 5, with each reduction followed by 50 training sessions. The average accuracy was recorded after each reduction. Table 5 illustrates how the accuracy varied depending on the number of GAN samples generated for Case 1. The results indicate that accuracy initially improved with more GAN samples but then declined, possibly due to the model overfitting to the generated data, as Figure 3 shows.

4.4. Amplification Case Analysis

Following the analysis of Case 1, the study expanded the experiment to include all five cases. After completing the analysis for Case 1, all GAN samples were removed, and Case 2 was analyzed using the same method (Table 6), followed by Cases 3, 4, and 5. The same process of reducing the sample size and generating GAN samples was repeated for each case, with 50 training sessions conducted for each combination of original and generated samples.

The results for all cases are presented in Table 7 and Figure 4. These results confirm that the improvement in accuracy due to GAN generation is more significant for cases with initially low accuracy. For cases where the initial accuracy was already high, GAN-generated samples offered little improvement. It is important to note that ‘Cases’ in Table 1 refer to speaker classes, while ‘Cases’ in Table 7 describe experimental configurations involving different ratios of GAN-generated to real samples.

The “Cases” presented in Table 8 correspond to experimental configurations designed to evaluate the impact of different GAN-to-real sample ratios and dataset sizes on the performance of the proposed voiceprint recognition model. Each case represents a unique combination of real and GAN-generated samples, as outlined below:

Baseline Case (Case 1): This case uses only real samples (60 per speaker) without any data augmentation, serving as the baseline for comparison.

Low GAN-to-Real Ratio (Cases 2–5): These cases incorporate increasing numbers of GAN-generated samples, ranging from 20% to 50% of the real dataset size. For example, Case 2 includes 12 GAN-generated samples per speaker (20%), while Case 5 includes 30 samples per speaker (50%).

High GAN-to-Real Ratio (Cases 6–9): These cases examine configurations with higher GAN-to-real ratios, with GAN samples constituting 60% to 90% of the dataset. For instance, Case 6 includes 36 GAN samples per speaker (60%), while Case 9 uses 54 GAN samples (90%).

Fully Augmented Dataset (Case 10): This case consists of 100% GAN-generated samples (60 samples per speaker) to evaluate the model’s performance when relying solely on synthetic data.

For all cases, the total number of samples per speaker was kept consistent at 100 to isolate the effects of the GAN-to-real sample ratio. The network configuration, including hyperparameters such as learning rate, batch size, and the number of training epochs, remained the same across all cases to ensure a fair comparison.

4.5. Decreasing Analysis

In cases where the improvement in accuracy after generating GAN samples was less than 5%, the study reduced the number of original samples. This reduction helped investigate whether the original high accuracy was the reason GAN-generated samples had little effect. Reducing the number of initial samples lowered the overall accuracy, which allowed the study to evaluate whether the GAN’s contribution was more noticeable with fewer original samples.

Table 9 shows the results for cases with an improvement of less than 5%. In these cases, reducing the number of original samples significantly increased the contribution of GAN-generated samples, confirming that the high initial accuracy limited the effect of GAN generation in previous tests.

4.6. Analysis Conclusions

The experiments verified that the use of GAN-generated samples significantly improves the performance of the CNN-MFCC voiceprint recognition model in cases with limited data, where the original accuracy is low. The number of GAN samples generated is also critical: blindly increasing the number of generated samples does not always lead to higher accuracy. Instead, selecting the right number of GAN-generated samples for each case can yield better results. Figure 5 illustrates the trend of accuracy (%) as the number of GAN-generated samples increases. Accuracy improves significantly up to 40 samples but declines beyond this point, indicating a potential overfitting effect with excessive synthetic data. This visualization complements the experimental results by showing how accuracy initially improves but later declines due to potential overfitting when the number of GAN-generated samples becomes excessive. The study evaluated various GAN-to-real sample mixing ratios to identify optimal configurations for practical applications. In resource-limited scenarios, a higher proportion of GAN-generated samples (e.g., 60–70%) is beneficial for enriching diversity, whereas a balanced ratio (30–50%) is preferable in data-rich environments to ensure robustness without overfitting.

5. Discussion

While GANs, CNNs, and MFCCs have been extensively studied as standalone components, their integration in this work addresses a pressing practical challenge in voiceprint recognition: mitigating overfitting and improving generalization in limited-data scenarios. The iterative reduction technique, which adjusts the balance between real and GAN-generated samples, provides a structured methodology to optimize model performance. This approach is particularly relevant for applications requiring robustness across diverse speakers and environments, where traditional data augmentation methods may fail to generalize. Our findings highlight the potential of this technique to enhance data diversity without compromising model stability, offering a reproducible framework for practitioners facing similar data limitations.

This research introduces a novel method for improving the precision of voiceprint recognition by utilizing GANs to generate synthetic audio samples. Integrating these GAN-produced samples with high-quality data effectively conquers the challenges posed by limited dataset size and diversity, which are crucial for effective voice recognition systems. The proposed GAN-enhanced CNN approach is well-suited for practical applications such as multilingual access control systems, automated customer service, and forensic speaker identification. Future research could explore advanced GAN architectures like StyleGAN for more realistic audio synthesis and adaptive sampling methods to optimize model performance across diverse datasets.

Our study successfully executed a hybrid data augmentation technique that maintains a careful equilibrium between synthetic and natural samples. This equilibrium notably enhanced the model’s ability to generalize across various voice samples, resulting in a robust method. Our tests revealed that incorporating GAN-generated samples enabled the model to reach higher accuracy, especially in scenarios with initially sparse real data, underscoring the importance of synthetic data for expanding voiceprint datasets without incurring excessive resource costs.

Further analysis revealed that, although GAN-generated data significantly boosted accuracy, their effectiveness varied based on context. When the initial accuracy was already high, adding synthetic data offered limited additional benefits, suggesting there is a point at which more data do not help much in data-rich situations. In situations where the baseline accuracy was low, adding synthetic samples led to considerable improvements, showcasing the effectiveness of GANs in environments with limited resources. The research also highlighted the necessity of controlling the volume of synthetic data to prevent overfitting, a situation where the model becomes too specialized to the training data and performs poorly on new, unseen data, identifying an ideal range that enhances accuracy while allowing the model to generalize well.

The approach and results of this research offer a hopeful framework for voiceprint recognition technologies. These technologies can gain advantages from flexible and resource-efficient methods of data augmentation. Future research opportunities include enhancing GAN architectures to produce even more lifelike audio and investigating adaptive sampling methods, such as stratified sampling or active learning, to improve model performance across various voiceprint situations.

Author Contributions

Conceptualization, M.-L.H. and H.-Y.C.; methodology, Y.-S.L. and H.-Y.C.; software, T.-Y.H.; validation, Y.-S.L. and T.-Y.H.; writing—original draft preparation, Y.-S.L. and H.-Y.C.; writing—review and editing, M.-L.H. and H.-Y.C.; supervision, M.-L.H. and Y.-S.L. All authors have read and agreed to the published version of the manuscript.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to the funding protocol of National Science and Technology Council.

Conflicts of Interest

The authors declare no conflicts of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. Algorithm flowchart.

Figure 2. CNN architecture. The output layer was configured with 5 nodes to classify the 5 speaker classes used in this study. (Left) Illustration of the CNN architecture used in the study. The input dimension (64, 64, 1) represents the reshaped MFCC feature matrix derived from audio samples, while the output value of 7 corresponds to the number of speaker classes in the classification task. (Right) GAN model training process for voiceprint recognition.

Figure 3. Accuracy comparison chart for Case 1.

Figure 4. Trend chart for all dataset accuracies.

Figure 5. Trend of accuracy with increasing numbers of GAN-generated samples.

Table 1

Number of training and testing samples of each case dataset. In this table, ‘Cases’ denote individual speaker classes in the dataset (e.g., Case 1 corresponds to Speaker A, Case 2 to Speaker B, etc.).

Datasets	Training Samples	Testing Samples
Case 1	50	10
Case 2	50	10
Case 3	50	10
Case 4	50	10
Case 5	50	10

Note: The CNN was originally designed with seven output nodes to accommodate potential future expansions. For this study, the architecture was adjusted to five output nodes to match the dataset, which included five speaker classes (four male and one female).

Table 2

Confusion matrix of testing set.

Actual Category/Forecast Results	Case 1	Case 2	Case 3	Case 4	Case 5
Case 1	8	0	0	0	2
Case 2	0	10	0	0	0
Case 3	0	0	10	0	0
Case 4	0	0	0	10	0
Case 5	0	0	2	0	8

Note: The CNN was originally designed with seven output nodes to accommodate potential future expansions. For this study, the architecture was adjusted to five output nodes to match the dataset, which included five speaker classes (four male and one female).

Table 3

Performance evaluation based on Table 2, incorporating precision, recall, and F1 score metrics. Precision, recall, and F1 score are calculated per class and can show variations even when the overall accuracy is 100%. This occurs due to minor discrepancies in class-level performance.

Datasets (Case)	Precision (%)	Recall (%)	F1-Score (%)
1	80.0	80.0	80.0
2	100	100	100
3	83.3	100	90.9
4	100	100	100
5	80.0	80.0	80

Note: The CNN was originally designed with seven output nodes to accommodate potential future expansions. For this study, the architecture was adjusted to five output nodes to match the dataset, which included five speaker classes (four male and one female).

Table 4

Averaged accuracies of training on hybrid samples (real data and generated samples) for 50 independent experiments.

Datasets	Number of Training Samples	Number of Testing Samples	Averaged Accuracy of 50 Experiments
Case 1	10 + 60 GAN	10	75.6%
Case 2	50	10	100%
Case 3	50	10	95.2%
Case 4	50	10	98%
Case 5	50	10	91.2%

Note: The CNN was originally designed with seven output nodes to accommodate potential future expansions. For this study, the architecture was adjusted to five output nodes to match the dataset, which included five speaker classes (four male and one female).

Table 5

Number of generated samples and corresponding average accuracy for Case 1.

Case 1: Number of Samples	Averaged Accuracy of 50 Independent Experiments (%)
10 (initial template)	96.6
15 (5 GAN)	52.8
20 (10 GAN)	81.6
25 (15 GAN)	94.6
30 (20 GAN)	92.2
35 (25 GAN)	93
40 (30 GAN)	96.6
45 (35 GAN)	92.2
50 (40 GAN)	69.4
55 (45 GAN)	86
60 (50 GAN)	86.4

Table 6

Number of training and testing samples for Case 2.

Datasets	Number of Training Samples	Number of Testing Samples
Case 1	50	10
Case 2	10 + 60 GAN	10
Case 3	50	10
Case 4	50	10
Case 5	50	10

Note: The CNN was originally designed with seven output nodes to accommodate potential future expansions. For this study, the architecture was adjusted to five output nodes to match the dataset, which included five speaker classes (four male and one female).

Table 7

Accuracy of generated training for each case. In this table, ‘Cases’ refer to experimental configurations where the proportion of GAN-generated samples to real samples is systematically varied.

Number of Samples	Case 1	Case 2	Case 3	Case 4	Case 5
10 (initial data)	96.6%	99.6%	72.6%	71.8%	96.5%
25 (15 GAN)	52.8%	95%	42.8%	39.2%	48.4%
30 (20 GAN)	81.6%	98.2%	56.6%	58.2%	75.8%
35 (25 GAN)	94.6%	97.4%	69.8%	78%	83.8%
40 (30 GAN)	92.2%	99.6% (0%)	65.8%	66.8%	99% (2.5%)
45 (35 GAN)	93%	99.4%	72.4%	61%	98.8%
50 (40 GAN)	96.6% (0%)	98%	76.8% (4.2%)	81.2% (9.4%)	93.6%
55 (45 GAN)	92.2%	99%	68.4%	72.8%	98.6%
60 (50 GAN)	69.4%	96.4%	56.8%	47.8%	66.4%
65 (55 GAN)	86%	98.2%	68%	70.6%	84.2%
70 (60 GAN)	86.4%	98.8%	72.2%	70.2%	91.8%

Table 8

Accuracy table over Case 1 to Case 10. In this table, ‘Cases’ represent experimental configurations with varying proportions of GAN-generated samples to real samples.

Case	Case 1	Case 2	Case 3	Case 4	Case 5	Case 6	Case 7	Case 8	Case 9	Case 10
10 (initial data)	79.4%	97%	69.8%	55.2%	83%	45.3%	96.4%	70%	74.3%	93%
25 (10 + 15)	78.8%	95.2%	74.2%	63.6%	87.4%	50.8%	97.4%	74.8%	83.4%	95.4% (2.4%)
30 (10 + 20)	81.4% (2%)	97.6%	79.8% (10%)	60.4%	89.2%	48.6%	96.4%	74%	81.6%	93.4%
35 (10 + 25)	58.6%	97%	66.6%	52.8%	75.6%	52%	95.4%	75.4%	83.6%	86.4%
40 (10 + 30)	74.2%	94%	69.4%	65.4%	79.8%	45.4%	84%	70.2%	58%	89.8%
45 (10 + 35)	66%	98.6% (1.6%)	68.8%	64.6%	90% (7%)	50.4%	95.2%	70%	72%	92.2%
50 (10 + 40)	76.2%	96.2%	69.6%	60.6%	87.6%	52.8%	90.2%	74.2%	77%	91.2%
55 (10 + 45)	75.4%	94%	77%	58.2%	84.8%	54.8% (9.5%)	95.8%	71.8%	88.8% (15.5%)	95.2%
60 (10 + 50)	74.6%	98.4%	76.6%	65.6% (10.4%)	82.8%	50.6%	98% (1.6%)	76.6% (6.6%)	80%	92.8%
65 (10 + 55)	65.2%	95%	45.2%	48.6%	65.2%	50.2%	97%	73.4%	79.4%	81.6%
70 (10 + 60)	64.2%	96.4%	71.2%	57%	83.6%	50%	97.4%	74%	85.8%	89.8%

Table 9

Accuracy table for Cases 1, 2, 7, and 10.

Number of Samples	Case 1	Case 2	Case 7	Case 10
5 (initial data)	47.8%	89.4%	91.2%	63.8%
20 (15 GAN)	56.2% (8.4%)	87.6%	85%	70.6%
25 (20 GAN)	63.4%	87.8%	88.2%	61.4%
30 (25 GAN)	64.2%	88.6%	92%	72.4% (8.6%)
35 (30 GAN)	46.4%	90%	91.2%	51.8%
40 (35 GAN)	53.8%	93%	97.2% (6%)	61.2%
45 (40 GAN)	63%	91.8%	88%	71.8%
50 (45 GAN)	51.4%	87.4%	86.8%	69.4%
55 (50 GAN)	59.6%	94.6% (5.4%)	92%	68.8%
60 (55 GAN)	61.8%	92.8%	91.6%	67.6%
65 (60 GAN)	47.4%	88.4%	87.8%	40.6%

References

1. Wan, L.; Wang, Q.; Papir, A.; Moreno, I.L. Generalized end-to-end loss for speaker verification. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); Calgary, AB, Canada, 15–20 April 2018; pp. 4879-4883.

2. Kang, Y.; Cai, Z.; Tan, C.-W.; Huang, Q.; Liu, H. Natural language processing (NLP) in management research: A literature review. J. Manag. Anal.; 2020; 7, pp. 139-172. [DOI: https://dx.doi.org/10.1080/23270012.2020.1756939]

3. Snyder, D.; Garcia-Romero, D.; Povey, D.; Khudanpur, S. Deep neural network embeddings for text-independent speaker verification. Proceedings of the Interspeech; Stockholm, Sweden, 20–24 August 2017; pp. 999-1003.

4. Chung, J.S.; Nagrani, A.; Zisserman, A. Voxceleb2: Deep speaker recognition. arXiv; 2018; arXiv: 1806.05622

5. Lei, Y.; Scheffer, N.; Ferrer, L.; McLaren, M. A novel scheme for speaker recognition using a phonetically-aware deep neural network. Proceedings of the 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); Florence, Italy, 4–9 May 2014; pp. 1695-1699.

6. Ravanelli, M.; Bengio, Y. Speaker recognition from raw waveform with sincnet. Proceedings of the 2018 IEEE Spoken Language Technology Workshop (SLT); Athens, Greece, 18–21 December 2018; pp. 1021-1028.

7. Heigold, G.; Moreno, I.; Bengio, S.; Shazeer, N. End-to-end text-dependent speaker verification. Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); Shanghai, China, 20–25 March 2016; pp. 5115-5119.

8. Hinton, G.; Deng, L.; Yu, D.; Dahl, G.E.; Mohamed, A.-r.; Jaitly, N.; Senior, A.; Vanhoucke, V.; Nguyen, P.; Sainath, T.N. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Process. Mag.; 2012; 29, pp. 82-97. [DOI: https://dx.doi.org/10.1109/MSP.2012.2205597]

9. Davis, S.; Mermelstein, P. Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Trans. Acoust. Speech Signal Process.; 1980; 28, pp. 357-366. [DOI: https://dx.doi.org/10.1109/TASSP.1980.1163420]

10. Rabiner, L.R.; Juang, B.-H. Fundamentals of Speech Recognition; Tsinghua University Press: Beijing, China, 1999.

11. Muda, L.; Begam, M.; Elamvazuthi, I. Voice recognition algorithms using mel frequency cepstral coefficient (MFCC) and dynamic time warping (DTW) techniques. arXiv; 2010; arXiv: 1003.4083

12. Logan, B. Mel frequency cepstral coefficients for music modeling. Proceedings of the ISMIR; Plymouth, MA, USA, 23–25 October 2000; 11.

13. Kim, C.; Stern, R.M. Power-normalized cepstral coefficients (PNCC) for robust speech recognition. IEEE/ACM Trans. Audio Speech Lang. Process.; 2016; 24, pp. 1315-1329. [DOI: https://dx.doi.org/10.1109/TASLP.2016.2545928]

14. Wang, C.; Zhang, M.; Xu, Z.-W.; Chen, C.; Sheng, D.-S. Effects of anisotropic ionospheric irregularities on space-borne SAR imaging. IEEE Trans. Antennas Propag.; 2014; 62, pp. 4664-4673. [DOI: https://dx.doi.org/10.1109/TAP.2014.2333055]

15. Alkhatib, B.; Eddin, M.M.K.; Syria, D. ASR Features Extraction Using MFCC And LPC: A Comparative Study. J. Digit. Inf. Manag.; 2023; 21, 39. [DOI: https://dx.doi.org/10.6025/jdim/2023/21/2/39-49]

16. Saito, Y.; Takamichi, S.; Saruwatari, H. Statistical parametric speech synthesis incorporating generative adversarial networks. IEEE/ACM Trans. Audio Speech Lang. Process.; 2017; 26, pp. 84-96. [DOI: https://dx.doi.org/10.1109/TASLP.2017.2761547]

17. Wang, S.; Yang, Y.; Wu, Z.; Qian, Y.; Yu, K. Data augmentation using deep generative models for embedding based speaker recognition. IEEE/ACM Trans. Audio Speech Lang. Process.; 2020; 28, pp. 2598-2609. [DOI: https://dx.doi.org/10.1109/TASLP.2020.3016498]

18. Pascual, S.; Bonafonte, A.; Serra, J. SEGAN: Speech enhancement generative adversarial network. arXiv; 2017; arXiv: 1703.09452

19. Sisman, B.; Yamagishi, J.; King, S.; Li, H. An overview of voice conversion and its challenges: From statistical modeling to deep learning. IEEE/ACM Trans. Audio Speech Lang. Process.; 2020; 29, pp. 132-157. [DOI: https://dx.doi.org/10.1109/TASLP.2020.3038524]

20. Kaneko, T.; Kameoka, H.; Hiramatsu, K.; Kashino, K. Sequence-to-Sequence Voice Conversion with Similarity Metric Learned Using Generative Adversarial Networks. Proceedings of the Interspeech; Stockholm, Sweden, 20–24 August 2017; pp. 1283-1287.

21. Chandna, P.; Miron, M.; Janer, J.; Gómez, E. Monoaural audio source separation using deep convolutional neural networks. Proceedings of the Latent Variable Analysis and Signal Separation: 13th International Conference, LVA/ICA 2017; Grenoble, France, 21–23 February 2017; Proceedings 13 pp. 258-266.

22. Skerry-Ryan, R.; Battenberg, E.; Xiao, Y.; Wang, Y.; Stanton, D.; Shor, J.; Weiss, R.; Clark, R.; Saurous, R.A. Towards end-to-end prosody transfer for expressive speech synthesis with tacotron. Proceedings of the International Conference on Machine Learning; Stockholm Sweden, 10–15 July 2018; pp. 4693-4702.

23. Pan, Q.; Pan, Y.; Zhou, J.; Wang, H.; Tao, L.; Kwan, H.K. Cyclegan with dual adversarial loss for bone-conducted speech enhancement. Proceedings of the TENCON 2022—2022 IEEE Region 10 Conference (TENCON); Hong Kong, 1–4 November 2022; pp. 1-4.

24. Campbell, W.M.; Sturim, D.E.; Reynolds, D.A. Support vector machines using GMM supervectors for speaker verification. IEEE Signal Process. Lett.; 2006; 13, pp. 308-311. [DOI: https://dx.doi.org/10.1109/LSP.2006.870086]

25. Reynolds, D.A.; Quatieri, T.F.; Dunn, R.B. Speaker verification using adapted Gaussian mixture models. Digit. Signal Process.; 2000; 10, pp. 19-41. [DOI: https://dx.doi.org/10.1006/dspr.1999.0361]

26. Bimbot, F.; Bonastre, J.-F.; Fredouille, C.; Gravier, G.; Magrin-Chagnolleau, I.; Meignier, S.; Merlin, T.; Ortega-García, J.; Petrovska-Delacrétaz, D.; Reynolds, D.A. A tutorial on text-independent speaker verification. EURASIP J. Adv. Signal Process.; 2004; 2004, 101962. [DOI: https://dx.doi.org/10.1155/S1110865704310024]

27. Kinnunen, T.; Li, H. An overview of text-independent speaker recognition: From features to supervectors. Speech Commun.; 2010; 52, pp. 12-40. [DOI: https://dx.doi.org/10.1016/j.specom.2009.08.009]

28. Mak, M.-W.; Yu, H.-B. A study of voice activity detection techniques for NIST speaker recognition evaluations. Comput. Speech Lang.; 2014; 28, pp. 295-313. [DOI: https://dx.doi.org/10.1016/j.csl.2013.07.003]

29. Burget, L.; Matejka, P.; Schwarz, P.; Glembek, O.; Cernocky, J.H. Analysis of feature extraction and channel compensation in a GMM speaker recognition system. IEEE Trans. Audio Speech Lang. Process.; 2007; 15, pp. 1979-1986. [DOI: https://dx.doi.org/10.1109/TASL.2007.902499]

30. Mozilla. Common Voice. Available online: https://commonvoice.mozilla.org/en/datasets (accessed on 2 May 2024).