CNN 1D: A Robust Model for Human Pose Estimation

Abstract

The purpose of this research is to develop an efficient model for human pose estimation (HPE). The main limitations of the study include the small size of the dataset and confounds in the classification of certain poses, suggesting the need for more data to improve the robustness of the model in uncontrolled environments. The methodology used combines MediaPipe for the detection of key points in images with a CNN1D model that processes preprocessed feature sequences. The Yoga Poses dataset was used for the training and validation of the model, and resampling techniques, such as bootstrapping, were applied to improve accuracy and avoid overfitting in the training. The results show that the proposed model achieves 96% overall accuracy in the classification of five yoga poses, with accuracy metrics above 90% for all classes. The implementation of the CNN1D model instead of traditional 2D or 3D architectures accomplishes the goal of maintaining a low computational cost and efficient preprocessing of the images, allowing for its use on mobile devices and real-time environments.

1. Introduction

Human pose estimation (HPE) is a vision and artificial intelligence technique that focuses on detecting and analyzing the position and orientation of human body joints from images or video sequences. To do this, reference points are identified and located in joints such as the hip, head, and shoulders, thus capturing an interpretation of the human figure. These reference points are called keypoints or landmarks [1].

Currently, several methods have been proposed for HPE, among them GraphMLP [2], which implements Graph Convolutional Networks (GCNs) for 3D detection, as well as the use of heat maps through architectures such as HRNet, which track joint points and calculate the probability of movement [3]. In addition, inertial sensors have been used to capture sequential patterns and perform pose estimation using algorithms such as Multi-Layer Perceptron (MLP) [4] and Long Short-Term Memory networks (LSTM) [5]. These techniques aim to optimize predictions by reducing the necessary amount of data for training or by using 2D predictions to estimate the pose in 3D.

BlazePose has been extensively utilized for human pose estimation in diverse applications, such as repetitive action counting and movement analysis, demonstrating its effectiveness in capturing key body landmarks [6]. Unlike other methods, BlazePose is specifically optimized for efficient landmark detection, providing accurate estimations while reducing computational complexity [7].

On the other hand, the research by [8] introduces an image-free segmentation technique based on the detection of individual pixels, reducing the amount of input data by two orders of magnitude and improving efficiency in data acquisition and processing. Other solutions include the use of multi-view matching algorithms [9], as well as geometric and parametric techniques that aim to correct erroneous estimates and handle occlusions [10]. To work with landmarks or keypoints, frameworks such as MoveENet [11] and PoseNet [12] have been developed. However, these solutions tend to require large volumes of data in both training and validation processes, in addition to having robust but computationally inefficient feature extraction methods for each input image [13].

In this research, a solution for HPE using BlazePose as a feature ejector is proposed, which eliminates the need for extended preprocessing, characteristic of other conventional methods. In comparison with previous studies that had used 3D convolutional networks with attention mechanisms [14,15], which improve the precision but requires a greater computing capacity, the combination of BlazePose and CNN1D in this proposal allows us to reduce the computational burden without compromising the precision. Alternatively, some investigations had employed CNN1D for determining recognition in EEG signals [16], optimizing computational efficiency, despite requiring a specific selection of features and channels.

The solution presented in this paper is based on the efficient features extraction through BlazePose, combined with CNN1D for processing temporal data, which allows for a lighter implementation, adaptable to devices with limited resources and adequate for real-time applications, where the balance between computational precision and efficiency is a key factor.

2. Related Work

In the field of HPE, solutions have been developed focused on improving computational efficiency without compromising the accuracy of predictions. The BlazePose and OpenPose models were analyzed in the detection and correction of postures, evaluating their performance in real-time applications and its integration with supervised learning algorithms.

MediaPipe has the BlazePose module, which has proven to have a low computational cost and the ability to operate in real time [17]. Although it has limitations in terms of accuracy compared to more complex models, such as OpenPose [18], BlazePose is even more efficient in resource-limited environments, showing better performance in speed and compatibility with algorithms of Supervised Learning (SL) such as Random Forest and Support Vector Machine (SVM) for physical activity tracking [19,20]. These features make it an efficient solution for real-time HPE.

A relevant aspect of using BlazePose is the incorporation of the Z axis for detecting the inclination of the human body; with minimal adjustments in the speed threshold, it is possible to increase the precision up to 90% [21], although this implies an increase in the amount of data to be processed and the characteristics to be extracted [13]. In real-time execution environments, where it is necessary to correct postures during physical exercises [22,23], even with multiple people, BlazePose has shown adequate tracking, focusing on the closest person to the camera to ensure the correct capturing of body patterns and error correction [24].

In the field of recognizing emotions in speech, deep neural networks have been used with attention mechanisms [21,25], maintaining stable precision, although with a greater computational load. The pose detection with a single camera has been studied for the classification of physiotherapy exercises [26], showing viability in this context, although with optimized models for specific tasks [7]; moreover, architectures that involve Transformers and CNN in parallel [14], applied to the recognition of emotions in voice, demonstrate good results in temporary feature extraction, but with a greater requirement of data and processing [27].

Finally, BlazePose has been used to provide real-time feedback on a person’s position and movement, with an accuracy of around 90% in Increased Reality (IR) environments [28]. As an alternative to improve latency, other frameworks such as OpenCV [29] or the MediaPipe feature extractor [30] have been used, achieving accuracies of up to 95%; however, these proposals impact the preprocessing of the images before being interpreted by the model [22]. The combination of OpenPose with Azure Kinect using the KinOp tool [31] has proven to be an accurate solution for capturing interactions in natural settings, being effective in correcting postures in real time [21], although difficulties may arise in complex scenarios or with multiple individuals.

The studies analyzed agreed that MediaPipe and BlazePose offer efficient and low computational cost solutions for real-time applications; although BlazePose presents limitations in terms of precision compared to more complex models such as OpenPose, its speed makes it an ideal option for applications that require rapid feedback, such as tracking physical activities or correcting postures in uncontrolled environments. Solutions that combine MediaPipe with machine learning algorithms, such as Random Forest [32], SVM [33] and LSTM [34], have proven effective in real-time posture detection and body tracking. Additionally, recent improvements such as Z axis correction and occlusion mitigation have significantly increased accuracy in 3D scenarios and those involving several people.

3. Materials and Methods

This work proposes a methodology for the real-time detection of human postures. The system can identify and predict postures in uncontrolled environments, which improves its adaptability to real situations and increases its generalization capacity. The methodology is based on CNN1D and computer vision to extract the key points of the body. Figure 1 describes the implemented methodology to develop the proposed system.

3.1. Data Acquisition

The database used in this research consists of five main categories: down dog, goddess, plank, tree, and warrior II. Each category contains approximately 300 images, with variations ranging between 50 and 75 additional samples per label. This dataset includes photographs and images from different angles and with variations in poses, as shown in Figure 2. The images were selected from different public repositories to ensure a wide variety of contexts and situations. Additionally, images containing two people and generic noise elements, such as letters or trees, that were included to enhance the robustness of the model. The dataset used in this research is publicly available at the following link: Yoga Poses dataset (https://www.kaggle.com/datasets/niharika41298/yoga-poses-dataset/data accessed on 15 July 2024).

3.2. Feature Extraction

For the processing of the images, first, each pose of the dataset is iterated, which in this case is composed of five poses, assigning each one its corresponding label in a dictionary: ‘downdog’: 0, ‘goddess’: 1, ‘plank’: 2, ‘tree’: 3, ‘warrior2’: 4. Each image is then transformed from Blue, Green, and Red (BGR) to Red, Green, and Blue (RGB) format for proper processing using the MediaPipe and BlazePose libraries. The latter uses a deep learning model to detect reference points, known as landmarks, on the human body [35]. The process is divided into two stages, as shown in Figure 3.

First, a person detection model is used to identify the presence of a human figure in the image. Once detected, a second more detailed model is used that extracts a set of 33 landmarks, which correspond to specific anatomical locations, such as shoulders, elbows, and knees, among others [36]. Each landmark $L_i$ is composed of four coordinates: $(x_i,y_i,z_i,v_i)$, as shown in Equation (1), where $x_i$ and $y_i$ are horizontal and vertical coordinates, respectively, normalized in the range $[0,1]$; $z_i$ is the relative depth coordinate, also normalized, and $v_i$ represents the visibility or confidence of landmark i, with values between 0 and 1, which indicate the probability of correct detection of the point [37].

(1)$L_i=(x_i,y_i,z_i,v_i).$

To analyze the relationship between different landmarks, 3D Euclidean distances between two reference points $L_i$ and $L_j$ are calculated using Equation (2).

(2)$d_{i,j}=\sqrt{{(x_i-x_j)}^2+{(y_i-y_j)}^2+{(z_i-z_j)}^2}.$

Additionally, the angle θ between three landmarks $L_i$, $L_j$ and $L_k$ is calculated using the cosine of the angle, defined in Equation (3):

(3)$\theta =arccos\left({\displaystyle \frac{\overrightarrow{u}·\overrightarrow{v}}{|\overrightarrow{u}|·|\overrightarrow{v}|}}\right),$

where the vectors $\overrightarrow{u}$ and $\overrightarrow{v}$ are the coordinate differences between the reference points, defined in Equations (4) and (5):

(4)$\overrightarrow{u}=(x_j-x_i,y_j-y_i,z_j-z_i),$

(5)$\overrightarrow{v}=(x_k-x_i,y_k-y_i,z_k-z_i).$

Once the landmarks have been extracted from each image, they are normalized to ensure consistency across samples. This normalization process involves scaling the x, y, and z coordinates to fall within a predefined range, typically $[0,1]$. The visibility values $v_i$ are retained as is, as they already fall within this range. This step minimizes the impact of variations in image resolution, camera perspective, or pose execution, ensuring that the features extracted are robust and comparable.

Following the normalization, a sequence is generated based on the patterns of the coordinates obtained. These are stored in files containing the landmarks, which are loaded and their information added to a sequence list. Furthermore, the label corresponding to the pose class is assigned, which is incorporated into another list, thus establishing a relationship between the extracted features and the assigned label.

3.3. K-Fold Strategy and Bootstrapping

To evaluate the model’s  performance in a robust way and take full advantage of the available dataset, a combination of cross-validation with K-folds and bootstrap resampling was applied to the training set. The K-fold cross-validation technique divided the dataset in approximately five equal splits; in each iteration, one of the subassemblies was used as trial set, while the remaining were used as training set [38]. This process, which ensures that all the samples are going to be used both for training and evaluation, allowed us to reduce the bias associated with the unique partition and to obtain a more representative model performance estimation [39].

Due to the reduced dataset size, the bootstrap resampling was used to increase the amount of samples available in the training set. This statistic technique generated a broaden dataset through the random selection with the original selection replacement, increasing a 75% in the size; such an increase introduced greater variability in the training set, replying the original observations but guaranteeing the diversity in the samples. Formally, $X=x_1,x_2,\dots ,x_n$ is the original dataset with n observations. Starting with X, B samples are generated with the replacement, each one of size n, defined as indicated in the Equation (6).

(6)$X_b^{*}=x_{b,1}^{\prime }{x}_{b,2}^{\prime }\dots ,x_{b,n}^{*}for b=1,2,\dots ,B,$

In this procedure, each observation could be selected more than once, which allowed for the generation of a wider and more diverse training set. However, this resampling could also introduce duplications among the training and trial set [40], which could bias the evaluation metrics, allowing the model to see data previously used for training. To avoid this pollution, a strict control was implemented that identified and removed duplicated observations currently in both partitions. This additional control guaranteed that the model would not have previous access to the used data for the evaluation, preserving the independence among both sets. Likewise, due to the initial 75% increase in the training set, the duplicated removal did not significantly affect the diversity of the training set, which allowed for robust learning without compromising the validity of the obtained metrics.

Incorporating the K-fold cross-validation and the bootstrap resampling, it was possible to maximize the use of available data, reduce possible biases in the evaluation of the model, and improve its capacity to generalize new observations. The careful elimination of duplicates was essential to maintain the reliability of the results and avoid an overestimation in the model’s  performance.

3.4. Convolutional Architecture Proposal

From the perspective of computer vision, human pose estimation is a complex task, especially regarding the training and validation of deep learning models such as convolutional neural networks [41]. This complexity comes from image analysis [42], which can be implemented using decoders [43] or through information augmentation techniques based on a single image [44]. To face this challenge, a CNN1D model was chosen, which reduces the computational load [45], using only the key points generated by the BlazePose module of the MediaPipe framework [17]. This allows specific features of the human figure to be extracted in the Region Of Interest (ROI) [7]. Body segmentation is performed to mark the coordinates of the reference points, generating a sequential pattern that ensures that the training of the model not only depends on the quality of the images or the present noise, but also focuses on the relevant information to the posture estimation.

The InputLayer of the CNN1D is a one-dimensional vector $\mathbf{x}\in {\mathbb{R}}^L$, where L is the length of the sequence or number of elements in the vector, resulting from the preprocessing of the dataset. Each element of the sequence can be a feature or value in a time series, defined in Equation (7).

(7)$\mathbf{x}=[x_1,x_2,\dots ,x_L].$

In the 1D convolutional layers, a kernel (k) of size $k=3$ was used, so in each convolution the kernel spans three consecutive values of the sequence. The number of filters applied is 128, 64, and 32, respectively, which means that each filter operates on the feature maps resulting from the previous layer. Each kernel can detect different present patterns in the input. For each kernel $w\in {\mathbb{R}}^3$, a convolution is performed on the input sequence, and each kernel $w_i$ at position j is calculated according to Equation (8), where $h_j$ is the resulting value of the feature map at position j, $x_{j+i-1}$ represents the values of the input sequence, and b is the bias term applied to the filter.

(8)$h_j=\sum _{i=1}^3{w}_i·x_{j+i-1}+b.$

In each convolutional layer, the ReLU activation function was used: ReLU: $h_{j,\mathit{ReLU}}=max(0,h_j)$, where if $h_j$ is negative, it returns 0; otherwise, it returns $h_j$. In this way, the network learns more complex representations by maintaining only positive activations.

Dropout was applied with a rate of 0.5, meaning that 50% of the neurons in the previous layer are randomly turned off, setting them to zero during training. If p is the probability of keeping a neuron activated, in this case, $p=0.5$, then the resulting value of neuron $h_j$ after applying Dropout is expressed in Equation (9). This process randomly turns off a fraction of the neurons, preventing overfitting during training.

(9)$h_{j,\mathit{dropout}}=\left\{\begin{array}{cc}{h}_j,\hfill & \mathit{with}\mathit{probability}p\hfill \\ 0,\hfill & \mathit{with}\mathit{probability}1-p.\hfill \end{array}\right.$

MaxPooling1D layers were applied to the outputs of the convolutional layers to reduce the dimensionality using a window of size 2. In each step, for the maximum value of a segment of the input sequence that spans two consecutive elements, if the input is $h=[h_1,h_2,h_3,\dots ,h_n]$, the output p is defined as $p_j=max(h_{2j-1},h_{2j}),j=1,2,\dots ,⌈{\displaystyle \frac{n}{2}}⌉$. In this way, the length of the output is reduced by approximately half, which preserves the most important characteristics and reduces the computational load.

The flatten layer converts the output tensor into a one-dimensional vector, preparing it for the next dense layer. If the output tensor of the last convolutional layer has dimensions $(d_1,d_2,\dots ,d_k)$, where $d_i$ are the dimensions of each feature, the flattening is expressed in Equation (10), where H is the output tensor and $\mathbf{f}$ is the resulting one-dimensional vector, which contains all the elements of H in a single array. This process allows the data to be used efficiently in the subsequent dense layer.

(10)$\mathbf{f}=\mathrm{Flatten}\left(H\right)=[h_1,h_2,\dots ,h_{d_1·d_2·\dots ·d_k}].$

For the output, a dense layer was implemented in which the number of neurons is set equal to the number of classes to be classified. The Softmax function is used to transform the logits into probabilities. For each neuron i, the output is calculated according to Equation (11), where ${\widehat{y}}_i$ is the probability that the input belongs to class i and $z_i$ represents the logit associated with said class. This function ensures that the outputs are interpreted as probabilities and that the addition of these for all classes is equal to 1.

(11)${\widehat{y}}_i={\displaystyle \frac{exp\left(z_i\right)}{{\sum }_{j=1}^{k}exp\left(z_j\right)}}.$

3.5. Real-Time Detection

For real-time detection of human poses, MediaPipe is used due to its lower computational complexity compared to frameworks such as OpenPose [46]. This makes it especially suitable for low-power devices, such as mobile phones or embedded devices. MediaPipe has proven to be effective in detecting people, even in variable lighting conditions or with occlusions in the input image [47].

To capture the input images, the UGREEN USB HD webcam with a resolution of 1080 p at 30 FPS was used. Figure 4 shows how the model performs detection of a person with a resolution of 1080 × 960 pixels.

The extraction and postures classification process starts with the detection of the person in the image through the BlazePose use, utilizing the cinematic method. This model identifies 33 landmarks, and each landmark its described through three spatial coordinates, x, y, and z, that indicate its relative location on a tridimensional space. This results in a set of 99 features per sample, as a result of the 33 landmarks and its three associated scales.

To assure a precise detection, BlazePose previously delimits a ROI in the image. This step implies the generation of a main panel that frames the whole figure of the person, along with the two additional panels that are focused in specific bodies segments, like the trunk and the extremities. This procedure enables us to reduce errors in the detection even under partial occlusion conditions or variations in light.

Once the landmarks were identified, the following is used to structure the data in fix length arrangements, which are introduced sequentially in the CNN1D model to be processed like an only input arrangement, but organized in a specific way so the model can interpret the structural relationship between them. During this process, each sample generates an output vector compound by five elements, where each value represents the probability that the posture corresponds to one of the five predefined classes. The final classification is assigned to the class with greater probability, allowing for a precise and efficient identification of the analyzed posture.

4. Analysis Results

This section presents the results obtained after the training and validation of the proposed model for the HPE; several evaluation metrics were used to measure its performance, such as accuracy, precision, recall and F1-Score, allowing for a comprehensive evaluation of the model’s ability to correctly classify the poses in the dataset used. In addition, the confusion matrices were analyzed, visually verifying the predictions to ensure an adequate interpretation of the performance in the classification of the five selected posture classes. For the implementation of CNNs, the Python environment in version 3.11.0, the Visual Studio Code IDE, and the following frameworks were required: OpenCV 4.8.1, Numpy 1.26.1, MediaPipe 0.10.7, and Keras 2.15.0. This implementation was tested on a computer with an Intel Core i7-13620H processor at 2.40 GHz (Intel, Santa Clara, CA, USA), a 24 MB cache, up to 4.90 GHz, 16 GigaBytes of DDR5-5600 SODIMM RAM, and an NVIDIA GeForce RTX 4060: 3072 GPU, with CUDA cores, 8 GigaByte GDDR6, 2.46 GHz, 17 Gbps, and a 128-bit interface (NVIDIA, Santa Clara, CA, USA). The device runs on the Windows 11 operating system.

4.1. Metrics

The metrics below were used to evaluate the model during its training and validation phases. Accuracy, defined in Equation (12), measures the right prediction ratio for the model regarding the total evaluated instances. In this equation, $y^{\prime }$ represents the class predicted by the model and y the true class or correct label. The function $(If,y^{\prime }=y)$ takes 1 as the value if the prediction is true ($y^{\prime }=y$); otherwise, the value is 0. The addition ${\sum }_{i=1}^N$ accumulates the correct predictions for the total N instances, while N normalizes the result, producing a value between 0 and 1.

(12)$\mathit{Accuracy}={\displaystyle \frac{{\sum }_{i=1}^N(If,y_i^{\prime }=y_i)}{N}}$

The confusion matrix presented in Equation (13) describes the distribution of performed predictions by the model in the five considered categories ($C_1,C_2,C_3,C_4,C_5$), which makes it appropriate for the multiclass classification problem. Each line of the matrix represents the real instances of a specific class, while each column corresponds to the performed predictions by the model. The main diagonal contains the values of the correctly classified instances for each class, known as true positives (TPs). On the other hand, the elements outside of the diagonal being $C_{i,j}$, with $i\ne j$, represent the incorrectly classified instances, indicating the false positives (FPs) per each class predicted and the false negatives (FNs) for each real class. This focus allows us to evaluate the performance of the model in each category, such as analyzing the errors between similar classes.

(13)$ConfMtr=\left[\begin{array}{ccccc}{C}_{1,1}& C_{1,2}& C_{1,3}& C_{1,4}& C_{1,5}\\ C_{2,1}& C_{2,2}& C_{2,3}& C_{2,4}& C_{2,5}\\ C_{3,1}& C_{3,2}& C_{3,3}& C_{3,4}& C_{3,5}\\ C_{4,1}& C_{4,2}& C_{4,3}& C_{4,4}& C_{4,5}\\ C_{5,1}& C_{5,2}& C_{5,3}& C_{5,4}& C_{5,5}\end{array}\right]$

For example, the element $C_{1,3}$ indicates the number of instances for the real class $C_1$ that were classified incorrectly as belonging to the class $C_3$. Similarly, $C_{3,1}$ reflects the amount of real instances of $C_3$ were classified incorrectly as $C_1$. The summand of the values in each line represents the number of total real instances for the corresponding class, while the summand of the values in each column indicates the total number of performed predictions for this class.

The matrix is size $5\times 5$, where the lines represent the real classes ($C_1,C_2,C_3,C_4,C_5$) and the predicted columns; this allows us to analyze both the individual prediction of each class and the general behavior of the model, enabling the global metric estimation of values like precision, recall, and F1-Score.

Precision, determined by Equation (14), is defined as the number of true correct predictions carried out.

(14)${\mathrm{Precision}}_i={\displaystyle \frac{{\mathrm{TP}}_i}{{\mathrm{TP}}_i+{\mathrm{FP}}_i}},$

where i represents the number of classes.

Precision Macro Equation (15) is defined as the average of all classes precisions treating them in a uniform way:

(15)${\mathrm{Precision}}_{\mathrm{macro}}={\displaystyle \frac{{\sum }_{i=1}^N{\mathrm{Precision}}_i}{N}},$

where N represents the number of classes.

For weighted precision, Equation (16) takes each class weight into account $w_i$ based on the sample numbers of such class:

(16)${\mathrm{Precision}}_{\mathrm{weighted}}=\sum _{i=1}^N{w}_i·{\mathrm{Precision}}_i,$

For recall, using Equation (17), this function allows us to obtain the number of important cases within the dataset.

(17)${\mathrm{Recall}}_i={\displaystyle \frac{{\mathrm{TP}}_i}{{\mathrm{TP}}_i+{\mathrm{FN}}_i}},$

Recall Macro Equation (18) calculates the arithmetic average of all classes’ recall, treating each class with the same importance, independently of the number of samples in each one. It is used to evaluate the general performance of the model with no preference to any ruling class.

(18)${\mathrm{Recall}}_{\mathrm{macro}}={\displaystyle \frac{{\sum }_{i=1}^N{\mathrm{Recall}}_i}{N}},$

Recall weighted Equation (19), is an adjusted version of the recall that takes the sample amount of each class in consideration. It is especially useful for an unbalanced dataset, since it prioritizes the classes with greater amount of samples.

(19)${\mathrm{Recall}}_{\mathrm{weighted}}=\sum _{i=1}^N{\omega }_i·{\mathrm{Recall}}_i,$

F1-Score Equation (20) is a metric used to evaluate the performance of a classifier. It represents the harmonic mean between precision and recovery, providing a balance between these two metrics, especially useful in problems with unbalanced classes.

(20)${\mathrm{F}1}_{\mathrm{class}}=2\times {\displaystyle \frac{{\mathit{Precision}}_{\mathrm{class}}\times {\mathit{Recall}}_{\mathrm{class}}}{{\mathit{Precision}}_{\mathrm{class}}+{\mathit{Recall}}_{\mathrm{class}}}}.$

F1-Score Macro Equation (21) is defined as the arithmetic average of the calculated F1-Scores for each class individually, treating them as equal without considering the size of each class. This could penalize the model if the performance is poor in minority classes, where n is the number of total classes.

(21)${\mathrm{F}1-\mathrm{Score}}_{\mathrm{macro}}={\displaystyle \frac{{\sum }_{i=1}^n{\mathrm{F}1}_i}{n}},$

Sample-weighted F1 Equation (22) is employed to evaluate the general performance in dataset with an unbalanced classes distribution. A pondered F1-Score average of each class is calculated, where the weights are determined by the amount of samples available in each class, guaranteeing that the classes with more instances have a greater influence in the final metric, providing a representative evaluation of the model in scenarios with unbalanced data.

(22)${\mathrm{F}1-\mathrm{Score}}_{\mathrm{weighted}}=\sum _{i=1}^N{w}_i·{\mathrm{F}1}_i,$

where $w_i$,

(23)$w_i={\displaystyle \frac{\mathrm{No}.\mathrm{of}\mathrm{samples}\mathrm{in}\mathrm{class}i}{\mathrm{Total}\mathrm{number}\mathrm{of}\mathrm{samples}}}.$

4.2. Experiment 1: Resampling of the Initial Set

In Algorithm 1, the dataset was initially split into training and test sets, with the training set representing 80% of the total data and the test set comprising the remaining 20%. This division was performed prior to applying bootstrapping to ensure a clean separation between training and test data.

Algorithm 1: Data augmentation with bootstrapping and K-fold validation

The bootstrapping was implemented with an increase of 75% in the number of original samples (n) in the training set for each partition of the K-fold cross-validation scheme. This ratio was chosen to introduce greater diversity in the training samples while minimizing excessive duplication and avoiding contamination between the training and test sets. This approach ensures that the model learns generalizable patterns effectively, maintains the independence of partitions, and reduces the risk of overfitting.

The use of K-fold cross-validation guarantees that all samples in the dataset are used for both training and evaluation, providing more reliable estimates of model performance and reducing bias. To prevent contamination between the training and test sets, a filtering mechanism was implemented to remove duplicate samples from both partitions. This ensures that the model is evaluated exclusively on unseen data, preserving the validity of the evaluation metrics.

As a result, the model trains with a greater variety of example combinations due to random resampling and maintains the independence of data partitions. This strategy enhances the model’s ability to generalize, ensuring stability and robustness in future predictions while maximizing the available information in the dataset.

4.3. Experiment 2: Training and Validation

The training of the model was carried out using a K-fold cross-validation with five partitions. In each iteration, the training set was submitted to a data-increase process through bootstrapping and any possible contamination between partitions was removed. Finally, the model was trained during 100 epochs with a batch size of 32. During training, 80% of the dataset was allocated for training and 20% for internal validation. This internal validation subset allowed for the monitoring of precision and loss at each epoch, ensuring the model’s performance was evaluated throughout the training process.

During training, the number of epochs was adjusted between 100 and 150. It was observed that the model converged after 100 epochs, which led to the decision to set this value as the training limit. Additionally, the learning rate was fixed at 0.0001; another setting that was modified was the dropout rate value adjustment, which was established in 0.2 and 0.5 to deactivate neurons randomly during training, reducing the over-adjustment risk. Moreover, an early stopping criterion was implemented with a patience of 10 epochs, so if the loss value does not show significant improvements during this period, the training will stop automatically.

4.3.1. Evaluation of Training Using Dropout Rate: 0.2

The model training with a dropout rate of 0.2 allowed us to evaluate the performance through K-fold cross-validation. The general metrics obtained are presented in the Table 1; the average accuracy reached a 0.8956 value, indicating a global robust and consistent performance in all the partitions of dataset. The low standard deviation of 0.0085 evidences a minimum variability between the different folds, which shows a significant stability in the model learning.

The macro F1-Score and macro recall reported values of $0.8962\pm 0.0081$ and $0.8978\pm 0.0094$, respectively, which suggests that the model achieved a balance between all classes, independently of their relative size, which shows that the model did not facilitate the class majority, spreading its performance in a uniform manner. On the other hand, the weighted metrics, like weighted F1-Score $0.8951\pm 0.0091$ and the weighted precision $0.8973\pm 0.0083$, are aligned with the macro metrics, but they reflect an adjustment in the classes’ distributions. The similarity between weighted metrics and the macro suggests that the dataset is not significantly unbalanced or that the model efficiently handled any imbalance that was present.

In terms of standard deviation, all the values are low: for example, the macro precision is $\pm 0.0068$, which indicates that the performance of the model is consistent between the K-fold folds. This level of consistency is key to guarantee that the obtained results are reproducible and not dependant on a specific data partition.

Figure 5 illustrates the average evolution of the accuracy in the training and validation set during the training process, using K-fold cross-validation. Since the very first epochs, a rapid augment in the precision validation is observed, which indicates that the model starts to learn useful patterns since the early training stages. The precision curve of the training shows a similar progression, but with lower initial values, so it suggests a gradual adjustment process for the data. Over the course of the training, the margin between training and validation precisions remains relatively narrow, which is desirable to reduce the risk of over-adjustment. Nevertheless, in the last few epochs, a marginal superiority in the training precision regarding the validation is identified. Although this difference is little, it could be indicative of a slight tendency towards the data adjustment of the training data.

The behavior observed in the graphics suggests that the model is able to learn relevant patterns and keep a coherent relationship between both precisions. The correspondent model to the third fold was identified as the best during the cross-validation process, based on its superior performance metrics in comparison to the other folds. This model was used to realize a full training with the dataset available. In particular, the model achieved values of 0.6829 for loss and 0.9046 for accuracy. These results show the model’s ability to generalize the unseen data in an effective manner, which supports the third fold selection as representative of the best performance during this cross-validation. This focus highlights the importance of identifying the most robust model in a K-fold validation scheme to maximize its performance in real scenarios.

4.3.2. Evaluation of Training Using Dropout Rate: 0.5

The trained models with a dropout rate of 0.5 showed a solid and consistent performance along the K-fold cross-validation partitions. In Table 2, it can be observed that the average accuracy reached a value of $0.9169$, which indicates the high global performance of the model. The loss standard deviation of the accuracy of $\pm 0.0121$ evidences a reduced variability between the folds, which suggests that the model learnt in a stable and generalizable manner on the basis of the provided data.

The macro F1-Score and the macro recall reported values of $0.9166\pm 0.0133$ and $0.9180\pm 0.0136$, respectively. These results highlight the ability of the model to balance its performance in all classes, independently of the relative size of each one, without fostering majority classes.

On its behalf, weighted metrics like weighted F1-Score $0.9170\pm 0.0118$ and the weighted precision $0.9188\pm 0.0122$ show a close alignment to the macro metrics. This indicates that, although the model took the classes’ distribution into account, its performance is still homogeneous. The slight improvement in the weighted metrics could be attributed to the correct representation of majority classes in the dataset. The low value of standard deviation in all metrics suggests a notable consistency between different folds of cross-validation. This reinforces the robustness of the model, ensuring that the obtained results are not impacted in any significant way by specific data partitions.

The graphics presented in Figure 6 show the behavior of the model during training with a dropout rate of 0.5, evaluated through K-fold cross-validation. The average loss for training and validation decreases rapidly during the first 20 epochs. Lately, both curves are stabilized in values close to zero. This behavior shows that the model achieves an adequate convergence, capturing relevant features of the training data. The stabilization of the validation loss, without significant augments towards the end of the training, suggests that there is no evidence of over-adjustment, which confirms the effectiveness of the dropout rate in the model regularization. Furthermore, regarding the average accuracy for the training and validation, both curves show a constant augmentation during the first 50 epochs, achieving values over 90% to the end of the training. It is noticeable that the validation precision is slightly greater that the training in several stages, which is indicative of a model that generalizes adequately. In addition, the confluence of both curves towards similar values reinforces the absence of over-adjustment and validates the robustness of the model for unseen data. Altogether, the results show that the model, trained with a dropout rate of 0.5, not only converges in a stable way, but also maintains its solid generalization ability, evidencing the effectiveness of this regulatory technique to avoid over-adjustment and to improve the performance in multiclass classification tasks.

The best performance in the K-fold validation scheme was obtained in fold 5, the model of which achieved a loss of 0.4240 and an accuracy of 0.9728 in the external trial set. This model was selected as the most representative to evaluate the final undivided performance in additional folds. For this evaluation, the model was trained again using the entirety of the training set available, excluding the external trial set, and preserving the settings of dropout rate, learning rate, and regularization applied in the folds. The results obtained reinforce the ability of the model to generalize over unseen data, maintaining a low loss and high precision, which validates the selection of fold 5 as representative of the best performance observed during the training. Performing a comparative analysis between the dropout settings of 0.2 and 0.5 shows that the performance of the model is higher with a rate of 0.5. This setting allowed us to better balance between the generalization ability and the reduction in the over-adjustment risk. Moreover, the improvement in the key metrics supports the ability of the model to adapt efficiently to the training data, keeping a consistent and stable performance in evaluated partitions. Therefore, the settings with dropout of 0.5 is established as the more adequate for the presented problem.

4.4. Experiment 3: Model Evaluation in Different Scenarios

To evaluate the model’s generalization ability, an individual evaluation is carried out, where a trial image that does not exist in the validation and trial set is predicted; in Figure 7, you can see a person inside a controlled background doing the down-dog posture, and the arms and legs are not completely visible due to the angle of the posture. Nevertheless, the model interprets the correct positions where the extremities are located based on the landmarks extracted and used during its training.

Additionally, the proposed model was approved using equipment with the following features: an AMD A9-9420 RADEON R5 (2 cores and 3 graphic cores) processor (AMD, Sunnyvale, CA, USA) with a base frequency of 3.00 GHz, 8 RAM GB, and a 64 bit operational system. This equipment, with characteristics of low power consumption in comparison with the high-range work stations, allowed us to evaluate the efficiency of the model in a limited-resource environments.

During the trials, an average inference time of 102.42 ms was observed per sample, with a processing speed of 9.18 FPS. These results show that the model is able to operate in real time on medium–low-range hardware, like mobile devices or embedded systems, accomplishing the acceptable requirements regarding latency for interactive applications.

An analysis to verify the proposed model is used to compare the general metrics obtained throughout validation; this analysis is based on verifying precision, recall, and F1-Score, the comparison results of which can be seen in Table 3. Although the CNN1D model does not achieve the best results in absolute terms, its performance is competitive, evidencing its efficiency and effectiveness. Its simplified architecture positions it as a feasible alternative for specific applications, especially in environments with computational-resource limitations. In comparison to the personalized models like GDeep and LDeep, CNN1D shows the ability to achieve a high efficiency without requiring excessively complex configurations, which makes it a practical an efficient option for diverse tasks.

4.5. Experiment 4: Real-Time Posture Detection

For this experiment, the real-time pose detection and classification model was evaluated using the CNN architecture together with BlazePose. We worked with 1080 × 960 pixel resolution images to ensure high quality in the detection of landmarks. The system was able to correctly identify postures such as warrior II and goddess, as seen in Figure 8, adjusting to the input parameters in real time. A detection process was implemented using 33 key points on the body, which were detected with an accuracy of 95%. Furthermore, the model demonstrated robustness against different lighting conditions and moderate motion, maintaining an accuracy greater than 90% even in low-light environments. However, it was observed that classification of more complex poses such as the plank resulted in an increase in false negatives, especially when the pose was not completely aligned with the camera.

5. Conclusions

The CNN1D model implemented together with BlazePose has shown to be an efficient tool for precise human posture classification. During its evaluations, the model achieved 91% global precision, and when it was submitted to a K-fold cross-validation scheme, the precision values remained between 92% and 91%, in addition to obtaining values close to 0.92 in recall and F1-Score metrics for the majority of classes. These results demonstrate its ability to capture the key features of the joints, even in tridimensional environments and with asimetric postures, guaranteeing a solid generalization through different data partitions. One of the main contibutions of this investigation is the application of a CNN1D architecture, which provides significant advantages in comparison with models based on CNN2D. The focus on sequential patterns enables us to capture relationships between body landmarks in an efficient manner, removing the need for processing full images. This not only simplifies the training and the model inference, but also reduces the processing time and the amount of data required to maintain high precision. The random resampling technique through bootstrapping, implemented as part of the training pipeline, strengthened the diversity of the dataset, ensuring the integrity of the selected samples and improving the model stability.

In terms of practical application, the model was approved on hardware with limited resources, achieving an average inference time of 102.42 ms per sample and a processing speed of 9.18 FPS. These results show its viability for real-time applications, like physical rehabilitation systems and posture analysis in the fitness field or in human–computer interactions. Moreover, techniques like dropout layers were implemented to avoid over-adjustment, which enabled the loss function to stabilize low values during training.

Finally, even if the current model focuses on five yoga postures, the expansion of the dataset is identified as an area of improvement to include a wider range of postures, particularly those with greater complexity or symmetric challenges. Also, an optimization of the model is proposed with tools like tensorflow lite or ONNX to maximize the efficiency in mobile devices and embedded systems, enabling its integration for practical applications. These improvements, along with the development of a prototype for home rehabilitation, represent potential steps in the evolution of this topic.

Author Contributions

Conceptualization, M.H.d.l.C. and U.S.; methodology, M.H.d.l.C.; software, U.S. and A.L.-Á.; validation, M.H.d.l.C., U.S. and S.R.Z.-B.; formal analysis, U.S. and S.R.Z.-B.; investigation, M.H.d.l.C. and D.A.M.L.; resources, D.A.M.L. and U.S.; data curation, D.M.-V.; writing—original draft preparation, M.H.d.l.C. and U.S.; writing—review and editing, U.S. and A.L.-Á.; visualization, S.R.Z.-B., D.A.M.L. and D.M.-V.; supervision, A.L.-Á.; project administration, D.A.M.L. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board of National Technological Institute of Mexico, Technological Institute of Chilpancingo 3 October 2023.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study. Written informed consent has been obtained from the patients to publish this paper.

Data Availability Statement

The datasets generated and/or analyzed during the present study are available from the corresponding author upon reasonable request. The original data presented in the study are openly available at https://www.kaggle.com/datasets/niharika41298/yoga-poses-dataset and https://www.kaggle.com/datasets/tr1gg3rtrash/yoga-posture-dataset accessed on 15 July 2024.

Conflicts of Interest

The authors declare no conflicts of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. Proposed methodology phases.

Figure 2. Various postures existing in the dataset representation.

Figure 3. The extraction process of features created by BlazePose. First, it recognizes the human silhouette, and subsequently, it detects the main joints of the human anatomy.

Figure 4. Real-time detection generated by CNN model.

Figure 5. K-fold training, precision, and loss function verification with dropout rate de 0.2. (a) Average accuracy in training with a 0.2 dropout rate. (b) Average loss function in training with a 0.2 dropout rate.

Figure 6. K-fold training, precision, and loss function verification with a 0.5 dropout rate. (a) Average accuracy in training with a 0.5 dropout rate. (b) Average loss function in training with a 0.5 dropout rate.

Figure 7. Verification of the model CNN1D’s generalization of images not seen in training and validation. (a) Down-dog posture—image nonexistent in the trial and training set. (b) Landmarks generated by the CNN1D model for prediction and recognition.

Figure 8. Correct predictions of the poses to be estimated by the CNN model. (a) Correct warrior II detection. (b) Correct goddess detection.

Table 1

Performance metrics of the model averaged across K-fold validation.

Metric	Mean	Standard Deviation
Accuracy (Average)	$0.8956$	$0.0085$
Macro F1-Score	$0.8962$	$0.0081$
Weighted F1-Score	$0.8951$	$0.0091$
Macro Precision	$0.8973$	$0.0068$
Weighted Precision	$0.8973$	$0.0083$
Macro Recall	$0.8978$	$0.0094$
Weighted Recall	$0.8956$	$0.0085$

Table 2

Performance metrics of the model with a dropout rate of 0.5 averaged across K-fold validation.

Metric	Mean	Standard Deviation
Accuracy (Average)	$0.9169$	$0.0121$
Macro F1-Score	$0.9166$	$0.0133$
Weighted F1-Score	$0.9170$	$0.0118$
Macro Precision	$0.9171$	$0.0144$
Weighted Precision	$0.9188$	$0.0122$
Macro Recall	$0.9180$	$0.0136$
Weighted Recall	$0.9169$	$0.0121$

Table 3

Metrics of precision, recall, F1-Score, and accuracy for each model.

Model	Accuracy (%)	Precision (%)	Recall (%)	F1-Score (%)
Xception	96.96	97.20	96.96	96.96
SqueezeNet	95.95	96.40	95.95	95.94
VGGNet	97.97	98.07	97.97	97.97
Deep	98.48	98.54	98.48	98.49
GDeep	99.49	99.50	99.49	99.49
LDeep	98.98	99.03	98.98	98.98
CNN1D	0.9728	0.9752	0.9728	0.9868

References

1. Peng, Q.; Zheng, C.; Chen, C. A Dual-Augmentor Framework for Domain Generalization in 3D Human Pose Estimation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; Seattle, WA, USA, 16–22 June 2024; pp. 2240-2249.

2. Li, W.; Liu, M.; Liu, H.; Guo, T.; Wang, T.; Tang, H.; Sebe, N. GraphMLP: A graph MLP-like architecture for 3D human pose estimation. Pattern Recognit.; 2024; 158, 110925. [DOI: https://dx.doi.org/10.1016/j.patcog.2024.110925]

3. Cao, X.; Yan, W.Q. Pose estimation for swimmers in video surveillance. Multimed. Tools Appl.; 2024; 83, pp. 26565-26580. [DOI: https://dx.doi.org/10.1007/s11042-023-16618-w]

4. An Thoutam, V.; Srivastava, A.; Badal, T.; Kumar Mishra, V.; Sinha, G.R.; Sakalle, A.; Bhardwaj, H.; Raj, M. Yoga pose estimation and feedback generation using deep learning. Comput. Intell. Neurosci.; 2022; 2022, 4311350.

5. Zhang, Y.; Xia, S.; Chu, L.; Yang, J.; Wu, Q.; Pei, L. Dynamic Inertial Poser (DynaIP): Part-Based Motion Dynamics Learning for Enhanced Human Pose Estimation with Sparse Inertial Sensors. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; Seattle, WA, USA, 16–22 June 2024; pp. 1889-1899.

6. Chen, H.; Zendehdel, N.; Leu, M.C.; Moniruzzaman, M.; Yin, Z.; Hajmohammadi, S. Repetitive Action Counting Through Joint Angle Analysis and Video Transformer Techniques. Proceedings of the International Symposium on Flexible Automation; Seattle, Washington, USA, 21–24 July 2024; American Society of Mechanical Engineers: New York, NY, USA, Volume 87882, V001T08A003.

7. Grishchenko, I.; Bazarevsky, V.; Zanfir, A.; Bazavan, E.G.; Zanfir, M.; Yee, R.; Raveendran, K.; Zhdanovich, M.; Grundmann, M.; Sminchisescu, C. BlazePose GHUM holistic: Real-time 3D human landmarks and pose estimation. arXiv; 2022; arXiv: 2206.11678

8. Liu, H.; Bian, L.; Zhang, J. Image-free single-pixel segmentation. Opt. Laser Technol.; 2023; 157, 108600. [DOI: https://dx.doi.org/10.1016/j.optlastec.2022.108600]

9. Park, S.; You, E.; Lee, I.; Lee, J. Towards robust and smooth 3D multi-person pose estimation from monocular videos in the wild. Proceedings of the IEEE/CVF International Conference on Computer Vision; Paris, France, 1–6 October 2023; pp. 14772-14782.

10. Dong, Z.; Song, J.; Chen, X.; Guo, C.; Hilliges, O. Shape-aware multi-person pose estimation from multi-view images. Proceedings of the IEEE/CVF International Conference on Computer Vision; Montreal, QC, Canada, 10–17 October 2021; pp. 11158-11168.

11. Goyal, G.; Di Pietro, F.; Carissimi, N.; Glover, A.; Bartolozzi, C. MoveEnet: Online high-frequency human pose estimation with an event camera. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; Vancouver, BC, Canada, 17–24 June 2023; pp. 4024-4033.

12. Aarthy, K.; Kruthi, M.; Upadhyay, R.; Darbhamulla, J.; Singh, S.; Pavikars, M.M. Advanced Yoga Pose Estimation: Enhancing PoseNet with Adaptive Key Point Elimination. Proceedings of the 2024 International Conference on Recent Advances in Electrical, Electronics, Ubiquitous Communication, and Computational Intelligence (RAEEUCCI); Chennai, India, 17–18 April 2024; pp. 1-9.

13. Lin, Y.; Jiao, X.; Zhao, L. Detection of 3D human posture based on improved MediaPipe. J. Comput. Commun.; 2023; 11, pp. 102-121. [DOI: https://dx.doi.org/10.4236/jcc.2023.112008]

14. Ullah, H.; Munir, A. Human action representation learning using an attention-driven residual 3DCNN network. Algorithms; 2023; 16, 369. [DOI: https://dx.doi.org/10.3390/a16080369]

15. Ullah, R.; Asif, M.; Shah, W.A.; Anjam, F.; Ullah, I.; Khurshaid, T.; Wuttisittikulkij, L.; Shah, S.; Ali, S.M.; Alibakhshikenari, M. Speech emotion recognition using convolution neural networks and multi-head convolutional transformer. Sensors; 2023; 23, 6212. [DOI: https://dx.doi.org/10.3390/s23136212]

16. Aldawsari, H.; Al-Ahmadi, S.; Muhammad, F. Optimizing 1D-CNN-based emotion recognition process through channel and feature selection from EEG signals. Diagnostics; 2023; 13, 2624. [DOI: https://dx.doi.org/10.3390/diagnostics13162624]

17. Bazarevsky, V. BlazePose: On-device Real-time Body Pose tracking. arXiv; 2020; arXiv: 2006.10204

18. Lovanshi, M.; Tiwari, V. Human pose estimation: Benchmarking deep learning-based methods. Proceedings of the 2022 IEEE Conference on Interdisciplinary Approaches in Technology and Management for Social Innovation (IATMSI); Gwalior, India, 21–23 December 2022; pp. 1-6.

19. Kale, S.; Kulkarni, N.; Kumbhkarn, S.; Khuspe, A.; Kharde, S. Posture Detection and Comparison of Different Physical Exercises Based on Deep Learning Using MediaPipe, OpenCV. Int. J. Sci. Res. Eng. Manag.; 2023; 7, pp. 1-29.

20. Mroz, S.; Baddour, N.; McGuirk, C.; Juneau, P.; Tu, A.; Cheung, K.; Lemaire, E. Comparing the quality of human pose estimation with BlazePose or OpenPose. Proceedings of the 2021 4th International Conference on Bio-Engineering for Smart Technologies (BioSMART); Paris, France, 8–10 December 2021; pp. 1-4.

21. Suhandi, V.; Santoso, H. Personal Training with Tai Chi: Classifying Movement using MediaPipe Pose Estimation and LSTM. Build. Inform. Technol. Sci. (BITS); 2024; 6, pp. 767-775.

22. Chaudhary, I.; Singh, N.T.; Chaudhary, M.; Yadav, K. Real-time yoga pose detection using OpenCV and MediaPipe. Proceedings of the 2023 4th International Conference for Emerging Technology (INCET); Belgaum, India, 26–28 May 2023; pp. 1-5.

23. Dill, S.; Rösch, A.; Rohr, M.; Güney, G.; De Witte, L.; Schwartz, E.; Hoog Antink, C. Accuracy Evaluation of 3D Pose Estimation with MediaPipe Pose for Physical Exercises. Curr. Dir. Biomed. Eng.; 2023; 9, pp. 563-566. [DOI: https://dx.doi.org/10.1515/cdbme-2023-1141]

24. Parashar, D.; Mishra, O.; Sharma, K.; Kukker, A. Improved Yoga Pose Detection Using MediaPipe and MoveNet in a Deep Learning Model. Rev. D’Intell. Artif.; 2023; 37, pp. 1197-1202. [DOI: https://dx.doi.org/10.18280/ria.370511]

25. Mountzouris, K.; Perikos, I.; Hatzilygeroudis, I. Speech emotion recognition using convolutional neural networks with attention mechanism. Electronics; 2023; 12, 4376. [DOI: https://dx.doi.org/10.3390/electronics12204376]

26. Arrowsmith, C.; Burns, D.; Mak, T.; Hardisty, M.; Whyne, C. Physiotherapy exercise classification with single-camera pose detection and machine learning. Sensors; 2022; 23, 363. [DOI: https://dx.doi.org/10.3390/s23010363]

27. Chen, H.; Tao, W.; Leu, M.C.; Yin, Z. Dynamic gesture design and recognition for human-robot collaboration with convolutional neural networks. Proceedings of the International Symposium on Flexible Automation; Virtual, 8–9 July 2020; American Society of Mechanical Engineers: New York, NY, USA, 2020; Volume 83617, V001T09A001.

28. Song, L.; Yu, G.; Yuan, J.; Liu, Z. Human pose estimation and its application to action recognition: A survey. J. Vis. Commun. Image Represent.; 2021; 76, 103055. [DOI: https://dx.doi.org/10.1016/j.jvcir.2021.103055]

29. Dedhia, U.; Bhoir, P.; Ranka, P.; Kanani, P. Pose Estimation and Virtual Gym Assistant Using MediaPipe and Machine Learning. Proceedings of the 2023 International Conference on Network, Multimedia and Information Technology (NMITCON); Bengaluru, India, 1–2 September 2023; pp. 1-7.

30. Bhamidipati, V.S.P.; Saxena, I.; Saisanthiya, D.; Retnadhas, M. Robust intelligent posture estimation for an AI gym trainer using MediaPipe and OpenCV. Proceedings of the 2023 International Conference on Networking and Communications (ICNWC); Chennai, India, 5–6 April 2023; pp. 1-7.

31. Diaz-Rojas, F.; Myowa, M. Estimation of human body 3D pose for parent-infant interaction settings using Azure Kinect and OpenPose. MethodsX; 2024; 13, 102861. [DOI: https://dx.doi.org/10.1016/j.mex.2024.102861]

32. Suyudi, I.; Sudadio, S.; Suherman, S. Pengenalan Bahasa Isyarat Indonesia menggunakan Mediapipe dengan Model Random Forest dan Multinomial Logistic Regression. J. Ilmu Siber Dan Teknol. Digit.; 2022; 1, pp. 65-80. [DOI: https://dx.doi.org/10.35912/jisted.v1i1.1899]

33. Bhattarai, R.; Bohara, V.; Balai, L.N. Human Activity Recognition using SVM, KNN & Logistic Regression. World J. Res. Rev. (WJRR); 2023; 16, pp. 46-50.

34. Alsabhan, W. Human–computer interaction with a real-time speech emotion recognition with ensembling techniques 1D convolution neural network and attention. Sensors; 2023; 23, 1386. [DOI: https://dx.doi.org/10.3390/s23031386] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36772427]

35. Hartley, R.; Zisserman, A. Multiple View Geometry in Computer Vision; Cambridge University Press: Cambridge, UK, 2003.

36. Szeliski, R. Computer Vision: Algorithms and Applications; Springer Nature: Berlin/Heidelberg, Germany, 2022.

37. Ohri, A.; Agrawal, S.; Chaudhary, G.S. On-device real-time pose estimation & correction. Int. J. Adv. Eng. Manag. (IJAEM); 2021; 3, pp. 1691-1696.

38. Gorriz, J.M.; Segovia, F.; Ramirez, J.; Ortiz, A.; Suckling, J. Is K-fold cross validation the best model selection method for Machine Learning?. arXiv; 2024; arXiv: 2401.16407

39. Shiddiq, M.; Candra, F.; Anand, B.; Rabin, M.F. Neural network with k-fold cross validation for oil palm fruit ripeness prediction. TELKOMNIKA (Telecommun. Comput. Electron. Control); 2024; 22, pp. 164-174. [DOI: https://dx.doi.org/10.12928/telkomnika.v22i1.24845]

40. Jian, Y.; Gao, C.; Vosoughi, S. Bootstrapping vision-language learning with decoupled language pre-training. Proceedings of the Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023; New Orleans, LA, USA, 10–16 December 2023.

41. Zhao, Q.; Zheng, C.; Liu, M.; Chen, C. A single 2D pose with context is worth hundreds for 3D human pose estimation. Proceedings of the Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023; New Orleans, LA, USA, 10–16 December 2023.

42. Jia, W.; Zhu, X.; Zhou, Y.; Hu, M.; Liu, C.; Song, Q. UV R-CNN: Stable and efficient dense human pose estimation. Multimed. Tools Appl.; 2024; 83, pp. 24699-24714. [DOI: https://dx.doi.org/10.1007/s11042-023-15379-w]

43. Shan, W.; Lu, H.; Wang, S.; Zhang, X.; Gao, W. Improving robustness and accuracy via relative information encoding in 3D human pose estimation. Proceedings of the 29th ACM International Conference on Multimedia; Virtual, China, 20–24 October 2021; pp. 3446-3454.

44. Yao, Z.; Su, J.; Yau, S.-T. Manifold fitting with CycleGAN. Proc. Natl. Acad. Sci. USA; 2024; 121, e2311436121. [DOI: https://dx.doi.org/10.1073/pnas.2311436121]

45. Alsaadi, S.; Anande, T.J.; Leeson, M.S. Comparative Analysis of 1D-CNN and 2D-CNN for Network Intrusion Detection in Software Defined Networks. Proceedings of the International Conference on Emerging Internet, Data & Web Technologies; Naples, Italy, 21–23 February 2024; pp. 480-491.

46. Roggio, F.; Trovato, B.; Sortino, M.; Musumeci, G. A comprehensive analysis of the machine learning pose estimation models used in human movement and posture analyses: A narrative review. Heliyon; 2024; 10, e39977. [DOI: https://dx.doi.org/10.1016/j.heliyon.2024.e39977]

47. Mundt, M.; Born, Z.; Goldacre, M.; Alderson, J. Estimating ground reaction forces from two-dimensional pose data: A biomechanics-based comparison of alphapose, blazepose, and openpose. Sensors; 2022; 23, 78. [DOI: https://dx.doi.org/10.3390/s23010078]