Full Text

Turn on search term navigation

1. Introduction

The adaptation of traditional educational materials to digital form is an important direction of development in the modern era. With the rapid advancement of electronic devices, an important aspect of everyday life is the adaptation of printed graphical information into interactive virtual form. This is particularly significant for blind and visually impaired people, for whom the accessibility of visual graphics can sometimes be challenging. Digital materials need to be adapted to meet their needs, particularly in terms of how information is conveyed through the other senses to compensate for the lost sense of sight. There are currently numerous solutions to help visually impaired people, but the adaptation of graphics is still a major challenge in many disciplines. One of these is the area of education, where the sciences very often rely on the use of images with graphs and diagrams. In the field of mathematics, for example, it is possible to specify a number or even a dozen items that can be placed on graphs to convey information concisely and intuitively. Most frequently, these elements take the form of additional descriptions on the graph, or details written below specific objects such as points or parabolas. Unfortunately, this is a significant barrier for blind people who cannot read this information, making the whole process of acquiring knowledge provided in this format increasingly difficult. Over the years, researchers have proposed many solutions to this problem, and among them was using tactile printouts and audio-tactile displays, but the basis for their positive impact is still the aspect of presenting educational material in a specified adapted way [1]. Another alternative approach is the DotPad—a small tactile graphic display that uses braille pins and AI (Artificial Intelligence) to present images in real time [2]. The last approach worth mentioning in the context of this article’s background is 3D shapes that can be used as real-life object representations [3]. In all of the described cases, the main issues still are the process of correctly adapting the fundamental material used so far and the complexity of this process. An ideal solution would involve creating a tool that allows for any graph or diagram to be adapted to an accessible form without needless effort. This adaptation includes the removal of redundant information contained in the graphics, such as theoretical notes, which are incomprehensible to a blind person in this way and need to be provided in another form, such as an audio description. The answer to these challenges may lie in the use of machine learning algorithms, which are becoming increasingly important in modern technological applications. Modern society is increasingly relying on AI to solve basic everyday tasks, as evidenced by the popular generative models that are used every day [4]. Their rapid development opens many avenues of potential opportunities to automatically or semi-automatically adapt graphics to convey all the information in an accessible way to a blind person. The described aspect is highly important because, in today’s society, there is a strong emphasis on equal development opportunities for people with and without disabilities.

In recent years, machine learning has become one of the key areas in which information technology continues to evolve. It has applications in a wide range of fields, from the aforementioned generative models for information retrieval to scientific research and, last but not least, the automation of basic everyday activities. Some specific cases include text classification [5], speech recognition [6], and weather forecasting [7]. In all of these cases, different AI algorithms are used as the standard. In the context of the main focus of this article, it must be said that machine learning is a general term that can be divided into several distinct subsections of algorithms that are used for particular purposes. One of the main divisions is related to the involvement of the data scientist and consists of the following four categories: supervised, unsupervised, semi-supervised, and reinforcement learning [8]. A further breakdown can be made according to the type of algorithm and its purpose. This division is broader than the previous one, but the most common examples include Decision Trees, Naive Bayes, kNN (k-nearest neighbors), SVM (Support Vector Machine), Rule-learners, and neural networks. The purpose of the latter is most often to analyze and segment images, as well as to detect and classify objects [9]. Neural networks are capable of replicating the actions of biological neurons communicating with each other in the brain, hence their name. The history of their development can be traced back to the second half of the 20th century, more precisely to 1958, when Rosenblatt presented the perceptron, considered to be the cornerstone of neural networks [10]. The year 1986 brought further developments as a group of researchers established the effectiveness of the backpropagation algorithm, which allowed for the creation of multi-layer networks capable of solving complex non-linear problems. The algorithm aimed to minimize cost functions by adjusting weights and biases, allowing the neural networks to learn new information based on the input data [11]. One of the most encouraging new approaches in this field at the moment is the YOLO (You Only Look Once) neural networks, which have been in continuous development since 2015 [12]. They are renowned for their ability to quickly and accurately analyze images and video in real time, outperforming previous detection algorithms. The core of their operation is based on detecting and classifying objects, assigning them to the appropriate type of class. These features have allowed YOLO to be used in many areas related to computer vision, such as surveillance systems. The potential of its use has also been demonstrated in many contexts related to the analysis of visual information, such as classifying crops [13,14], cancer detection [15,16], and pill identification [17]. Despite the wide range of applications of YOLO, the context of adapting graphics for the visually impaired has not been fully explored.

The purpose of this article is to present the results of the research performed on the effectiveness of selected versions of YOLO networks in the context of removing redundant information, such as additional textual descriptions placed on images, and detecting elements in mathematical graphs and diagrams that are relevant from the point of view of a visually impaired person. These mentioned elements contained in the graphs include axes, linear functions, parabolas, hyperbolas, trigonometric functions, and polynomials [18]. The primary evaluation steps focused on the effectiveness of object recognition and classification, measured by the basic coefficients used in this type of network, i.e., the average accuracy, the error matrix, and the time needed to analyze an image. The results of the comparison will make it possible to identify the advantages and limitations of the selected versions of the YOLO network and thus indicate possible further directions for the development of a solution aimed at adapting graphs to the needs of blind people. The research performed will also constitute a first step towards the full automation of processes related to the preparation of exercises for visually impaired people, which may result in the creation of a stand-alone application to support current educational activities. The whole solution is aimed at developing cooperation between visually impaired and blind people, thus breaking down barriers in social and academic life, which has been shown to have a real positive impact [19].

2. Development of YOLO Neural Networks

In machine learning, YOLO-type neural networks are currently an area of intense development. The basic premise of identifying objects in images and videos in real time has proven to be one of the most important elements in the solutions commonly used today. Among the many detection algorithms that have emerged over the years, these networks have a reasonable balance between the speed and accuracy of identified objects [20].

The development of the YOLO network began in the second decade of the 21st century, and the first version was presented at CVPR 2016 (2016 IEEE Conference on Computer Vision and Pattern Recognition) by Joseph Redmon et al. [21]. It was the first approach to end-to-end object detection in real time. The novelty of the work presented is that a single image analysis using the network was sufficient to perform the classification, in contrast to previous solutions that required hundreds of runs to detect possible regions of interest and, in a second step, to classify the object in these regions. This characteristic gave rise to both the name and the acronym that would later be identified with this branch of the development of neural networks. The YOLO network has also simplified the outputs, basing them only on regression, in contrast to the Fast R-CNN [22], which used two outputs with classification parameters. The main architecture was based on 24 convolutional layers and 2 fully connected layers, but the authors also added a lightweight model with only 9 layers called Fast YOLO. Despite its novelty, the first version also had certain drawbacks, such as limiting the number of simultaneously detected elements of a given class to two objects in a grid, which made it difficult to detect neighboring objects of the same type. The result of the AP (Average Precision) measure, which, together with the AP₅₀ (Average Precision with a threshold of 0.5), is the main evaluation metric for YOLO networks, was 63.4% for the PASCAL VOC2007 [23] dataset, which contained 20 classes and was the benchmark used for comparisons at the time. The detailed architecture of this version was also presented in the aforementioned paper [21].

The second version, presented as part of CVPR 2017 (2017 IEEE Conference on Computer Vision and Pattern Recognition) by Joseph Redmon and Ali Farhaid [24], fixed some of the issues of YOLOv1 by introducing, among other things, features such as normalizing the batch to reduce overfitting, removing dense layers to make the architecture fully convolutional, improving the performance of the network on higher-resolution images, and introducing anchor boxes to be used as prototype shapes. In addition, changes were made to the training process of the network; for example, it was possible to use images of different sizes during the procedure, as the authors adapted the model to take random sizes as possible inputs. The fundamentals of the architecture were also changed resulting in using the Darknet-19 backbone with 19 convolutional and 5 max-pooling layers. All of these changes improved the average precision, with the YOLOv2 version achieving a score of 78.6% on the same PASCAL VOC2007 dataset. The authors also introduced a solution called YOLO9000, which was a combination of the COCO dataset with ImageNet classification data and a backpropagation training algorithm. The result was a network that was able to detect more than 9000 classes. The detailed architecture of this version was also presented in the mentioned article [24].

YOLOv3 [25] was published on ArXiv in 2018 by the same authors as the previous version, introducing further changes related to the performance of the network and its architecture. The changes included an improvement in the bounding box logic, which allowed a reduction in the number of anchor boxes assigned to an object during classification, and an enhancement of the class prediction to be able to identify multiple objects in the same box for complex images with labels that could sometimes overlap. The architecture was changed from Darknet-19 to Darknet-53, consisting of 53 convolutional layers, while reducing the number of max-pooling layers to 0. The third version of the network achieved an AP score of 36.2% and an AP₅₀ of 60.6%. These results are lower than those obtained in previous versions, but this is due to the change in the benchmark dataset from PASCAL VOC to Microsoft COCO, which has since become the default. Along with this change, a new AP₅₀ measure has been introduced that sets the threshold for the IoU parameter to 0.5. The detailed architecture of this version was also presented in the aforementioned paper [25].

The next version was published 2 years later, in 2020, in the journal ArXiv [26] by Alexey Bochkovskiy, Chien-Yao Wang, and Hong-Yuan Mark Liao. Since the article did not have the same authors as the previous versions, the community was initially skeptical about the new version and did not know if it was the official version. After some time, the presented concept was accepted as YOLOv4 and as a continuation of the evolution of the network, because it maintained all the characteristics of its predecessors while improving the training process and parameter optimization. Improvements included additional data augmentation methods, allowing the combination of four images to be analyzed out of their standard context during the training process, and the use of genetic algorithms for hyperparameter optimization. The basic architecture remained the same, as the Darknet-53 was also chosen, but a few modifications were made to better adapt it to the overall YOLO standards. The average precision metric for this network version on the MS COCO dataset was 43.5% for AP and 65.7% for AP₅₀. The detailed architecture of this version was also described in the mentioned paper [26].

A fifth version appeared a few months later, authored by Ultalytics founder and CEO Glenn Jocher, but this time, it was not published but only appeared on Github [27]. Among the new features of YOLOv5 was the creation of mobile network versions for iOS and Android. In addition, the entire solution has been scaled and subdivided into networks of five different sizes so that they can be configured according to the user’s needs and the capabilities of the environment in which they will be deployed. The main difference between this version and YOLOv4 is the foundation, because instead of being developed on Darknet, Pytorch was used as the baseline. The different versions were named according to their complexity. They included nano, small, medium, large, and extra-large. The average precision for each subdivision is currently 45.7%, 56.8%, 64.1%, 67.3%, and 68.9%, respectively. In addition, Yolov5 version 6.0 was released in 2021, introducing increased accuracy, optimized runtime performance, and five new complexity models. They include nano6, small6, medium6, large6, and extra-large6. The average accuracy for each new subdivision is currently 54.4%, 63.7%, 69.3%, 71.3%, and 72.7%, respectively. The architecture of this version is also available in the Github repository [27].

One year after the publication of YOLOv4, Scaled-YOLOv4 [28] was presented at CVPR 2021 (2021 IEEE Conference on Computer Vision and Pattern Recognition), where a new concept was introduced that allowed the model to be scaled. It gave the developer of a given model the choice of whether he or she prefers higher precision at the expense of speed, or lower accuracy at high speed. This is well illustrated by the AP parameter, which was 22% for the YOLOv4-tiny version focused on reducing detection time, while it was 56% for the higher-precision YOLOv4-large version. The architecture of this version was also described in detail in the mentioned paper [28].

Further development has led to new approaches being published on ArXiv: YOLOR [29] in May 2021 and YOLOX [30] in July 2021. In both cases, modifications of the basic principles have been explored. In the case of YOLOR (You Only Learn One Representation), published by the same authors as YOLOv4, a task-based learning solution was used, where subnetworks responsible for performing specific actions, i.e., classification and detection, were separated within the main network. The modifications resulted in the network being able to use implicit knowledge in multiple tasks. This is comparable to humans, who use the experience gained to solve new problems. These modifications resulted in an average AP accuracy of 55.4% and an AP₅₀ of 73.3%. The second approach, YOLOX, was based on YOLOv3 and introduced changes related to the architecture by eliminating an anchor-based one. In addition, changes were made to label assignment, optimization, and detection performance, which were divided into two factors: classification confidence and location accuracy. The AP score of this solution was 50.1%. Both architectures of YOLOR and YOLOX were described in detail in previous papers [29,30].

In 2022, three more versions of the network were published on ArXiv: YOLOv6 [31], YOLOv7 [32], and DAMO-YOLO [33]. All of them, like the previous ones, introduced changes to the existing architecture, while each had individual features to distinguish it from the rest. The first of these introduced a new architecture based on RepVGG [34], improvements in label assignment, and new regression and classification losses. The release contained eight sub models: YOLOv6-N, YOLOv6-S, YOLOv6-M, YOLOv6-L, and once more complex from YOLOv6-N6 to YOLOv6-L6. The achieved AP values for each were 37.5%, 45.0%, 50.0%, and 52.8% for the base ones, and 44.9%, 50.3%, 55.2%, and 57.2% for the more complex ones, respectively. The second mentioned version, presented by the same authors as YOLOR and YOLOv4, focused on the introduction of a new strategy called E-ELAN (Extended efficient layer aggregation network) [35] combined with architectural changes and implicit knowledge known from YOLOR. YOLOv7 was also divided into variants, where the basic model achieved an AP value of 51.4% and an AP₅₀ value of 69.7%, and the most complex one achieved 56.8% and 74.4%, respectively. The latter version introduced changes in the learning process, based on the school model, where the teacher first imparts knowledge to the students and the students then try to consolidate it themselves. In addition, changes were made to the architecture and network layers. The authors presented eight scaled models. The results of the AP ranged from 43.6% for the basic case to 51.9% for the most complex variant. All three architectural changes were presented in the mentioned papers [31,32,33].

YOLOv8 [36] and YOLO-NAS [37] are the versions of the network released in 2023. The eighth version, developed by Ultralitycs, in addition to the new features added by YOLOv5, introduced task sharing, i.e., the use of a single solution for multiple computer vision tasks such as segmentation, detection, classification, and tracking. The architectural principles remained the same as in the previous version by the same authors but were modified by the addition of a C2f module that combines contextual information with high-level features, thus improving the accuracy of detection. As with YOLOv5, Ultralytics supplied five scaled versions, which are nano, small, medium, large, and extra-large. The AP results for each of the above are 37.3%, 44.9%, 50.2%, 52.9%, and 53.9%, respectively. The second version mentioned was a solution developed by Deci, a company specializing in the development of software based on artificial intelligence. This version focused on several aspects, i.e., small object detection and location accuracy, so that it could be used in edge devices, among others. The AP value varies from 47.5% for the YOLO-NAS S to 52.22% for the YOLO-NAS L. Both architectures were described in detail in their respective Github repositories [36,37].

Since 2020, a solution called PP-YOLO has been developed, and its versions PP-YOLO [38], PP-YOLO2 [39], and PP-YOLOE [40] have been published in the AvXiv journal. The changes made in this regard naturally had an impact on the entire official distribution of the main YOLO network. All changes, features, and architectural modifications in these variants have been described in their respective cited papers.

The latest versions published so far are YOLOv9 [41], YOLOv10 [42], and YOLOv11 [36,43]. The changes to the first version were mainly architectural and involved the introduction of two frameworks: PGI (Programmable Gradient Information) and GELAN (Generalized Efficient Layer Aggregation Network), which made it possible to reduce the number of computations and parameters compared to the previous YOLOv8 version. The scaled models for YOLOv9 included five variants with AP and AP₅₀ values ranging from 38.3% and 53.1% for the basic T variant to 55.6% and 72.8% for the most complex E variant. The second version, like its predecessors, focused on architectural changes but also eliminated the NMS algorithm in post-processing, which allowed the training process to be sped up. In place of the algorithm, dual label assignment was proposed. This allowed for the obtainment of an AP value at the level of 38.5% for the basic model YOLOv10-N and 54.4% for the most complex YOLOv10-X. The authors provided a total of six scaled versions. The latest version was also developed by Ultralytics and featured architectural changes, improved accuracy with fewer parameters, and optimizations for speed and efficiency. The authors provided five scale models as in YOLOv5 and YOLOv8, which were nano, small, medium, large, and extra-large. The AP for each of them was 39.5%, 47.0%, 51.5%, 53.4%, and 54.7%, respectively. All three architectures have been described in detail in their respective papers [41,42,43]. All of the average precision values that are described and cited are given in Table 1.

The main measure used to assess the performance of the YOLO neural networks is the AP metric, sometimes referred to as mAP (Mean Average Precision) [44]. Its value is calculated taking into account three factors including the average precision of all the categories included in the dataset and allows for the evaluation of different configurations and models. The first factor can be described as precision and recall and determines the relationship between the accuracy of positive predictions and the actual number of objects correctly recognized. A large number of items being simultaneously identified may result in some of them being misclassified, which will inevitably affect the result obtained. To account for this factor, the AP metric takes into consideration the precision–recall curve. The second component focuses on assessing the effectiveness of recognizing objects from multiple categories in a single image and can be summarized as handling multiple object classes. This factor works as previously described in the general characterization of the AP metric, but the important aspect is that it considers all categories separately. This helps to ensure that the evaluation of each model’s performance is more rounded and provides a more complete picture than if all errors were grouped into one category. The last factor determines the probability with which an element in the image can be said to have been recognized correctly. This aspect is represented by a metric called IoU (Intersection over Union) and is important from the point of view of the result when assessing the quality of the prediction. The output of the YOLO network is an image with classified objects marked by rectangular areas called bounding boxes, which represent the relative position of the detected object in the analyzed space. The numerical value of IoU represents the intersection of two objects—the predicted and the actual object location bounding boxes. The higher the metric value is, the greater the confidence in the correctness of the classification. A simplified computational schematic of the calculation of the IoU value is shown in Figure 1.

To better illustrate how all three factors contribute to the final value of the AP metric, an example can be made on the model Microsoft COCO dataset [45]. The main characteristics of this data are that it consists of 80 object classes and has a 101-point interpolation, which allows the precision of 101 recall thresholds from 0 to 1 in 0.01 intervals. These factors allow the AP to be calculated over multiple IoU values, resulting in more accurate results than a common AP₅₀ metric, which is limited to a threshold of 0.5. The example steps for the final calculations can be described as below:

Compute the recall–precision curve for each class with a varying threshold;
Calculate the AP value for each category with 101-point thresholds;
Compute the AP with different IoU thresholds—in most cases, from 0.5 up to 0.95 with 0.05 intervals;
Calculate the mean AP for every IoU threshold from step 3;
Compute the final average value by taking into account all the above metrics.

Another important aspect related to YOLO neural networks is Non-Maximum Suppression (NMS). This is a process that takes place after the detection of objects and allows one to discard less-precise bounding boxes in the case where there are multiple instances of them on a single element. The algorithm is based on the IoU parameter and selects a bounding box with the highest confidence score for a detected class, removing the excess ones so that the result is one per element. An example of the Non-Maximum Suppression is shown in Figure 2.

In this publication, several of the networks described above were selected to compare the effectiveness of their application in the context of analyzing mathematical graphs. The selected versions were YOLOv5, YOLOv6, YOLOv7, YOLOv8, YOLOv9, YOLOv10, and YOLOv11.

3. Materials and Methods

Due to the lack of existing datasets with suitable images, this research was divided into several phases, the first of which focused on the decomposition of the elements present in the mathematical graphs and the generation and preparation of the knowledge base. The final stages consisted of training, verifying the effectiveness, and assessing the suitability of a given version of the network in the context under consideration.

At the beginning of the research, the graphical information contained in the graphs was decomposed into basic elements relevant to the further use of the resulting models. The images were divided into the smallest identifiable units, which in further steps represented the classes recognized by the neural network. The decomposition assumed a general to detailed approach, where the first element was always a mathematical graph and the last was the smallest possible object, such as the pitch on the x and y axes. The result of the class decomposition process is shown in Figure 3.

Analyzing the branches of the proposed decomposition, it is important to notice certain classes that are a direct result of the need to adapt the solution to the needs of blind people. Specifically, the division proposed for, among others, parabola and hyperbola graphs, where the graph with positive and negative coefficients was additionally divided into separate parts. This is motivated by the approach of later diversifying the description of each element in the picture according to the student’s needs. For example, the graph of a function often requires the addition of a separate descriptor for each arm of a parabola or hyperbola. Previous studies have shown that this approach of using the senses of touch and hearing to compensate for the loss of sight is very effective [46]. This division is also important in the context of the further use of the models with an already working system based on .svg images. The result to be achieved is a fully automatically generated vector image with descriptions in which each element is a separate line, and that is why this division was a crucial step in the research carried out. The decomposition presented also formed the basis for the further verification of the average precision for each essential element of an image.

The second step of the overall research methodology was based on the preparation of datasets to train models based on selected versions of the YOLO network. For this purpose, scripts were written in Python to generate images in .png format for each of the selected graph types and for the individual elements included within them. A rule of thumb was followed that some part of the dataset should contain invalid or corrupted data, to avoid network training only in ideal cases. The discontinuities were generated randomly in several ways, including narrowing the domain of the function and generating random points outside the domain. The percentage distribution of incorrect data was 10% of the final dataset.

The final dataset contained over 40,000 generated images of individual elements as well as complete function graphs, with each class represented by a minimum of 1500 occurrences. Since nearly 20,000 images represented entire charts, the number of cases for classes such as x and y tick, x and y axis, and x and y arrow were significantly higher than all other cases. In some instances, these classes were important for recognizing the plot, especially for trigonometric functions where the context allows for them to be distinguished from each other. Taking all these facts into account, it must be said that the dataset was balanced for all plot-related elements such as parabolas and linear functions, with a slight imbalance in the area of axis-related objects, since these are the essential parts of each image. The final step in the data preparation process was to label all objects in the drawings using tools adapted to the YOLO network. This format contains an image file and a separate text file with the same name and the labels—class name and position of the object. This was one of the most critical steps, as the accuracy of the labels largely determines the final results obtained. An example of the image annotation process is shown in Figure 4.

All the steps described above allowed us to proceed to the process of training the network on the prepared dataset. The test environment was a desktop computer with the following parameters: AMD Ryzen 9 3950X 16-core processor, 128 GB RAM, and NVIDIA GeForce 3090Ti graphics card. This allowed for the use of the most complex models for networks that offered this choice.

The main experiments were related to training the data on different data splits. For this article, the authors decided to obtain the results for 3 splits of the generated dataset:

90% training data (36,000 images) and 10% validation data (4000 images);
80% training data (32,000 images) and 20% validation data (8000 images);
80% training data (32,000 images), 10% test data (4000 images), and 10% validation data (4000 images).

The decision to remove the test data in two experimental cases was because most training processes carried out with the YOLO networks suggested using one dataset for both testing and validation. The main data partitioning algorithm used to ensure that each class was correctly represented was to divide the labeled images for each subtype separately according to the suggested percentages. This means, for example, that for every 1000 images representing each class, 100 or 200 were used as test or validation graphs, depending on the selected split.

After deciding how to split the data, the models from each YOLO version selected for comparison need to be chosen. To ensure that the results were comparable, the authors selected all models that defined the input image to be 640 × 640, which excluded, for example, the YOLOv5 version 6.0 models as the input image for them must be 1280 × 1280. The next step in the process was to train the models for at least 100 epochs. During each of these epochs, the model was able to learn all of the training data. Subsequent analysis involved validating the models using tools provided by the creators of each version and comparing the parameters, as well as field testing the best models with newly generated images to verify actual effectiveness in a real-world scenario. Validation parameters provided by the developers included precision, recall, AP₅₀, and confusion matrices. The algorithm steps to calculate the first two mentioned values are presented in Figure 5. The field-testing scenario was emulated by generating images of mathematical graphs with a specified number of elements on them and then calculating the standard percentage values for the classification process. All of the above analyses are described in detail in the Section 4.

4. Results

To illustrate the results of this study and the number of tests performed, the following section is divided into subsections, each describing the results obtained for a selected version of YOLO. In addition, the experiments for each version are divided internally by the results for each data split proposed.

4.1. YOLOv5

YOLOv5 is, as described above, the oldest of all the versions of the network released so far, but its distinct features allow us to analyze its usability in the context of adapting mathematical graphs to the needs of the blind. The authors of this version developed five scaled models that operate on a 640 × 640 input.

4.1.1. 90% Training Data and 10% Validation Data

As described in the experimental steps, the first and most important step was to train the models on the prepared datasets. From the proposed partitions, the 90% training data and 10% validation data division was always selected first. After the training period, which varied depending on the complexity of the model, the results were collected and compared. The values obtained after 100 epochs are shown in Table 2.

The results show that the medium complexity version achieved the best result with an AP₅₀ value of 0.568 (56.8%) and 0.652 and 0.556 for the precision and recall, respectively. The nano and small versions had significantly lower results for each of these parameters, and the large and extra-large models had only slightly worse values, with the precision of the l model being higher than that of the m model.

4.1.2. 80% Training Data and 20% Validation Data

The division between 80% training data and 20% validation data was always chosen as the second experimental data split. In this case, the parameters for evaluation were collected after the period of 100 epochs and are presented in Table 3.

The results obtained are noticeably worse than in the case of the previous data split, as can be seen, for instance, in the medium version model, where the AP₅₀ value was previously at the level of 0.568 (56.8%), and now, it is only 0.525 (52.5%). Moreover, in each case and for each parameter, the values are equal or lower, except for the recall, which in the case of the large model is higher by a value of 0.015.

4.1.3. 80% Training Data, 10% Test Data, and 10% Validation Data

The last experiment was carried out with 80% training data, 10% test data, and 10% validation data split. As in the previous cases, the whole process was based on training the model for 100 epochs and collecting the results, which are presented in Table 4.

In the case of the last data split, it can be seen that the results are the worst of all three proposed divisions. In this case, the medium model achieved the highest values, but compared to the two previous ones, they are significantly lower for precision, recall, and AP50. In this data split, each sub-model performed equally or worse in all the parameters examined, except the s model, which had a precision value of 0.02 higher than the second proposed division.

4.1.4. Confusion Matrix and Other Parameters for the Best Model

The next procedure was to study the confusion matrix for the training process of the model with the best results achieved in the first stage, to further distinguish the classes that were problematic and difficult to recognize. For YOLOv5, this was the medium model for the 90% training data and 10% validation data split. The mentioned confusion matrix is presented in Figure 6.

It can be noted that the basic classes such as point, x and y arrow, x-axis, and y-axis gave a satisfactory result in terms of classification. The most problematic cases were parabolas and hyperbolas, where the results varied, and the model was not sure how to classify them. This difficulty may be because these functions can be complicated and confused with others such as polynomials or trigonometric functions, making it harder to recognize them correctly, especially taking into account the proposed division between the left and right parts of the graph within the parabola and hyperbola classes. It is also worth noting that, despite their complexity and the need for contextual cues to distinguish them from each other, the trigonometric function classes also achieved acceptable results.

4.2. YOLOv6

YOLOv6 is the successor to YOLOv5 and was chosen as the second version to verify the training results in the context of adapting mathematical graphs and charts to the needs of the blind. The authors of this version developed four scaled models that operate on a 640 × 640 input.

4.2.1. 90% Training Data and 10% Validation Data

The first experiment followed the same route, using the 90% training data and 10% validation data split. After training the models for 100 epochs, the results were analyzed and compared (Table 5).

The results obtained indicate that the YOLOv6 is an inferior fit to analyze the dataset provided. This is clear by looking at the AP₅₀ parameter, where this version achieved a maximum value of 0.413 (41.3%), as well as the recall, where the average value for all models oscillates around 0.427. A positive aspect of YOLOv6 can be seen in the results for precision, where the parameter reached high values for such low AP₅₀ and recall.

4.2.2. 80% Training Data and 20% Validation Data

In the second experiment, the data were divided into 80% training data and 20% validation data. In this case, as in all the previous ones, the parameters for evaluation were collected after a period of 100 epochs and are presented in Table 6.

The values obtained show that, as in the case of YOLOv5, the second data split achieves visibly poorer results. The parameters evaluated are lower than in the case of the first split, with differences ranging from 0.010 to 0.03. In all cases, the precision, recall, and AP₅₀ are inferior, and there is no higher value to be noted in this experiment, but a positive aspect to be said is, as in the first experiment, the first parameter from the table with values that are relatively high in contrast to all the other once.

4.2.3. 80% Training Data, 10% Test Data, and 10% Validation Data

The last experiment was carried out with 80% training data, 10% test data, and 10% validation data split. The whole process was based on training the model for 100 epochs and obtaining the results, which are presented in Table 7.

The last experiment achieved the worst results of all three proposed data splits. These results are far poorer than all the other models, and due to the training performed, it has become clear that this data split combined with YOLOv6 cannot produce satisfactory results. The AP₅₀ has barely reached the point of 0.4 (40%) with the precision for the third time, achieving high values in relation to all the other parameters.

4.2.4. Confusion Matrix for the Best Model

The next experimental step was to examine the confusion matrix for the training process of the model with the best results achieved in the first stages. This allowed us to further distinguish the classes that were troublesome and difficult to detect for the analyzed network version. For YOLOv6, from all the trained models, it was the large model for the 90% training data and 10% validation data split. The mentioned confusion matrix is presented in Figure 7.

The simple classes such as y-axis, point, or ticks achieved satisfactory results, but the bigger problem appears in the case of the parabola, hyperbola, and, surprisingly, x-axis elements. This may also be the reason for the poor results of the models in the AP₅₀ parameter. Moreover, despite the good results for trigonometric functions, it is visible that the context is crucial, distinguishing them from each other as illustrated by the errors made between the sines and cosines or tangent and cotangent classification. It is also important to note that this version achieved a good result for classes that could seem difficult at first sight, for example, the polynomial.

4.3. YOLOv7

YOLOv7 was the next version chosen for experiments to verify the results of model training in the context of adapting mathematical graphs and diagrams to the needs of the blind. The authors of this version developed three scaled models working on a 640 × 640 input.

4.3.1. 90% Training Data and 10% Validation Data

The first experiment conducted on the YOLOv7 version used the 90% training data and 10% validation data division. After training the models for 100 epochs, the results were accumulated and analyzed (Table 8).

The results suggest that the training process was completed without a problem. The values obtained indicate that the extra-large model was the best fit, with an AP₅₀ of 0.480 (48%) and precision and recall at the levels of 0.594 and 0.547, respectively. There is an interesting gap in the values between the tiny model and the normal model, which is not observed in the case of the normal to extra-large models.

4.3.2. 80% Training Data and 20% Validation Data

The next experiment performed on this network version used the second proposed data split: 80% training data and 20% validation data. The training parameters obtained for this process are presented in Table 9.

The values obtained are lower for the second data split, with the best model achieving an AP₅₀ value of 0.440 (44.0%). It is interesting to note that each model achieved the best value in one of the parameters: the t version has the highest AP₅₀, the 7 model has the best recall, and the last x version has the highest precision.

4.3.3. 80% Training Data, 10% Test Data, and 10% Validation Data

The last experiment was carried out with the third data split: 80% training data, 10% test data, and 10% validation data. As in the previous cases, the whole process was based on training the model for 100 epochs and collecting the results (Table 10).

In the case of the third data split, the results obtained are similar to those of the 80/20 data division. The best model was the normal model with an AP₅₀ value of 0.435 (43.5%). An interesting fact about this experiment is that for this data division, the middle model proved to be the one with the highest results, while the x and t had mostly inferior parameters.

4.3.4. Confusion Matrix for the Best Model

The next experimental phase focused on examining the confusion matrix of the training process of the model with the best results. This allowed us to further describe the classes that were difficult to detect for the network version. For YOLOv7, this was the extra-large model for the 90% training and 10% validation data split, which achieved an AP₅₀ of 0.480 (48%). The confusion matrix is shown in Figure 8.

The primary classes such as y-axis, point, ticks, and x and y arrows achieved satisfactory results. A problem arises when analyzing parabolas and hyperbolas, which have been problematic in all instances so far. In YOLOv7, the second mentioned class proved to be the most mismatched in the case of detecting the negative or positive coefficient of the function, as well as confusing the right and left parts. Another surprising fact is that, as in the case of YOLOv6, the x-axis element was misclassified very often. This may be the reason why the results of the models are below 50% for the AP₅₀ parameter. Good results mixed with confusion for trigonometric functions indicate, as before, that the context is crucial in distinguishing between them. It is also important to note that the asymptotic elements also generated worse results than in the previous cases.

4.4. YOLOv8

YOLOv8 was published by the same authors as YOLOv5 and was therefore chosen for experiments to verify the results of model training in the context of adapting mathematical graphs and diagrams to the needs of the blind. The authors of this version generated five scaled models working on a 640 × 640 input.

4.4.1. 90% Training Data and 10% Validation Data

In the first experiment, the 90% training data and 10% validation data split was used. After training the models for 100 epochs, the results were accumulated and compared (Table 11).

The results indicate that this version has a certain applicability potential, taking into account the achieved values where for the best-performing model, the AP₅₀ parameter was equal to 0.524 (52.4%), the precision was 0.667, and the recall was 0.493. The smaller models also performed well and have potential applications, except for the nano model where the AP₅₀ was equal to 0.439 (43.9%).

4.4.2. 80% Training Data and 20% Validation Data

The second experiment carried out on YOLOv8 used the data split of 80% training data and 20% validation data. The training parameters obtained for this process are shown in Table 12.

The values obtained show that the second data split produced worse results for YOLOv8, with the best model being the extra-large, for which the AP₅₀ was 0.493 (49.3%), the precision was 0.669, and the recall was 0.459. On a positive note, although the two metrics are lower than in the first experiment, the precision remained at the same high level, even exceeding 0.6 for one model.

4.4.3. 80% Training Data, 10% Test Data, and 10% Validation Data

The last experiment was performed with the third data split: 80% training data, 10% test data, and 10% validation data. As in the prior cases, the whole process was based on the model being trained for 100 epochs, and the results were collected (Table 13).

The last experiment produced considerably worse results than the two previous data splits, with the best also being the x model, but with values of AP₅₀ equal to 0.473 (47.3%), precision barely above 0.6, and recall of 0.451. On the positive side, the precision is still high compared to the drop in the other parameters.

4.4.4. Confusion Matrix for the Best Model

Further analysis focused on studying the confusion matrix of the training process for the model with the best results. This allowed the classes to be identified that were the most challenging for the network version to detect. For YOLOv8, this was the extra-large model for the 90% training and 10% validation data split, with an AP₅₀ value of 0.524 (52.4%) and a precision of 0.667. The aforementioned confusion matrix is shown in Figure 9.

The results for the primary classes, such as x and y arrows and points, are at a satisfactory level. It is important to note that the values for problematic classes for previous versions are also at a good or excellent level, for example, the x-axis and asymptotes. The most difficult to classify were objects connected to hyperbolas, but the level of mismatch was significantly lower than for YOLOv7 and YOLOv6. Trigonometric functions also remained at an acceptable classification rate with fewer misclassifications. These observations are reflected in the model parameters, which are higher than in the previous versions, except for YOLOv5.

4.5. YOLOv9

YOLOv9 as the successor to YOLOv8 was chosen as the next version to verify the training results in the context of adapting mathematical graphs and charts to the needs of the blind. The authors of this version delivered five scaled models that operate on a 640 × 640 input.

4.5.1. 90% Training Data and 10% Validation Data

The first experiment was performed using the 90% training data and 10% validation data split. After training the models for 100 epochs, the results were compared (Table 14).

The results obtained indicate that this version has slightly worse results than the previous leading one. The best model in this experiment was the e model with an AP₅₀ value of 0.472 (47.2%), precision of 0.623, and recall of 0.497. Although all models except e performed poorly on the AP₅₀ parameter, the precision values were relatively high and increased with each complexity level, and the recall values remained comparable between models.

4.5.2. 80% Training Data and 20% Validation Data

The next experiment performed used the data split of 80% training data and 20% validation data. The training results obtained are shown in Table 15.

The results are visibly lower than in the first experiment, with the best model being the e model with values for the AP₅₀, precision, and recall of 0.440 (44.0%), 0.602, and 0.449, respectively. Despite the lower numbers, the trend towards high precision, even above 0.6 in one model, and a stable recall has been maintained.

4.5.3. 80% Training Data, 10% Test Data, and 10% Validation Data

The last experiment was performed with the third data partition, consisting of 80% training data, 10% test data, and 10% validation data. As in the previous cases, the whole process consisted of training the model for 100 epochs and collecting the results (Table 16).

The results for the third proposed data split are visibly lower than in the two previous experiments, with the AP₅₀ value even falling below 0.4 (40%) in one case. The trend remains the same for the other two parameters—relatively high precision with stable recall.

4.5.4. Confusion Matrix for the Best Model

The next step in the carried-out analysis involved examining the confusion matrix of the training process for the model with the best results. This allowed us to identify the objects that were the most difficult for the YOLOv9 version to detect. For YOLOv9, this was the e model for the 90% training and 10% validation data split. The AP₅₀ value for the mentioned model was 0.472 (47.2%), and the precision was 0.63. The aforementioned confusion matrix is shown in Figure 10.

The generated confusion matrix shows that the primary classes such as x and y ticks, x and y arrows, or points gave satisfactory results. As in the previous cases, the problematic objects were related to parabolas and hyperbolas, where the misclassifications are clearly visible. In the case of trigonometric functions, the results are promising, but the level of mismatches is higher than in the previous cases.

4.6. YOLOv10

YOLOv10 was the next version chosen for experiments to verify the results of model training in the context of adapting mathematical graphs and diagrams to the needs of the blind. The authors of this version created the largest number of scaled models and delivered six of them working on a 640 × 640 input.

4.6.1. 90% Training Data and 10% Validation Data

In the experiment, the first proposed data split with the 90% training data and 10% validation data split was used. After training the models for 100 epochs, the results were collected for comparison. The values obtained are shown in Table 17.

The results obtained show that the medium model achieved the best results, with an AP₅₀ value close to 50%, a precision above 0.6, and a recall close to 0.5. The values are worse than in some other prior versions, but the high precision and the stable recall allow it to have some application potential. An interesting observation can be made when inspecting the complexity of the model—up to the medium model, the values were growing, but increasing the complexity further caused the scores to fall.

4.6.2. 80% Training Data and 20% Validation Data

The second experiment used the data division of 80% training data and 20% validation data. The training parameters obtained for this process were accumulated and compared (Table 18).

The obtained values are visibly lower, with the best model achieving only 0.439 (43.9%) AP₅₀, 0.564 precision, and 0.422 recall. As in the first experiment, the tendency is visible whereby the values increase until the medium model and then decrease.

4.6.3. 80% Training Data, 10% Test Data, and 10% Validation Data

The final experiment was performed with the last data split: 80% training data, 10% test data, and 10% validation data. The procedure was based on training the model for 100 epochs and collecting the results (Table 19).

As in other versions, the results for the last data split are the lowest of all three experiments, with AP₅₀ values oscillating around 0.4 (40%) and recall around 0.4. It is important to note that the precision values are relatively high, while the other values are lower.

4.6.4. Confusion Matrix for the Best Model

The next experiment focused on the analysis of the confusion matrix for the training process of the model with the best results. This made it possible to distinguish the elements that were most problematic to detect. For this version, this was the medium model for the 90% training and 10% validation data split, with an AP₅₀ value of 0.493 (49.3%), precision of 0.601, and recall of 0.473. The confusion matrix is shown in Figure 11.

The confusion matrix shows that the primary classes, such as arrows and axes, gave satisfactory results. The most problematic cases were related to hyperbolas and parabolas, where the level of confusion is visible. In addition, the misclassification of trigonometric functions is also higher than in previous versions. The lower results can also be explained by the poor AP results for asymptotes, where the values were significantly higher in the better-performing versions.

4.7. YOLOv11

YOLOv11 was released by the same authors as YOLOv5 and YOLOv8 and was chosen for experiments to verify the results of training the models in the context of adapting mathematical graphs to the needs of the blind. The authors of this version delivered five scaled models working on a 640 × 640 input.

4.7.1. 90% Training Data and 10% Validation Data

In the first experiment, the data were split into 90% training data and 10% validation data. After training for 100 epochs, the values were prepared for comparison. The results are shown in Table 20.

The results obtained for the best model were at the level of 0.463 (46.3%) for AP₅₀, 0.595 for precision, and 0.487 for recall. The trend of better performance with each higher complexity can be observed, except for the large model. The results are relatively low, with the positive aspects being the high precision value and the stable recall.

4.7.2. 80% Training Data and 20% Validation Data

The second training process used the data split of 80% training data and 20% validation data. The resulting parameters were accumulated for comparison (Table 21).

The results obtained for the second data split are relatively lower than in the first experiment, with the highest AP₅₀ being 0.425 (42.5%). The two trends observed in the first data split are also visible in this case, the first being that the l model is an exception in terms of values increasing with complexity, followed by the relatively high precision values with a stable recall.

4.7.3. 80% Training Data, 10% Test Data, and 10% Validation Data

The last experiment was performed with the third data split of 80% training data, 10% test data, and 10% validation data. The results of training the model for 100 epochs are shown in Table 22.

The results obtained are lower than in the case of the first experiment, but it is important to note that in the YOLOv11 version the values for the second and third data splits are comparable and almost equal. The best model achieved an AP₅₀ of 0.429 (42.9%), a precision of 0.598, and a recall of 0.459. The tendency for a model to score lower as complexity increases, with high precision and stable recall, is maintained.

4.7.4. Confusion Matrix for the Best Model

The following analysis involved the confusion matrix for the training process of the model with the best-achieved results. This made it possible to investigate which classes were the most problematic to detect. For YOLOv11, this was the extra-large model for the 90% training and 10% validation data split, which achieved an AP₅₀ value of 0.463 (46.3%), a precision of 0.594, and a recall of 0.487. The resulting confusion matrix is shown in Figure 12.

The confusion matrix shows the same problems as in previous versions, where the basic classes such as arrows and axes are recognized with satisfactory results, while the more complex classes such as parabolas and hyperbolas, together with trigonometric functions, continue to be mismatched. In the case of YOLOv11, the same interesting problem can be observed that the asymptotic classes also achieve poor results. All these factors may be the reason for the low achieved AP₅₀ value.

4.8. Real-World Scenarios and Conclusions

The final experiment involved testing the models in a simulated real-world scenario. The simulation included generating new charts and counting the number of elements in them to ensure that the training process did not involve those graphics, using the best model from each version to analyze the chart, and as a last step, calculating the number of correctly recognized objects. The results of this experiment for 100 images with 1562 objects in them are summarized in Table 23. In addition, an example output of the real-world scenario test for one of the generated graphics is shown in Figure 13.

It can be observed that the models that achieved the best results in the training parameters were also able to perform well in the simulated field test. The two leading models, YOLOv5m and YOLOv8x, proved that by using methods such as changing the optimization parameters, prolonged training, and enchaining the dataset, they can be a good solution in the process of adapting mathematical graphs to the needs of the blind.

4.9. Conclusions

To summarize the performance of the compared models, it is important to note that two versions of the network were leaders in terms of the achieved results. YOLOv5 and YOLOv8 obtained the best values in the training process, as shown in Table 24 and Figure 14.

Although the results are good for the two mentioned versions, some key points need to be corrected before they can be implemented in a tool for teachers to adapt materials to the needs of the blind. First and foremost, the problematic classes such as parabolas and hyperbolas, which were visible in the confusion matrix for each version, need to be adjusted. Secondly, more time needs to be spent on training for both versions. Thirdly, the dataset needs to be revised and enriched with new data or adapted to the current images to make it a better fit for the proposed models. Figure 14 shows that although YOLOv5 has higher AP₅₀ values, YOLOv8 has slightly higher precision values. The theory of how high these values can go needs to be studied further.

It is also important to note that although two versions achieved the highest results, the others should also be corrected. Their scores were lower, but by analyzing the causes of the problems, they can also provide valuable clues as to how to achieve better results in the future, or even surpass the current top performers. This can be done by the means already mentioned, such as looking more closely at the datasets and making changes, analyzing the optimization parameters more closely, and extending the training process.

5. Discussion and Conclusions

The article described a brief history of the development of YOLO-type neural networks, highlighting the distinctive features of each version and exploring the progress made in a relatively short time. However, the main focus was to analyze the applicability of this type of AI in the context of adapting graphical information to the needs of blind people. The experimental steps were described in detail, including the phase associated with generating and preparing the dataset from scratch. The available testing environment allowed the setting of a training limit of 100 epochs with three separate data divisions for each developed complexity level model in each version. Although the duration of the experiments was in some cases longer than expected, the authors were able to carry out each training process and obtain the results for the most important metrics.

The values obtained for each important parameter allowed a comparison to be made for each version, describing in detail how each data division and model complexity affected the training process. For each experiment, the default optimization parameters were used to ensure consistency throughout the experiments. The next step in the results section, the analysis of the confusion matrix, allowed us to identify problematic classes and difficult-to-classify objects. The final part of the results presented a real-world scenario test and solidified the conclusions drawn from the training metric comparison.

In the experiments carried out, the best results were obtained for the YOLOv5 and YOLOv8 versions, which at this point allows us to define the next stages of development to create a complete automated solution in terms of material adaptation for the needs of the blind. This idea was the basis of the experiments performed and described, since the models developed are the first element of a complex tool intended to help teachers in their everyday teaching duties. The best-performing networks identified in this research will be further analyzed, and the influence of training-related parameters on the achieved classification accuracy will be verified. The behavior of the parameters will be monitored in the processes that include extended training, changes in the dataset, and optimization and fine-tuning of the algorithms.

Although the models obtained in this publication have some problems, they can already be used in further work as a starting point for a tool to automate the process of adapting materials to the needs of blind people. The next planned elements include an intelligent editor for .svg graphics and generative models for creating descriptions for the blind. The editor will allow teachers to freely edit parameters related to scanned and drawn images from textbooks based on the results of a model. This is expected to speed up a process that is currently very time-consuming when working in inclusive classrooms with both sighted and blind students at the same time.

On the negative side, it should be mentioned that the results obtained for some classes need to be significantly improved, including parabolas, hyperbolas, trigonometric functions, and in some cases asymptotes and axes. Before developing a final version of the program to serve as an editor, the results obtained for these elements need to be improved. This is planned to be achieved by enriching the dataset with more cases of problematic classes and by diversifying the examples, taking into account the number of samples with erroneous or biased data. It is also planned to extend the training period, depending on the availability of computational resources, and to change the optimization parameters and fine-tune the model. This should help to achieve more accurate parameters in the future.

In conclusion, the networks described and analyzed in this article are at this point applicable in the specific context but also need further development and time to work on their precision, so that a complete automated tool to help teachers working with blind and visually impaired students can be proposed.

Author Contributions

Conceptualization, M.K. and M.M.; methodology, M.K.; software, M.K.; validation, M.K. and M.M.; formal analysis, M.M.; investigation, M.K.; resources, M.M.; data curation, M.K.; writing—original draft preparation, M.K.; writing—review and editing, M.M.; visualization, M.K.; supervision, M.M.; project administration, M.K.; funding acquisition, M.K. and M.M. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Dataset available on request from the authors.

Conflicts of Interest

The authors declare no conflicts of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. The schematics of computing the IoU value for YOLO networks.

Figure 2. An example of the effects of the NMS algorithm.

Figure 3. The proposed class decomposition for mathematical graphs.

Figure 4. An example of the labeling process for an image contained in the generated dataset.

Figure 5. The steps to calculate precision and recall values.

Figure 6. Confusion matrix for the YOLOv5m model trained for 100 epochs with a 90/10 dataset split.

Figure 7. Confusion matrix for the YOLOv6L model trained for 100 epochs with a 90/10 dataset split.

Figure 8. Confusion matrix for the YOLOv7x model trained for 100 epochs with a 90/10 dataset split.

Figure 9. Confusion matrix for the YOLOv8x model trained for 100 epochs with a 90/10 dataset split.

Figure 10. Confusion matrix for the YOLOv9e model trained for 100 epochs with a 90/10 dataset split.

Figure 11. Confusion matrix for the YOLOv10m model trained for 100 epochs with a 90/10 dataset split.

Figure 12. Confusion matrix for the YOLOv11x model trained for 100 epochs with a 90/10 dataset split.

Figure 13. Example results from the conducted real-world scenario experiment.

Figure 14. Comparison of training parameters for the versions with the best results.

Table 1

Summarization of AP parameters for the main YOLO versions.

YOLO Version	Scaled Models	AP	AP₅₀
YOLOv1	-	63.4%	-
YOLOv2	-	78.6%	-
YOLOv3	-	36.2%	60.6%
YOLOv4	-	43.5%	65.7%
YOLOv5	n	-	45.7%
	s	-	56.8%
	m	-	64.1%
	l	-	67.3%
	x	-	68.9%
YOLOv6	N	-	37.5%
	S	-	45.0%
	M	-	50.0%
	L	-	52.8%
YOLOv7	t	38.7%	56.7%
	7	51.4%	69.7%
	x	53.1%	71.2%
YOLOv8	n	37.3%	-
	s	44.9%	-
	m	50.2%	-
	l	52.9%	-
	x	53.9%	-
YOLOv9	t	38.3%	53.1%
	s	46.8%	63.4%
	m	51.4%	68.1%
	c	53.0%	70.2%
	e	55.6%	72.8%
YOLOv10	n	38.5%	-
	s	46.3%	-
	m	51.1%	-
	b	52.5%	-
	l	53.2%	-
	x	54.4%	-
YOLOv11	n	39.5%	-
	s	47.0%	-
	m	51.5%	-
	l	53.4%	-
	x	54.7%	-

Table 2

The validation results after 100 epochs of training for YOLOv5 with a 90/10 dataset split.

YOLOv5 Model	Precision	Recall	AP₅₀
n	0.565	0.514	0.466
s	0.570	0.523	0.492
m	0.652	0.556	0.568
l	0.708	0.512	0.555
x	0.649	0.540	0.553

Table 3

The validation results after 100 epochs of training for YOLOv5 with an 80/20 dataset split.

YOLOv5 Model	Precision	Recall	AP₅₀
n	0.538	0.507	0.437
s	0.563	0.523	0.470
m	0.612	0.530	0.525
l	0.595	0.537	0.519
x	0.621	0.530	0.521

Table 4

The validation results after 100 epochs of training for YOLOv5 with an 80/10/10 dataset split.

YOLOv5 Model	Precision	Recall	AP₅₀
n	0.522	0.522	0.432
s	0.565	0.513	0.461
m	0.603	0.527	0.514
l	0.589	0.536	0.509
x	0.611	0.527	0.510

Table 5

The validation results after 100 epochs of training for YOLOv6 with a 90/10 dataset split.

YOLOv6 Model	Precision	Recall	AP₅₀
N	0.586	0.415	0.382
S	0.594	0.424	0.397
M	0.602	0.433	0.410
L	0.605	0.437	0.413

Table 6

The validation results after 100 epochs of training for YOLOv6 with an 80/20 dataset split.

YOLOv6 Model	Precision	Recall	AP₅₀
N	0.581	0.410	0.373
S	0.591	0.421	0.393
M	0.597	0.428	0.406
L	0.601	0.433	0.408

Table 7

The validation results after 100 epochs of training for YOLOv6 with an 80/10/10 dataset split.

YOLOv6 Model	Precision	Recall	AP₅₀
N	0.577	0.401	0.367
S	0.585	0.413	0.389
M	0.591	0.422	0.402
L	0.598	0.427	0.405

Table 8

The validation results after 100 epochs of training for YOLOv7 with a 90/10 dataset split.

YOLOv7 Model	Precision	Recall	AP₅₀
t	0.540	0.503	0.448
7	0.585	0.538	0.478
x	0.594	0.547	0.480

Table 9

The validation results after 100 epochs of training for YOLOv7 with an 80/20 dataset split.

YOLOv7 Model	Precision	Recall	AP₅₀
t	0.548	0.518	0.440
7	0.555	0.547	0.434
x	0.572	0.523	0.438

Table 10

The validation results after 100 epochs of training for YOLOv7 with an 80/10/10 dataset split.

YOLOv7 Model	Precision	Recall	AP₅₀
t	0.532	0.525	0.417
7	0.554	0.539	0.435
x	0.537	0.542	0.428

Table 11

The validation results after 100 epochs of training for YOLOv8 with a 90/10 dataset split.

YOLOv8 Model	Precision	Recall	AP₅₀
n	0.586	0.457	0.439
s	0.598	0.471	0.475
m	0.605	0.481	0.489
l	0.626	0.474	0.506
x	0.667	0.493	0.524

Table 12

The validation results after 100 epochs of training for YOLOv8 with an 80/20 dataset split.

YOLOv8 Model	Precision	Recall	AP₅₀
n	0.574	0.436	0.418
s	0.571	0.446	0.432
m	0.594	0.451	0.449
l	0.596	0.457	0.465
x	0.669	0.459	0.493

Table 13

The validation results after 100 epochs of training for YOLOv8 with an 80/10/10 dataset split.

YOLOv8 Model	Precision	Recall	AP₅₀
n	0.574	0.428	0.413
s	0.587	0.426	0.421
m	0.589	0.438	0.438
l	0.597	0.434	0.449
x	0.604	0.451	0.473

Table 14

The validation results after 100 epochs of training for YOLOv9 with a 90/10 dataset split.

YOLOv9 Model	Precision	Recall	AP₅₀
t	0.556	0.479	0.434
s	0.596	0.468	0.445
m	0.574	0.472	0.437
c	0.601	0.483	0.447
e	0.623	0.497	0.472

Table 15

The validation results after 100 epochs of training for YOLOv9 with an 80/20 dataset split.

YOLOv9 Model	Precision	Recall	AP₅₀
t	0.573	0.440	0.412
s	0.591	0.444	0.422
m	0.572	0.440	0.414
c	0.585	0.445	0.416
e	0.602	0.449	0.440

Table 16

The validation results after 100 epochs of training for YOLOv9 with an 80/10/10 dataset split.

YOLOv9 Model	Precision	Recall	AP₅₀
t	0.565	0.428	0.399
s	0.585	0.435	0.411
m	0.571	0.422	0.402
c	0.579	0.430	0.406
e	0.582	0.452	0.435

Table 17

The validation results after 100 epochs of training for YOLOv10 with a 90/10 dataset split.

YOLOv10 Model	Precision	Recall	AP₅₀
n	0.571	0.455	0.429
s	0.582	0.463	0.465
m	0.601	0.473	0.493
b	0.591	0.466	0.473
l	0.587	0.459	0.467
x	0.579	0.453	0.437

Table 18

The validation results after 100 epochs of training for YOLOv10 with an 80/20 dataset split.

YOLOv10 Model	Precision	Recall	AP₅₀
n	0.535	0.431	0.402
s	0.545	0.443	0.413
m	0.564	0.442	0.439
b	0.552	0.454	0.429
l	0.549	0.448	0.416
x	0.571	0.436	0.407

Table 19

The validation results after 100 epochs of training for YOLOv10 with an 80/10/10 dataset split.

YOLOv10 Model	Precision	Recall	AP₅₀
n	0.527	0.425	0.399
s	0.557	0.435	0.406
m	0.571	0.441	0.414
b	0.563	0.439	0.408
l	0.554	0.431	0.402
x	0.561	0.442	0.401

Table 20

The validation results after 100 epochs of training for YOLOv11 with a 90/10 dataset split.

YOLOv11 Model	Precision	Recall	AP₅₀
n	0.569	0.477	0.439
s	0.585	0.475	0.446
m	0.598	0.483	0.451
l	0.591	0.479	0.448
x	0.594	0.487	0.463

Table 21

The validation results after 100 epochs of training for YOLOv11 with an 80/20 dataset split.

YOLOv11 Model	Precision	Recall	AP₅₀
n	0.568	0.454	0.415
s	0.574	0.452	0.421
m	0.583	0.453	0.422
l	0.574	0.453	0.421
x	0.574	0.456	0.425

Table 22

The validation results after 100 epochs of training for YOLOv11 with an 80/10/10 dataset split.

YOLOv11 Model	Precision	Recall	AP₅₀
n	0.551	0.453	0.411
s	0.585	0.443	0.419
m	0.601	0.461	0.424
l	0.592	0.451	0.417
x	0.598	0.459	0.429

Table 23

The results of a simulated real-world scenario for 100 generated images with 1562 objects in them.

Selected YOLO Model	Correctly Classified (%)
YOLOv5m 90/10 data split	82.3
YOLOv6L 90/10 data split	63.2
YOLOv7x 90/10 data split	71.4
YOLOv8x 90/10 data split	76.6
YOLOv9e 90/10 data split	64.2
YOLOv10m 90/10 data split	74.0
YOLOv11x 90/10 data split	70.1

Table 24

Comparison of the results obtained for the best model in each YOLO version.

Selected YOLO Model	Precision	Recall	AP₅₀
YOLOv5m 90/10 data split	0.652	0.556	0.568
YOLOv6L 90/10 data split	0.605	0.437	0.413
YOLOv7x 90/10 data split	0.594	0.547	0.480
YOLOv8x 90/10 data split	0.667	0.493	0.524
YOLOv9e 90/10 data split	0.623	0.497	0.472
YOLOv10m 90/10 data split	0.601	0.473	0.493
YOLOv11x 90/10 data split	0.594	0.487	0.463

References

1. Maćkowski, M.; Brzoza, P.; Kawulok, M.; Knura, T. Mobile E-Learning Platform for Audio-Tactile Graphics Presentation. Computers Helping People with Special Needs; Miesenberger, K.; Kouroupetroglou, G.; Mavrou, K.; Manduchi, R.; Covarrubias Rodriguez, M.; Penáz, P. Lecture Notes in Computer Science Springer International Publishing: Cham, Germany, 2022; Volume 13341, pp. 82-91. ISBN 978-3-031-08647-2

2. Grylls, B. The Digital Divide. New Electron.; 2019; 52, pp. 34-35. [DOI: https://dx.doi.org/10.12968/S0047-9624(22)61049-6]

3. Buehler, E.; Kane, S.K.; Hurst, A. ABC and 3D: Opportunities and Obstacles to 3D Printing in Special Education Environments. Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility—ASSETS ’14; Rochester, NY, USA, 20–22 October 2014; pp. 107-114.

4. Elliott, A. The Culture of AI: Everyday Life and the Digital Revolution; 1st ed. Routledge: London, UK, 2019; ISBN 978-1-315-38718-5

5. Lai, S.; Xu, L.; Liu, K.; Zhao, J. Recurrent Convolutional Neural Networks for Text Classification. Proceedings of the AAAI Conference on Artificial Intelligence; Austin, TX, USA, 25–30 January 2015; [DOI: https://dx.doi.org/10.1609/aaai.v29i1.9513]

6. Zhang, X.-Y.; Yin, F.; Zhang, Y.-M.; Liu, C.-L.; Bengio, Y. Drawing and Recognizing Chinese Characters with Recurrent Neural Network. IEEE Trans. Pattern Anal. Mach. Intell.; 2018; 40, pp. 849-862. [DOI: https://dx.doi.org/10.1109/TPAMI.2017.2695539] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28436845]

7. Akram, M.; El, C. Sequence to Sequence Weather Forecasting with Long Short-Term Memory Recurrent Neural Networks. Int. J. Comput. Appl.; 2016; 143, pp. 7-11. [DOI: https://dx.doi.org/10.5120/ijca2016910497]

8. Mahesh, B. Machine Learning Algorithms—A Review. Int. J. Sci. Res.; 2020; 9, pp. 381-386. [DOI: https://dx.doi.org/10.21275/ART20203995]

9. Wu, Y.; Feng, J. Development and Application of Artificial Neural Network. Wirel. Pers. Commun.; 2018; 102, pp. 1645-1656. [DOI: https://dx.doi.org/10.1007/s11277-017-5224-x]

10. Macukow, B. Neural Networks—State of Art, Brief History, Basic Models and Architecture. Computer Information Systems and Industrial Management; Saeed, K.; Homenda, W. Lecture Notes in Computer Science Springer International Publishing: Cham, Germany, 2016; Volume 9842, pp. 3-14. ISBN 978-3-319-45377-4

11. Chauvin, Y.; Rumelhart, D.E. Backpropagation: Theory, Architectures, and Applications; Lawrence Erlbaum Associates, Inc.: Hillsdale, NJ, USA, 1995; ISBN 978-0-203-76324-7

12. Kang, C.H.; Kim, S.Y. Real-Time Object Detection and Segmentation Technology: An Analysis of the YOLO Algorithm. JMST Adv.; 2023; 5, pp. 69-76. [DOI: https://dx.doi.org/10.1007/s42791-023-00049-7]

13. Tian, Y.; Yang, G.; Wang, Z.; Wang, H.; Li, E.; Liang, Z. Apple Detection during Different Growth Stages in Orchards Using the Improved YOLO-V3 Model. Comput. Electron. Agric.; 2019; 157, pp. 417-426. [DOI: https://dx.doi.org/10.1016/j.compag.2019.01.012]

14. Wu, D.; Lv, S.; Jiang, M.; Song, H. Using Channel Pruning-Based YOLO v4 Deep Learning Algorithm for the Real-Time and Accurate Detection of Apple Flowers in Natural Environments. Comput. Electron. Agric.; 2020; 178, 105742. [DOI: https://dx.doi.org/10.1016/j.compag.2020.105742]

15. Al-masni, M.A.; Al-antari, M.A.; Park, J.-M.; Gi, G.; Kim, T.-Y.; Rivera, P.; Valarezo, E.; Choi, M.-T.; Han, S.-M.; Kim, T.-S. Simultaneous Detection and Classification of Breast Masses in Digital Mammograms via a Deep Learning YOLO-Based CAD System. Comput. Methods Programs Biomed.; 2018; 157, pp. 85-94. [DOI: https://dx.doi.org/10.1016/j.cmpb.2018.01.017] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29477437]

16. Nie, Y.; Sommella, P.; O’Nils, M.; Liguori, C.; Lundgren, J. Automatic Detection of Melanoma with Yolo Deep Convolutional Neural Networks. Proceedings of the 2019 E-Health and Bioengineering Conference (EHB); Iasi, Romania, 21–23 November 2019; pp. 1-4.

17. Tan, L.; Huangfu, T.; Wu, L.; Chen, W. Comparison of RetinaNet, SSD, and YOLO v3 for Real-Time Pill Identification. BMC Med. Inf. Decis. Mak.; 2021; 21, 324. [DOI: https://dx.doi.org/10.1186/s12911-021-01691-8] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34809632]

18. Maćkowski, M.; Brzoza, P.; Kawulok, M.; Meisel, R.; Spinczyk, D. Multimodal Presentation of Interactive Audio-Tactile Graphics Supporting the Perception of Visual Information by Blind People. ACM Trans. Multimedia Comput. Commun. Appl.; 2023; 19, pp. 1-22. [DOI: https://dx.doi.org/10.1145/3586076]

19. Maćkowski, M.; Kawulok, M.; Brzoza, P.; Spinczyk, D. Method and Tools to Supporting Math Learning in Inclusive Education of Blind Students. Augmented Intelligence and Intelligent Tutoring Systems; Frasson, C.; Mylonas, P.; Troussas, C. Lecture Notes in Computer Science Springer Nature Switzerland: Cham, Germany, 2023; Volume 13891, pp. 42-53. ISBN 978-3-031-32882-4

20. Diwan, T.; Anirudh, G.; Tembhurne, J.V. Object Detection Using YOLO: Challenges, Architectural Successors, Datasets and Applications. Multimed. Tools Appl.; 2023; 82, pp. 9243-9275. [DOI: https://dx.doi.org/10.1007/s11042-022-13644-y] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35968414]

21. Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You Only Look Once: Unified, Real-Time Object Detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); Las Vegas, NV, USA, 27–30 June 2016; pp. 779-788.

22. Girshick, R. Fast R-CNN. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV); Santiago, Chile, 7–13 December 2015; pp. 1440-1448.

23. Everingham, M.; Van Gool, L.; Williams, C.K.I.; Winn, J.; Zisserman, A. The Pascal Visual Object Classes (VOC) Challenge. Int. J. Comput. Vis.; 2010; 88, pp. 303-338. [DOI: https://dx.doi.org/10.1007/s11263-009-0275-4]

24. Redmon, J.; Farhadi, A. YOLO9000: Better, Faster, Stronger. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); Honolulu, HI, USA, 21–26 July 2017; pp. 6517-6525.

25. Redmon, J.; Farhadi, A. YOLOv3: An Incremental Improvement. arXiv; 2018; arXiv: 1804.02767

26. Bochkovskiy, A.; Wang, C.-Y.; Liao, H.-Y.M. YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv; 2020; arXiv: 2004.10934

27. Jocher, G. YOLOv5 by Ultralytics. arXiv; 2020; arXiv: 2207.02696

28. Wang, C.-Y.; Bochkovskiy, A.; Liao, H.-Y.M. Scaled-YOLOv4: Scaling Cross Stage Partial Network. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; Virtual Conference, 20–25 June 2021; pp. 13029-13038.

29. Wang, C.-Y.; Yeh, I.-H.; Liao, H.-Y.M. You Only Learn One Representation: Unified Network for Multiple Tasks. arXiv; 2021; arXiv: 2105.04206

30. Ge, Z.; Liu, S.; Wang, F.; Li, Z.; Sun, J. YOLOX: Exceeding YOLO Series in 2021. arXiv; 2021; arXiv: 2107.08430

31. Li, C.; Li, L.; Jiang, H.; Weng, K.; Geng, Y.; Li, L.; Ke, Z.; Li, Q.; Cheng, M.; Nie, W. et al. YOLOv6: A Single-Stage Object Detection Framework for Industrial Applications. arXiv; 2022; arXiv: 2209.02976

32. Wang, C.-Y.; Bochkovskiy, A.; Liao, H.-Y.M. YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; Vancouver, BC, Canada, 17–24 June 2023; pp. 7464-7475.

33. Xu, X.; Jiang, Y.; Chen, W.; Huang, Y.; Zhang, Y.; Sun, X. DAMO-YOLO: A Report on Real-Time Object Detection Design. arXiv; 2023; arXiv: 2211.15444

34. Ding, X.; Zhang, X.; Ma, N.; Han, J.; Ding, G.; Sun, J. RepVGG: Making VGG-Style ConvNets Great Again. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); Nashville, TN, USA, 20–25 June 2021; pp. 13728-13737.

35. Wang, C.-Y.; Liao, H.-Y.M.; Yeh, I.-H. Designing Network Design Strategies Through Gradient Path Analysis. arXiv; 2022; arXiv: 2211.04800

36. Jocher, G.; Qiu, J.; Chaurasia, A. Ultralytics YOLO 2023. Available online: https://github.com/ultralytics/ultralytics (accessed on 15 November 2024).

37. Deci-AI/Super-Gradients 2024. Available online: https://github.com/Deci-AI/super-gradients (accessed on 15 November 2024).

38. Long, X.; Deng, K.; Wang, G.; Zhang, Y.; Dang, Q.; Gao, Y.; Shen, H.; Ren, J.; Han, S.; Ding, E. et al. PP-YOLO: An Effective and Efficient Implementation of Object Detector. arXiv; 2020; arXiv: 2007.12099

39. Huang, X.; Wang, X.; Lv, W.; Bai, X.; Long, X.; Deng, K.; Dang, Q.; Han, S.; Liu, Q.; Hu, X. et al. PP-YOLOv2: A Practical Object Detector. arXiv; 2021; arXiv: 2104.10419

40. Xu, S.; Wang, X.; Lv, W.; Chang, Q.; Cui, C.; Deng, K.; Wang, G.; Dang, Q.; Wei, S.; Du, Y. et al. PP-YOLOE: An Evolved Version of YOLO. arXiv; 2022; arXiv: 2203.16250

41. Wang, C.-Y.; Yeh, I.-H.; Liao, H.-Y.M. YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information. European Conference on Computer Vision; Springer: Cham, Germany, 2024; pp. 1-21.

42. Wang, A.; Chen, H.; Liu, L.; Chen, K.; Lin, Z.; Han, J.; Ding, G. YOLOv10: Real-Time End-to-End Object Detection. arXiv; 2024; arXiv: 2405.14458

43. Khanam, R.; Hussain, M. YOLOv11: An Overview of the Key Architectural Enhancements. arXiv; 2024; arXiv: 2410.17725

44. Terven, J.; Córdova-Esparza, D.-M.; Romero-González, J.-A. A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS. Mach. Learn. Knowl. Extr.; 2023; 5, pp. 1680-1716. [DOI: https://dx.doi.org/10.3390/make5040083]

45. Lin, T.-Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Dollár, P.; Zitnick, C.L. Microsoft COCO: Common Objects in Context. Computer Vision–ECCV 2014; Fleet, D.; Pajdla, T.; Schiele, B.; Tuytelaars, T. Lecture Notes in Computer Science Springer International Publishing: Cham, Germany, 2014; Volume 8693, pp. 740-755. ISBN 978-3-319-10601-4

46. Maćkowski, M.; Kawulok, M.; Brzoza, P.; Janczy, M.; Spinczyk, D. An Alternative Audio-Tactile Method of Presenting Structural Information Contained in Mathematical Drawings Adapted to the Needs of the Blind. Appl. Sci.; 2023; 13, 9989. [DOI: https://dx.doi.org/10.3390/app13179989]

Word count: 13120

Show less

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.

Abstract

Translate

This publication focuses on verifying the AI effectiveness in adapting traditional educational materials to digital form, with a focus on blind people. Despite the existence of solutions to assist visually impaired people, the adaptation of graphics is still problematic. To address these challenges, the use of machine learning, which is becoming increasingly prominent in modern solutions, can be effective. Of particular note are YOLO neural networks, known for their ability to analyze images accurately and in real time. The potential of these networks has not yet been fully validated in the context of mathematical graphics for the visually impaired. This research allowed for the determination of the effectiveness of selected versions of YOLO in recognizing relevant elements in mathematical graphs and the identification of the advantages and limitations of each version. It also helped to point out further potential developments in adapting graphs to accessible forms for blind people. The obtained results indicate that YOLOv5 and YOLOv8 have the most potential in this field. This research not only highlights the applicability of machine learning to accessibility challenges but also provides a foundation for the development of automated tools that can assist teachers in inclusive classroom environments.

Details

Title

YOLO-Type Neural Networks in the Process of Adapting Mathematical Graphs to the Needs of the Blind

Author

Kawulok, Mateusz

; Maćkowski, Michał

First page

11829

Publication year

2024

Publication date