Classification of Cloud Particle Habits Using Transfer Learning with a Deep Convolutional Neural Network

Abstract

The habits of cloud particles are a significant factor impacting microphysical processes in clouds. The accurate identification of cloud particle shapes within clouds is a fundamental requirement for calculating various cloud microphysical parameters. In this study, we established a cloud particle image dataset encompassing nine distinct habit categories, totaling 8100 images. These images were captured using three probes with varying resolutions: the Cloud Particle Imager (CPI), the Two-Dimensional Stereo Probe (2D-S), and the High-Volume Precipitation Spectrometer (HVPS). Furthermore, this study performs a comparative analysis of ten different transfer learning (TL) models based on this dataset. It was found that the VGG-16 model exhibits the highest classification accuracy, reaching 97.90%. This model also demonstrates the highest recall, precision, and F1 measure. The results indicate that the VGG-16 model can reliably classify the shapes of ice crystal particles measured by both line scan imagers (2D-S, HVPS) and an area scan imager (CPI).

1. Introduction

Covering more than 70% of the Earth’s surface, clouds play a crucial role in the radiation balance of the Earth–atmosphere system and are a central focus of atmospheric science research [1,2]. Cloud microphysical characteristics are essential for describing cloud features and play a significant role in cloud parameterization schemes in weather and climate models [3,4], as well as in characterizing cloud radiation in atmospheric remote sensing [5,6]. Through the analysis of the shape and size of cloud particles, various cloud microphysical characteristics, such as effective particle radius, cloud phase states, extinction coefficients, liquid and ice water content, and single scattering properties, can be determined [4]. Furthermore, the morphology of ice crystal particles impacts their scattering properties [7], growth rates [8], and terminal velocities [9]. The scattering properties of ice crystals exert a substantial influence on the global climate and radiation balance [10] and are of great importance in inverse corrections applied through remote sensing techniques, such as satellites and radar [11]. Consequently, categorizing the shape of ice crystals is of paramount importance.

Initially, the classification of ice crystal particles primarily relied on thresholds based on the geometric feature parameters of ice crystal images to distinguish different morphological categories [12,13,14]. However, due to the disparate detection conditions and the heterogeneity in ice crystal characteristics collected from varied regional environments, there is a necessity to adopt distinct thresholds for particle shape identification. This method is beset with the challenge of determining appropriate universal thresholds, thereby compromising its general applicability. Over the last decade, traditional machine learning algorithms have made significant progress and garnered attention in atmospheric science. Techniques such as principal component analysis [15], support vector machine [16], decision trees [17], polynomial logistic regression [18], and k-medoid classification [19], among others, have been progressively applied to the automatic classification of particle shapes. However, traditional classification methods typically require manual feature engineering and cannot directly utilize particle images as input. The complex structure of ice crystal particles poses a challenge in designing effective feature representations, and traditional methods may struggle to adapt effectively to new data distributions [20].

In recent years, the rapid progress and widespread utilization of deep learning algorithms have made substantial contributions to advancements in the field of image processing and image classification. Deep learning has demonstrated its superiority over traditional machine learning methods across multiple domains, including remote sensing image processing [21], medical image processing [22], and multimodal information processing [23]. Presently, deep convolutional neural networks (CNNs) are employed for the task of classifying ice crystal particle images. Xiao et al. (2019) proposed TL-ResNet152, an automated ice crystal habit classification model utilizing deep CNN and transfer learning (TL) techniques [24]. This model achieved an accuracy surpassing 96% in categorizing ice crystals into ten distinct shape categories. To train and evaluate TL-ResNet152, a dataset comprising 7282 images was utilized, partitioned into training and testing sets with a ratio of 4:1. Touloupas et al. (2020) applied a CNN to categorize cloud particles obtained by a holographic imager into droplets, ice crystal particles, and artifacts with 96.8% accuracy [20]. Liao et al. (2021) proposed a CNN embedded with a hypergraphic convolutional module, Hy-INet, which achieved a high accuracy of 98.08% in a ten-category classification task for ice crystals [25]. Jaffeux et al. (2022) devised two models to classify ice crystal images acquired by the Precipitation Imaging Probe (PIP) and the Two-Dimensional Stereo (2D-S) Probe into sixand nineclasses, respectively, using a CNN model and obtained 85.44% and 91.02% accuracy [26]. Zhang et al. (2023) introduced a CNN-based classification method for 2D-S cloud particle images, achieving an average accuracy of 97% in classificationof eight cloud particle habits [27]. All of these studies have shown that CNN not only enhances the generalization capability but also requires minimal image preprocessing, enabling the direct utilization of ice crystal images as input to automatically extract features from the data and make reliable predictions.

However, in the realm of current ice crystal habit research, the images used for classifying ice crystal shapes are predominantly captured by specific probes, with few automatic ice crystal shape classification models applicable to both line scan imagers (e.g., Two-Dimensional Stereo Probe (2D-S), Precipitation Imaging Probe (PIP), and High-Volume Particle Spectrometer (HVPS)) and area scan imagers (e.g., Cloud Particle Imager (CPI)). Chen et al. (2023) proposed a CNN model trained and evaluated using ice crystal images from both a line scan imager and area scan imager, achieving an accuracy of over 90% [28]. To the best of our knowledge, this model is the first and only CNN model capable of simultaneously classifying ice crystal images from different types of instruments and achieving commendable classification results. However, this model demonstrated limited classification accuracy, and only ice crystal images sampled by CPI and 2D-S probes were utilized as datasets for model training. In this study, we manually constructed a cloud particle database containing nine distinct habit categories with a total of 8100 images. Building upon the CNN structure and employing transfer learning, this paper proposes an automatic ice crystal classification model named VGG-16. This model facilitates the automatic classification of ice crystal particles measured by both types of instruments, line scan imagers and area scan imagers, eliminating the need for tedious manual feature extraction from the images. The subsequent sections of this paper are organized as follows. Section 2 provides a brief overview of the three detection devices and the datasets used. Section 3 introduces the basic concepts of CNN and transfer learning, along with a description of the experimental platforms, training processes, and methods for evaluating model performance. Section 4 conducts a comparative analysis of the classification performance of several representative deep CNN models, selecting the most effective CNN model (VGG-16) for further evaluation, and finally analyzes the classification results of each probe’s test set separately. Section 5 summarizes and provides an outlook on the research work in this paper.

2. Instrument and Dataset

Presently, airborne in situ measurement is regarded as one of the most direct and effective methods for obtaining cloud microphysical characteristics [29]. Baumgardner et al. (2011) provide an overview of the instruments commonly used for cloud microphysical measurements. Airborne cloud particle probes can be categorized into two types based on their image recording methods: the first type uses a linear photodiode array, while the second type employs square photodetector arrays for instant imaging of ice crystal particles [30]. The 2D-S and HVPS probes utilize a linear photodiode array that scans at a rate proportional to aircraft true air speed. Images of ice crystals captured by both probes are binary, with each diode undergoing thresholding (typically 50%) to identify valid particle images [31,32]. In comparison to the HVPS, the 2D-S probe features two sets of vertically orthogonal linear arrays, each equipped with 128 photodiodes. In contrast, the CPI relies on a charge-coupled device (CCD) camera [33]. When a particle passes through the sample volume, it triggers a pulsed laser to emit a burst of light, and the resulting particle image is projected onto the digital CCD. This method requires pulsed illumination and a triggering system to detect the presence of an ice crystal within the sample volume, resulting in discontinuous particle measurement. Although measurement discontinuity is not a critical issue for ice crystal classification studies, it hinders the precise estimation of particle concentration [34]. In contrast to the 2D-S and HVPS probes, the CPI provides grayscale images with 256 levels, thus offering additional insights into the structure and transparency of the particle surface. Table 1 summarizes the cloud particle detection equipment used in this study and their principal properties.

For conducting a study on cloud particle shape recognition, a dataset comprising a sufficient number of accurately labeled particles is essential. Our dataset of particle images is derived from several field observation projects. Data for CPI particles were collected from East China’s Anhui Province during the Weather Modification Project in 2014 (WMP-Anhui) [35], North China’s Hebei Province during the Weather Modification Project in the eastern foothills of Taihang Mountain in 2017 (WMP-TH) [36], and six field observation projects: ATTREX [37], IDEAS-4 [38], POSIDON [39], PREDICT [40], SEAC4RS [41], and OLYMPEX [42]. The cloud types measured in China were mainly the stratiform and embedded convective clouds, while the other six field campaigns sampled cloud particles mainly in convective clouds. Particle images for the 2D-S probe were obtained from project WMP-TH and the IDEAS-4 field project. HVPS particle image data were obtained from project WMP-Anhui and the OLYMPEX field campaign. Specific details on these field observation projects are provided in Table 2. This studymanually constructeda cloud particle image dataset, and the habit categories in this dataset with their descriptions are presented in Table 3. The dataset contains 8100 representative images of particles, with 300 images for each of the different habit classes for each probe.

The size and shape of natural ice crystal particles exhibit intricate diversity, with the process of ice crystal formation and growth being equally complex. Vázquez-Martín et al. (2020) [43] expanded upon the Magono and Lee (1966) [44] classification method for snow crystals, ice crystals, and solid precipitation particles, increasing the number of ice crystal particle shape categories from 80 to 135. However, various researchers have adopted different numbers and types of shape categories in automatic classifiers for cloud particle images obtained from different probes. For instance, the classification method proposed by Praz et al. (2018) classifies CPI, 2D-S and HVPS probe images into sphere, column, plate, rosette, aggregate and compact particle, and the out-of-focus images were added to the classification of the 2D-S probe, while the plate crystal was removed from the HVPS classification [18]. Wu et al. (2020) employed nine distinct categories for classifying cloud particle images captured by the Cloud Imaging Probe (CIP), including tiny, sphere, column, needle, irregular, dendrite, graupel, plate, and aggregate [45]. Jaffeux et al. (2022) classified the particles from the 2D-S probe into nine categories—graupel, fragile aggregate, column, rosette, plate, sphere, capped column, artifact, and aggregate—and classified particle habits for the PIP into six categories: graupel, fragile aggregate, needle, plate, rosette, and rimed aggregate [26]. Chen et al. (2022) classified images of ice crystals sampled by 2D-S and CPI probes into ten categories: plate, plate aggregate, irregular, rosette, aggregate, sphere, graupel, dendrite, column, and column aggregate [28].

The HVPS probe has a significantly lower resolution compared to the 2D-S and CPI probes. As a result, the HVPS cannot capture images of small particles and struggles to accurately depict the more complex shapes of ice crystals. Considering these substantial differences in resolution among the HVPS, 2D-S, and CPI probes, as well as in comparison to other probes with varying resolutions (such as the CIP: 25 µm; PIP: 100 µm), this study classified particle images obtained from the HVPS, 2D-S, and CPI into nine categories: plate (Pla), rosette (Ros), sphere (Sph), column (Col), column aggregate (Col. Agg.), dendrite (Den), graupel (Gra), aggregate (Agg), and irregular (Irr). These nine categories were chosen primarily for cloud microphysical research, especially to reveal ice crystal growth mechanism inside a cloud. The designed classes in this study are rather comparable to those used in Jaffeux et al. (2022) [26], which were chosen primarily to account for the three ice growth mechanisms of vapor deposition, aggregation, riming. Figure 1 displays images of each habit captured by the HVPS, 2D-S, and CPI probes.

Since the projected 2D imagery does not describe the third particle dimension, the classification of particle habit from 2D images contains certain ambiguity. For example, because of the ice particle orientation to the laser illumination, the planar ice particles may give a rectangular projection, which may be recognized as a column. However, such errors appear randomly and estimated as no larger than a few percent [13].

3. Methodology

3.1. Network Architecture

A CNN is a feed-forward neural network that incorporates convolutional computation and has a deep structure [46,47]. Key characteristics of CNN include weight sharing and a local receptive field, which serve to reduce the number of network parameters. A typical CNN consists of an input layer, convolutional layers, pooling layers, fully connected layers, and an output layer. As one of the representative algorithms of deep learning, CNNs have been widely used in image classification tasks due to their powerful data processing capability [48]. However, in practical applications, obtaining a sufficiently large labeled image dataset for accurate classification can be challenging, and training with substantial image datasets becomes difficult without access to high-performance computing resources. Consequently, few researchers train new CNN models from scratch. Most pretrained CNNs are trained and optimized for convergence based on the ImageNet database to minimize false-prediction errors [49]. The pretrained image classification CNNs have learned a rich set of image features that can be leveraged as a basis for learning new classification tasks. Therefore, to enhance the accuracy of cloud particle habit classification, a relatively effective approach is to employ transfer learning. This involves using a publicly available pretrained CNN model as a starting point and then training the CNN based on the particle image dataset to fine-tune the network’s parameters. In this paper, we utilize ten pretrained CNN models for transfer learning on the ice crystal dataset: AlexNet, SqueezeNet, GoogleNet, EfficientNet-B0, ResNet (ResNet-18, ResNet-50, ResNet-101), ShuffleNet, and VGG (VGG-16, VGG-19) [50,51,52,53,54,55,56]. These ten models are classical and commonly used in computer vision classification tasks.

3.2. Training Details

The experimental operating platform used in this study included MATLAB R2022A and Anaconda 3.4. The hardware configuration consisted of an Nvidia Quadro RTX 4000 with 64GB RAM operating on Windows 11 OS.

The training details of the deep CNN model for ice crystal shape classification, based on transfer learning, are outlined as follows. Each of the HVPS, 2D-S, and CPI probes contains 300 images for each particle category, which were visually selected by us from the abovementioned field campaigns after strict data qualification control. First, we randomly divided the particle image data from these three probes in a ratio of 4:1 for the training set and test set, respectively. Then, the training and test sets for each of the three probes were combined. Before training, we applied the same data preprocessing steps to each image in the dataset: resizing all particle images to 227 × 227 × 3 and normalizing the image pixel matrix. These operations are designed to expedite model convergence and gradient descent speed. To prevent network overfitting, data augmentation operations such as flip transformations, scaling, or random angle rotation were implemented with a 50% probability before each input of particle images into the network. The cross-entropy function was selected as the loss function. To optimize the model parameters, the stochastic gradient descent (SGD) with momentum optimizer was used, and the decay rate of its momentum parameter was set to 0.9. Additionally, the initial learning rate was set to η = 10⁻⁴, with the learning rate halved after every 10 epochs. The batch size was configured as 16, and the training epoch was set to 20. Subsequently, we loaded the pretrained CNN model and adjusted the output of the final fully connected layer to match the nine ice crystal classes in our dataset. To obtain a suitable model for ice crystal habit classification, transfer learning was performed using the 10 different pretrained CNN models on the ice crystal dataset. Then, the results of different models during the training and testing processes were recorded, and the model configurations with the highest classification accuracy during testing were saved. Finally, the classification performance of the models was evaluated using the performance metrics described in Section 3.3. An overview of the training methodology is shown in Figure 2.

3.3. Performance Metrics

(1) Accuracy

The fundamental measure for assessing a classification model’s performance is accuracy, which represents the ratio of correctly classified samples to the total number of samples:

(1)$A_c\left(y,\stackrel{\wedge }{y}\right)={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^{N}eq(y_i,\stackrel{\wedge }{y_i}}),$

where ${\widehat{y}}_i$ is the predicted label of the $i$-th image predicted by the model, $y_i$ represents the corresponding true label, $N$ is the total sample count, and $eq\left(y_i,{\widehat{y}}_i\right)$ is the value of 1 if the predicted label of the ice crystal image is the same as true label, and 0 otherwise.

(2) Cross-entropy loss

Cross-entropy loss reflects the difference between the true distribution of ice crystal labels and the predicted probability distribution. The formula is computed as (De Boer et al. 2005) [57]:

(2)$H(p,q)=-{\displaystyle \sum _{i=1}^{N}p(x_i)\log \left[q(x_i)\right]},$

where $p\left(x_i\right)$ represents the label of the sample data and $q\left(x_i\right)$ is the classification model label predicted. A smaller cross-entropy loss value indicates that the predicted probability distribution closely aligns with the true value distribution.

(3) Standard deviation

Standard deviation of the model recognition accuracy can be used to measure the robustness of the model and is calculated as:

(3)$\sigma =\sqrt{{\displaystyle \frac{{\displaystyle \sum _{k=1}^N{\left(A_{ck}-A_c\right)}^2}}{N-1}}},$

where $A_{ck}$ is the model recognition accuracy for the $k$-th category of ice crystal particle and $A_c$ is the model recognition accuracy ($k$ = 1, 2, 3…9).

(4) Precision

Precision is the number of ice crystal particles correctly identified as the $k$-th category as a proportion of the total number of ice crystals identified as the $k$-th category:

(4)$P={\displaystyle \frac{TP}{TP+FP}},$

where TP is a positive example correctly judged as a positive example and FP is a negative example incorrectly judged as a positive example.

(5) Recall

Recall is the number of ice crystal particles correctly identified as the $k$-th category as a proportion of the total number of particles in the $k$-th category:

(5)$R={\displaystyle \frac{TP}{TP+FN}},$

where FN is a positive example incorrectly judged as a counterexample.

(6) F1 measure

F1 measure is the harmonic mean of precision and recall:

(6)$F_1={\displaystyle \frac{2TP}{2TP+FN+FP}}.$

(7) Macro-precision, macro-recall and macro-F1

Precision, recall and F1 measure are calculated separately for each category of ice crystals and then averaged to obtain macro-precision ($P_{macro}$), macro-recall ($R_{macro}$),and macro-F1 ($F_{1macro}$). These three performance metrics evaluate the overall performance of model. The calculation formula is:

(7)$P_{macro}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{k=1}^N{P}_k},$

(8)$R_{macro}={\displaystyle \frac{1}{N}}{\displaystyle \sum _{k=1}^N{R}_k},$

(9)$F_{1macro}=2{\displaystyle \frac{P_{macro}\ast R_{macro}}{P_{macro}+R_{macro}}}.$

4. Results and Discussion

4.1. Experimental Results of Ice Crystal Classification Model

During the training and testing phases, accuracy and cross-entropy loss values were primarily employed to evaluate and compare ten CNN models based on transfer learning. Subsequently, the model demonstrating the highest accuracy on the test set was selected for further analysis of other performance metrics. The accuracy and cross-entropy loss curves of the ten CNN models throughout the training and validation periods are depicted in Figure 3. Training and testing were conducted over 8100 iterations, with results indicating a rapid increase in accuracy for both the training and test sets across all CNN models, accompanied by a swift decrease in loss values. Furthermore, both accuracy and cross-entropy loss tended to stabilize around 15 epochs, exhibiting only minor fluctuations. For training, the EfficientNet-B0 model exhibited the poorest performance, characterized by the lowest accuracy and highest cross-entropy loss values. The SqueezeNet model was also ineffective, with its classification accuracy and cross-entropy loss only surpassing those of the EfficientNet-B0 model. Conversely, the VGG-16 model achieved the best classification results, boasting the highest accuracy and lowest cross-entropy loss. For the testing stage, the VGG-16 model similarly displayed the highest accuracy and lowest cross-entropy loss, while theEfficientNet-B0, SqueezeNet, and ResNet-18 models exhibited relatively lower classification accuracies and higher cross-entropy losses, with the remaining classification models performing worse than VGG-16. The discrepancy in validation accuracy between the worst- and best-performing CNN models was approximately 7% (see the top right panel of Figure 3).

To elucidate the differential performance of classification models, an examination was conducted to compare the highest classification accuracy, corresponding standard deviation, and the duration of training and testing times across each CNN model based on transfer learning (as detailed in Table 4). As for the highest accuracy, VGG-16 performed the best (97.90%), while the worst was 94.01% by EfficientNet-B0. The performances of the worst and best models were not substantially different (less than 4%), suggesting that all pretrained models do relatively well, at least from the point of the accuracy metric. The standard deviation of the recognition accuracy of VGG-16 model was smaller than that of the other nine transfer learning models, indicating better robustness in VGG-16. The time for the VGG-16 to train and test is not optimal and relatively long compared with the other models, which means that on some occasions, the VGG-16 model may not be the most ideal, but that would be under specific conditions. This may be related to its larger layer numbers [58]. Despite the VGG-16 model requiring a longer duration for training (7873s) and testing (28.39 s) compared to its counterparts, such as AlexNet, SqueezeNet, and ResNet, the trade-off is justified by its superior performance metrics. The slightly elevated computational cost is overshadowed by the model’s precision and the significant reduction in variability, as evidenced by its minimal standard deviation. This aspect is particularly commendable, given the complex nature of ice crystal particle classification, where accuracy and reliability are paramount.

The performance of VGG-16 in this three-probe dataset was similar to that in the CPI probe dataset, which was constructed and executed by Przybylo et al. (2022) [58]. The similar results and the lowest standard deviation in this study demonstrate the great ability and the robustness of VGG-16 in the task of ice particle habit classification.

4.2. Evaluation of VGG-16 Model

To better evaluate the performance of the VGG-16 model, a detailed assessment of the precision, recall, and F1 measure for each particle category was conducted (see Figure 4). Observations from Figure 4 reveal that the VGG-16 model achieved a 100% recall rate for the spherical category, while the recall rate for the aggregate category was comparatively lower at 95%. The classification of columnar and spherical ice crystals exhibited the best performance, likely due to their unique and easily recognizable features. In contrast, aggregates were the least accurately classified category, possibly because the formation of aggregates shares certain structural similarities with the basic morphology of other ice crystal categories. Moreover, except for the precision of rosette shape and the recall rates for graupel and aggregate shapes, the performance metrics for precision, recall, and F1 measure of other particle classifications exceeded 96%. This indicates that overall, the VGG-16 classification model demonstrates superior classification effectiveness for each category of particles, showcasing broad applicability. Additionally, the macro-average precision, recall, and F1 measure across the nine categories exceeded 97.5%. This demonstrates the stable and efficient overall classification effectiveness of the VGG-16 model.

The classification efficacy of the VGG-16 model on the test dataset was further assessed using confusion matrices, a method commonly utilized for evaluating the performance of classification models. Confusion matrices serve as a fundamental, intuitive, and straightforward metric for appraising the performance of classification models, facilitating comparison between the model’s predicted outcomes and the actual categories. The diagonal elements of the confusion matrix represent the number of samples correctly predicted. Figure 5 illustrates the confusion matrix results of the VGG-16 model for classifying cloud particles on the test dataset. Inthe figure, it is evident that the majority of ice crystal shapes are well distinguished. Specifically, the VGG-16 model exhibited minimal prediction errors in classifying column, irregular, plate, dendrite, and sphere shapes. However, some challenges were encountered in distinguishing between aggregate and column aggregate categories, potentially due to the high visual similarity in their morphological features. Nevertheless, considering the overall classification results from the test dataset, the VGG-16 model demonstrated high accuracy and robustness in classifying most ice crystal shapes. In summary, the VGG-16 model displayed a high degree of precision in classifying cloud particle shapes across the CPI, 2D-S, and HVPS datasets, meeting the research requirements.

We also investigated the performance of the VGG-16 model on the test sets from the CPI, 2D-S, and HVPS probes. Figure 6 displays the confusion matrices for the VGG-16 model across these probe test sets. Overall, the sphere morphology exhibited the most commendable performance across the particle image datasets from these three probes. As illustrated in Figure 6, the VGG-16 model demonstrated minimal classification prediction errors for aggregate, column, dendrite, graupel, irregulars, plate, dendrite, and sphere in the CPI test set; column, column aggregate, graupel, plate, and sphere in the 2D-S test set; and column, column aggregate, irregular, plate, dendrite, and sphere in the HVPS test set, with no more than one particle misclassified for each category in test dataset predictions. Notably, the overall classification performance of the VGG-16 model on the HVPS test dataset was inferior to that on the CPI and 2D-S test datasets. Table 5 presents the macro-average values of precision, recall, and F1 measure across the test datasets from the three different probes. It is evident from the table that the evaluation metrics of the VGG-16 classification model on the CPI test dataset surpassed those on the other two probe test datasets, with all metrics on the CPI probe test dataset exceeding 0.98. The evaluation metrics on the HVPS test dataset were the least favorable, slightly below those on the 2D-S test dataset. This discrepancy may be attributed to the lower pixel count of images captured by the HVPS probe, complicating the model’s ability to discern ice crystal morphologies from images with limited pixel resolution.

5. Conclusions

We manually constructed a cloud particle image dataset comprising nine habit categories totaling 8100 images utilizing observations from the airborne CPI, 2D-S, and HVPS probes. Through the transfer learning method, we employed 10 pretrained CNN models to establish automated classification models for cloud particle habits. We compared the classification accuracy, cross-entropy loss, standard deviation, and training and testing times of each model during their training and testing phases to evaluate model performance. Ultimately, we selected the VGG-16 model for its superior classification accuracy during testing. For a comprehensive assessment of the VGG-16 model’s performance across ten particle categories, we employed multiple performance measures to evaluate its classification effectiveness on the particle image test set. The results of this evaluation demonstrated that the VGG-16 model achieves a classification accuracy of 97.90%. Additionally, the classification model exhibited confusion with a small proportion of ice crystals, for example, misclassifying aggregates as column aggregate or dendrite and misclassifying column aggregate as rosette, but the confusion errors were within 3%.

Furthermore, we analyzed the classification ability of the VGG-16 model on the test sets from the CPI, 2D-S, and HVPS probes. Despite the HVPS images having a low pixel count and being binary, the VGG-16 model efficiently learned features related to the defined particle shape categories. Based on the results and analyses of these experiments, the VGG-16 classification model proposed in this study can effectively and automatically classify ice crystals of different shape categories from two types of imagers.

However, it is worth noting that the current accuracy of the VGG-16 model in recognizing aggregate, column aggregate, dendrite, and graupel still requires improvement. Future research should aim to broaden the particle image dataset by incorporating images from a wider range of atmospheric conditions and geographic locations, leveraging both additional field data and synthetic image generation techniques, such as generative adversarial networks (GANs), to enhance diversity. To improve model classification performance, exploring advanced neural network architectures, such as incorporating attention mechanisms or transformers, and conducting thorough hyperparameter optimization are recommended strategies. These steps are expected to refine the model’s ability to accurately differentiate among various ice crystal shapes. Additionally, incorporating images captured by other optical imaging probes such as PIP and four-level grayscale CIP could further enrich the ice crystal dataset and aid in the development of more powerful models for the automatic classification of ice crystal shapes.

Author Contributions

Y.X.: conceptualization, investigation, data curation, methodology, formal analysis, software, writing—original draft. R.J.: supervision, writing—review and editing. Q.L.: writing—review and editing. M.H.: conceptualization, data curation, supervision, writing—review and editing. All authors contributed equally to this work. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this article.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. Typical examples of ice crystal shapes for the HVPS, 2D-S and CPI probes(images standardized to the same size).

Figure 2. Overview of training methodology.

Figure 3. Accuracy (top) and cross-loss entropy curves (bottom) of ten CNN models based on transfer learning during training and testing stages.

Figure 4. The performance of the VGG-16 in each category on precision (P), recall (R) and F1 measure (F1) with the test dataset.

Figure 5. Confusion matrix of ice crystal particle classification results for the VGG-16 classification model on the test set (maximum possible value in the matrix is 180).

Figure 6. Confusion matrix of ice crystal particle classification results for the VGG-16 classification model on the CPI (a), 2D-S (b) and HPVS (c) probe test sets (maximum possible value in the matrix is 60).

Table 1

Overview of the cloud particle detection equipment used in this paper.

Specifications	CPI	2DS	HVPS
Resolution (μm/pixel)	2.3	10	150
Measurement technique	Square photodiode array	Linear photodiode array	Linear photodiode array
Measurement range (μm)	3–2355	10–1280	150–19,200
Image type	256 levels grayscale	Black and white	Black and white

Table 2

Details of data acquisition for field observation projects.

Campaign	Date	Aircraft	Location	Reference	Major Ice Crystals Found
WMP-Anhui	Jul 2014	King Air	East China’s Anhui Province	[35]	CPI: AllHVPS: All
WMP-TH	May 2017	King Air	North China’s Hebei Province	[36]	CPI: All2D-S:All
ATTREX	Jan–Mar 2014	Global Hawk	Western Pacific	[37]	CPI: Pla.Agg., Ros, Bud, Col, Agg
IDEAS-4	Nov 2011	NSF C-130	North Central U.S.	[38]	CPI: All2D-S:All
POSIDON	Oct 2016	NASA’s WB-57	Western Pacific	[39]	CPI: Ros, Bud, Col, Den, Gra, Agg
PREDICT	Aug–Sep 2010	Gulfstream-V	Atlantic	[40]	CPI: Pla, Pla.Agg., Sph, Den
SEAC4RS	Sep 2013	SPEC-Learjet	Gulf of Mexico	[41]	CPI: Pla, Pla.Agg., Col.Agg., Den, Gra, Agg
OLYMPEX	Nov 2015–May 2016	UND Citation	Olympic Mountains	[42]	HVPS: All

Note: ATTREX (Airborne Tropical Tropopause Experiment); IDEAS-4 (Instrument Development and Education in Airborne Science phase 4); POSIDON (Pacific Oxidants, Sulfur, Ice, Dehydration, and Convection Experiment); PREDICT (Pre-Depression Investigation of Cloud Systems in the Tropics); SEAC4RS (Studies of Emissions and Atmospheric Composition, Clouds/Climate Coupling by Regional Surveys); OLYMPEX (Olympic Mountain Experiment).

Table 3

Category descriptions.

Category	Description
Plate (Pla)	Ice crystals have a flat shape, similar to a thin plate structure.
Rosette (Ros)	Ice crystals with a central core and radiating branches, columnar polycrystals, resembling a flower.
Sphere (Sph)	Particles of water that are predominantly round or nearly round, including cloud droplets and raindrops.
Column (Col)	The prism facets on columnar ice crystals, one axis longer than the other, appearing to be rectangular in low-resolution, two-dimensional images.
Column Aggregate (Col. Agg.)	An aggregate formed by combining multiple columnar ice crystals together.
Dendrite (Den)	The six branch structures are uniformly distributed, without other ice crystals condensing on the branch corners.
Graupel (Gra)	Ice crystals encounter liquid water droplets during their descent and heavily rimed as a compact particle.
Aggregate (Agg)	Complex structures from multiple ice crystals joined together.
Irregular (Irr)	Ice crystals without a distinct shape or symmetry.

Table 4

The highest accuracy of ten transfer learning models on the test set and corresponding standard deviation, training time, and testing time of the models.

Transfer Learning Model	Number of Layers	${\mathbf{A}}_{\mathbf{c}}$ (%)	$\mathsf{\sigma }$ (%)	Training Time (s)	Testing Time (s)
AlexNet	8	97.16	2.14	1799	1.75
SqueezeNet	18	94.07	4.28	2017	1.80
GoogleNet	22	96.17	2.55	3240	3.96
EfficientNet-B0	82	94.01	4.50	13,607	10.54
ResNet-18	18	95.93	2.90	2148	2.72
ResNet-50	50	97.10	1.98	5517	4.94
ResNet-101	101	96.23	2.70	12,693	5.11
ShuffleNet	50	95.80	2.40	4507	4.57
VGG-16	16	97.90	1.59	7873	28.39
VGG-19	19	95.93	3.02	6776	28.61

Table 5

Comparison of $P_{macro}$, $R_{macro}$ and $F_{1macro}$ on three probe test sets.

	CPI	2D-S	HPVS
$P_{macro}$	0.9891	0.9763	0.9735
$R_{macro}$	0.9889	0.9759	0.9722
$F_{1macro}$	0.9890	0.9761	0.9728

References

1. Stephens, G.L.; Li, J.L.; Wild, M.; Clayson, C.A.; Loeb, N.; Kato, S.; L’Ecuyer, T.; Stackhouse, P.W.; Lebsock, M.; Andrews, T. An update on Earth’s energy balance in light of the latest global observations. Nat. Geosci.; 2012; 5, pp. 691-696. [DOI: https://dx.doi.org/10.1038/ngeo1580]

2. Zhou, C.; Zelinka, M.D.; Klein, S.A. Impact of decadal cloud variations on the Earth’s energy budget. Nat. Geosci.; 2016; 9, pp. 871-874. [DOI: https://dx.doi.org/10.1038/ngeo2828]

3. Shupe, M.D.; Daniel, J.S.; de Boer, G.; Eloranta, E.W.; Kollias, P.; Long, C.N.; Luke, E.P.; Turner, D.D.; Verlinde, J. A focus on mixed-phase clouds—The status of ground-based observational methods. Bull. Am. Meteorol. Soc.; 2008; 89, pp. 1549-1562. [DOI: https://dx.doi.org/10.1175/2008BAMS2378.1]

4. Wu, W.; McFarquhar, G.M. On the Impacts of Different Definitions of Maximum Dimension for Nonspherical Particles Recorded by 2D Imaging Probes. J. Atmos. Ocean. Technol.; 2016; 33, pp. 1057-1072. [DOI: https://dx.doi.org/10.1175/JTECH-D-15-0177.1]

5. Sassen, K.; Benson, S. A midlatitude cirrus cloud climatology from the facility for atmospheric remote sensing. Part II: Microphysical properties derived from lidar depolarization. J. Atmos. Sci.; 2001; 58, pp. 2103-2112. [DOI: https://dx.doi.org/10.1175/1520-0469(2001)058<2103:AMCCCF>2.0.CO;2]

6. Palchetti, L.; Natale, D.G.; Bianchini, G. Remote sensing of cirrus cloud microphysical properties using spectral measurements over the full range of their thermal emission. J. Geophys. Res. Atmos.; 2016; 121, pp. 10804-10819. [DOI: https://dx.doi.org/10.1002/2016JD025162]

7. Bi, L.; Yang, P. Improved ice particle optical property simulations in the ultraviolet to far-infrared regime. J. Quant. Spectrosc. Radiat. Transf.; 2017; 189, pp. 228-237. [DOI: https://dx.doi.org/10.1016/j.jqsrt.2016.12.007]

8. Kiliani, J.; Baumgarten, G.; Lübken, F.-J.; Berger, U. Impact of particle shape on the morphology of noctilucent clouds. Atmos. Chem. Phys.; 2015; 15, pp. 12897-12907. [DOI: https://dx.doi.org/10.5194/acp-15-12897-2015]

9. Zou, Z.; Chen, Q.; Lu, C.; Yang, L.; Zou, Q.; Sun, B. Parameterization of Mean Terminal Velocity of Hydrometeors in Convective Clouds. Atmosphere; 2022; 13, 1404. [DOI: https://dx.doi.org/10.3390/atmos13091404]

10. Yi, B.; Yang, P.; Baum, B.A.; L’Ecuyer, T.; Oreopoulos, L.; Mlawer, E.J.; Heymsfield, A.J.; Liou, K.-N. Influence of ice particle surface roughening on the global cloud radiative effect. J. Atmos. Sci.; 2013; 70, pp. 2794-2807. [DOI: https://dx.doi.org/10.1175/JAS-D-13-020.1]

11. Baran, A.J. A review of the light scattering properties of cirrus. J. Quant. Spectrosc. Radiat. Transf.; 2009; 110, pp. 1239-1260. [DOI: https://dx.doi.org/10.1016/j.jqsrt.2009.02.026]

12. Holroyd, E.W. Some techniques and uses of 2D-C habit classification software for snow particles. J. Atmos. Ocean. Technol.; 1987; 4, pp. 498-511. [DOI: https://dx.doi.org/10.1175/1520-0426(1987)004<0498:STAUOC>2.0.CO;2]

13. Korolev, A.; Sussman, B. A technique for habit classification of cloud particles. J. Atmos. Ocean. Technol.; 2000; 17, pp. 1048-1057. [DOI: https://dx.doi.org/10.1175/1520-0426(2000)017<1048:ATFHCO>2.0.CO;2]

14. Lawson, R.P.; Baker, B.A.; Zmarzly, P.; O’connor, D.; Mo, Q.; Gayet, J.-F.; Shcherbakov, V. Microphysical and optical properties of atmospheric ice crystals at South Pole Station. J. Appl. Meteorol. Climatol.; 2006; 45, pp. 1505-1524. [DOI: https://dx.doi.org/10.1175/JAM2421.1]

15. Lindqvist, H.; Muinonen, K.; Nousiainen, T.; Um, J.; McFarquhar, G.M.; Haapanala, P.; Makkonen, R.; Hakkarainen, H. Ice-cloud particle habit classification using principal components. J. Geophys. Res. Atmos.; 2012; 117, pp. 1505-1524. [DOI: https://dx.doi.org/10.1029/2012JD017573]

16. Grazioli, J.; Tuia, D.; Monhart, S.; Schneebeli, M.; Raupach, T.; Berne, A. Hydrometeor classification from two-dimensional video disdrometer data. Atmos. Meas. Tech.; 2014; 7, pp. 2869-2882. [DOI: https://dx.doi.org/10.5194/amt-7-2869-2014]

17. Bernauer, F.; Hürkamp, K.; Rühm, W.; Tschiersch, J. Snow event classification with a 2D video disdrometer—A decision tree approach. Atmos. Res.; 2016; 172, pp. 186-195. [DOI: https://dx.doi.org/10.1016/j.atmosres.2016.01.001]

18. Praz, C.; Ding, S.; McFarquhar, G.M.; Berne, A. A versatile method for ice particle habit classification using airborne imaging probe data. J. Geophys. Res. Atmos.; 2018; 123, pp. 13472-13495. [DOI: https://dx.doi.org/10.1029/2018JD029163]

19. Leinonen, J.; Berne, A. Unsupervised classification of snowflake images using a generative adversarial network and K-medoids classification. Atmos. Meas. Tech.; 2020; 13, pp. 2949-2964. [DOI: https://dx.doi.org/10.5194/amt-13-2949-2020]

20. Touloupas, G.; Lauber, A.; Henneberger, J.; Beck, A.; Lucchi, A. A convolutional neural network for classifying cloud particles recorded by imaging probes. Atmos. Meas. Tech.; 2020; 13, pp. 2219-2239. [DOI: https://dx.doi.org/10.5194/amt-13-2219-2020]

21. Thapa, A.; Horanont, T.; Neupane, B.; Aryal, J. Deep Learning for Remote Sensing Image Scene Classification: A Review and Meta-Analysis. Remote Sens.; 2023; 15, 4804. [DOI: https://dx.doi.org/10.3390/rs15194804]

22. Korot, E.; Guan, Z.; Ferraz, D.; Wagner, S.K.; Zhang, G.; Liu, X.; Faes, L.; Pontikos, N.; Finlayson, S.G.; Khalid, H. et al. Code-free deep learning for multi-modality medical image classification. Nat. Mach. Intell.; 2021; 3, pp. 288-298. [DOI: https://dx.doi.org/10.1038/s42256-021-00305-2]

23. Zhang, C.; Yu, Z.; Wang, X.; Chen, Z.-J.; Deng, C.; Xie, S.Q. Exploration of deep learning-driven multimodal information fusion frameworks and their application in lower limb motion recognition. Biomed. Signal Process.; 2024; 96 Pt B, 106551. [DOI: https://dx.doi.org/10.1016/j.bspc.2024.106551]

24. Xiao, H.; Zhang, F.; He, Q.; Liu, P.; Yan, F.; Miao, L.; Yang, Z. Classification of ice crystal habits observed from airborne Cloud Particle Imager by deep transfer learning. Earth Space Sci.; 2019; 6, pp. 1877-1886. [DOI: https://dx.doi.org/10.1029/2019EA000636]

25. Liao, M.Y.; Duan, J.; Zhang, R.; Zhou, X.; Wu, X.; Wang, X.; Hu, J. A hypergraph-embedded convolutional neural network for ice crystal particle habit classification. Intell. Autom. Soft Comput.; 2021; 29, pp. 787-801. [DOI: https://dx.doi.org/10.32604/iasc.2021.018190]

26. Jaffeux, L.; Schwarzenböck, A.; Coutris, P.; Duroure, C. Ice crystal images from optical array probes: Classification with convolutional neural networks. Atmos. Meas. Tech.; 2022; 15, pp. 5141-5157. [DOI: https://dx.doi.org/10.5194/amt-15-5141-2022]

27. Zhang, R.; Xiao, H.; Gao, Y.; Su, H.; Li, D.; Wei, L.; Li, J.; Li, H. Shape Classification of Cloud Particles Recorded by the 2D-S Imaging Probe Using a Convolutional Neural Network. J. Meteorol. Res.-PRC.; 2023; 37, pp. 521-535. [DOI: https://dx.doi.org/10.1007/s13351-023-2146-2]

28. Chen, Y.; Jiao, R.; Xu, Y.; Huang, M. Application of Convolutional Neural Networks to Shape Classification of Cloud Particles. Proceedings of the IEEE 6th International Conference on Computer and Communication Engineering Technology (CCET); Beijing, China, 2 August 2023.

29. Chang, Y.; Guo, X.; Tang, J.; Lu, G. Aircraft measurement campaign on summer cloud microphysical properties over the Tibetan Plateau. Sci. Rep.; 2019; 9, 4912. [DOI: https://dx.doi.org/10.1038/s41598-019-41514-5]

30. Baumgardner, D.; Brenguier, J.L.; Bucholtz, A.; Coe, H.; DeMott, P.; Gayet, J.F.; Garrett, T.J.; Hermann, M.; Heymsfield, A.; Korolev, A. et al. Airborne instruments to measure atmospheric aerosol particles, clouds and radiation: A cook’s tour of mature and emerging technology. Atmos. Res.; 2011; 102, pp. 10-29. [DOI: https://dx.doi.org/10.1016/j.atmosres.2011.06.021]

31. Lawson, R.P.; O’Connor, D.; Zmarzly, P.; Weaver, K.; Baker, B.; Mo, Q.; Jonsson, H. The 2D-S (stereo) probe: Design and preliminary tests of a new airborne, high-speed, high-resolution particle imaging probe. J. Atmos. Ocean. Technol.; 2006; 23, pp. 1462-1477. [DOI: https://dx.doi.org/10.1175/JTECH1927.1]

32. Lawson, R.P.; Stewart, R.E.; Strapp, J.W.; Isaac, G.A. Aircraft observations of the origin and growth of very large snowflakes. Geophys. Res. Lett.; 1993; 20, pp. 53-56. [DOI: https://dx.doi.org/10.1029/92GL02917]

33. Lawson, R.P.; Baker, B.A.; Schmitt, C.G.; Jensen, T.L. An overview of microphysical properties of Arctic clouds observed in May and July 1998 during FIRE ACE. J. Geophys. Res. Atmos.; 2001; 106, pp. 14989-15014. [DOI: https://dx.doi.org/10.1029/2000JD900789]

34. Baum, B.A.; Heymsfield, A.J.; Yang, P.; Bedka, S.T. Bulk scattering properties for the remote sensing of ice clouds. Part I: Microphysical data and models. J. Appl. Meteorol. Clim.; 2005; 44, pp. 1885-1895. [DOI: https://dx.doi.org/10.1175/JAM2308.1]

35. Zha, S.; Zhang, H.; Li, X.; Hua, S.; Chen, B. Numerical Simulation of Precipitation Processes during the Opening Ceremony of the Nanjing 2014 Youth Olympic Games. Chin. J. Atmos. Sci.; 2020; 44, pp. 1258-1274. (In Chinese)

36. Hou, T.J.; Lei, H.C.; He, Y.J.; Yang, J.; Zhao, Z.; Hu, Z. Aircraft measurements of the microphysical properties of stratiform clouds with embedded convection. Adv. Atmos. Sci.; 2021; 38, pp. 966-982. [DOI: https://dx.doi.org/10.1007/s00376-021-0287-8]

37. Jensen, E.J.; Pfister, L.; Jordan, D.E.; Bui, T.V.; Ueyama, R.; Singh, H.B.; Thornberry, T.D.; Rollins, A.W.; Gao, R.S.; Fahey, D.W. et al. The NASA Airborne Tropical Tropopause Experiment: High-Altitude Aircraft Measurements in the Tropical Western Pacific. Bull. Am. Meteorol. Soc.; 2017; 98, pp. 129-143. [DOI: https://dx.doi.org/10.1175/BAMS-D-14-00263.1]

38. Jackson, R.C.; McFarquhar, G.M. An Assessment of the Impact of Antishattering Tips and Artifact Removal Techniques on Bulk Cloud Ice Microphysical and Optical Properties Measured by the 2D Cloud Probe. J. Atmos. Ocean. Technol.; 2014; 31, pp. 2131-2144. [DOI: https://dx.doi.org/10.1175/JTECH-D-14-00018.1]

39. Rollins, A.W.; Thornberry, T.D.; Atlas, E.; Navarro, M.; Schauffler, S.; Moore, F.; Elkins, J.W.; Ray, E.; Rosenlof, K.; Aquila, V. et al. SO₂ Observations and Sources in the Western Pacific Tropical Tropopause Region. J. Geophys. Res. Atmos.; 2018; 123, pp. 13549-13559. [DOI: https://dx.doi.org/10.1029/2018JD029635]

40. Montgomery, M.T.; Davis, D.; Dunkerton, T.; Wang, Z.; Velden, C.; Torn, R.; Majumdar, S.J.; Zhang, F.; Smith, R.K.; Bosart, L. et al. The Pre-Depression Investigation of Cloud-Systems in the Tropics (PREDICT) Experiment: Scientific Basis, New Analysis Tools, and Some First Results. Bull. Am. Meteorol. Soc.; 2012; 93, pp. 153-172. [DOI: https://dx.doi.org/10.1175/BAMS-D-11-00046.1]

41. Toon, O.B.; Maring, H.; Dibb, J.; Ferrare, R.; Jacob, D.J.; Jensen, E.J.; Luo, Z.J.; Mace, G.G.; Pan, L.L.; Pfister, L. et al. Planning, implementation, and scientific goals of the Studies of Emissions and Atmospheric Composition, Clouds and Climate Coupling by Regional Surveys (SEAC4RS) field mission. J. Geophys. Res. Atmos.; 2016; 121, pp. 4967-5009. [DOI: https://dx.doi.org/10.1002/2015JD024297]

42. Houze, R.A.; McMurdie, L.A.; Petersen, W.A.; Schwaller, M.R.; Baccus, W.; Lundquist, J.D.; Mass, C.F.; Nijssen, B.; Rutledge, S.A.; Hudak, D.R. et al. The Olympic Mountains Experiment (OLYMPEX). Bull. Am. Meteorol. Soc.; 2017; 98, pp. 2167-2188. [DOI: https://dx.doi.org/10.1175/BAMS-D-16-0182.1] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/30140097]

43. Vázquez-Martín, S.; Kuhn, T.; Eliasson, S. Shape Dependence of Falling Snow Crystals’ Microphysical Properties Using an Updated Shape Classification. Appl. Sci.; 2020; 10, 1163. [DOI: https://dx.doi.org/10.3390/app10031163]

44. Magono, C.; Lee, C.W. Meteorological classification of natural snow crystals. J. Fac. Sci.; 1966; 2, pp. 321-335.

45. Wu, Z.; Liu, S.; Zhao, D.; Yang, L.; Xu, Z.; Yang, Z.; Zhou, W.; He, H.; Huang, M.; Liu, D. et al. Neural network classification of ice-crystal images observed by an airborne cloud imaging probe. Atmos.-Ocean; 2020; 58, pp. 303-315. [DOI: https://dx.doi.org/10.1080/07055900.2020.1843393]

46. LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature; 2015; 521, pp. 436-444. [DOI: https://dx.doi.org/10.1038/nature14539]

47. Gu, J.; Wang, Z.; Kuen, J.; Ma, L.; Shahroudy, A.; Shuai, B.; Liu, T.; Wang, X.; Wang, G.; Cai, J. et al. Recent advances in convolutional neural networks. Pattern Recognit.; 2018; 77, pp. 354-377. [DOI: https://dx.doi.org/10.1016/j.patcog.2017.10.013]

48. Haq, A.U.; Li, J.P.; Kumar, R.; Ali, Z.; Khan, I.; Uddin, M.I.; Agbley, B.L.Y. MCNN: A multi-level CNN model for the classification of brain tumors in IoT-healthcare system. J. Ambient Intell. Hum.; 2023; 14, pp. 4695-4706. [DOI: https://dx.doi.org/10.1007/s12652-022-04373-z]

49. Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; Fei-Fei, L. ImageNet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition; Miami, FL, USA, 20–25 June 2009.

50. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. Proceedings of the Advances in Neural Information Processing System 25: 26th Annual Conference on Neural Information Processing Systems 2012; Lake Tahoe, NV, USA, 3–6 December 2012.

51. Siraj, F.M.; Ayon, S.T.K.; Samad, M.A.; Uddin, J.; Choi, K. Few-Shot Lightweight SqueezeNet Architecture for Induction Motor Fault Diagnosis Using Limited Thermal Image Dataset. IEEE Access; 2024; 12, pp. 50986-50997. [DOI: https://dx.doi.org/10.1109/ACCESS.2024.3385430]

52. Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015; Boston, MA, USA, 7–12 June 2015.

53. Tan, M.; Le, Q. Efficientnet: Rethinking model scaling for convolutional neural networks. Proceedings of the 36th International Conference on Machine Learning (PMLR); Long Beach, CA, USA, 9–15 June 2019.

54. He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); Las Vegas, NV, USA, 27–30 June 2016.

55. Zhang, X.; Zhou, X.; Lin, M.; Sun, J. Shufflenet: An extremely efficient convolutional neural network for mobile devices. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); Salt Lake City, UT, USA, 18–23 June 2018.

56. Shah, S.R.; Qadri, S.; Bibi, H.; Shah, S.M.W.; Sharif, M.I.; Marinello, F. Comparing Inception V3, VGG 16, VGG 19, CNN, and ResNet 50: A Case Study on Early Detection of a Rice Disease. Agronomy; 2023; 13, 1633. [DOI: https://dx.doi.org/10.3390/agronomy13061633]

57. de Boer, P.T.; Kroese, D.P.; Mannor, S.; Rubinstein, Y. A Tutorial on the Cross-Entropy Method. Ann. Oper. Res.; 2025; 134, pp. 19-67. [DOI: https://dx.doi.org/10.1007/s10479-005-5724-z]

58. Przybylo, V.M.; Sulia, K.J.; Schmitt, C.G.; Lebo, Z.J. Classification of Cloud Particle Imagery from Aircraft Platforms Using Convolutional Neural Networks. J. Atmos. Ocean. Technol.; 2022; 39, pp. 405-424. [DOI: https://dx.doi.org/10.1175/JTECH-D-21-0094.1]