Evaluation of Artificial Precipitation Enhancement Using UNET-GRU Algorithm for Rainfall Estimation

Abstract

The evaluation of the effects of artificial precipitation enhancement remains one of the most important and challenging issues in the fields of meteorology. Rainfall is the most important evaluation metric for artificial precipitation enhancement, which is mainly achieved through physics-based models that simulate physical phenomena and data-driven statistical models. The series of effect evaluation methods requires the selection of a comparison area for effect comparison, and idealized assumptions and simplifications have been made for the actual cloud precipitation process, leading to unreliable quantitative evaluation results of artificial precipitation effects. This paper proposes a deep learning-based method (UNET-GRU) to quantitatively evaluate the effect of artificial rainfall. By comparing the residual values obtained from inverting the natural evolution grid rainfall of the same area under the same artificial rainfall conditions with the actual rainfall amount after artificial rainfall operations, the effect of artificial rainfall can be quantitatively evaluated, effectively solving the problem of quantitative evaluation of artificial precipitation effects. Wuhan and Shiyan in China are selected to represent typical plains and mountainous areas, respectively, and the method is evaluated using 6-min resolution radar weather data from 2017 to 2020. During the experiment, we utilized the UNET-GRU algorithm and developed separate algorithms for comparison against common persistent baselines (i.e., the next-time data of the training data). The prediction of mean squared error (MSE) for these three algorithms was significantly lower than that of the baseline data. Moreover, the indicators for these algorithms were excellent, further demonstrating their efficacy. In addition, the residual results of the estimated 7-h grid rainfall were compared with the actual recorded rainfall to evaluate the effectiveness of artificial precipitation. The results showed that the estimated rainfall was consistent with the recorded precipitation for that year, indicating that deep learning methods can be successfully used to evaluate the impact of artificial precipitation. The results demonstrate that this method improves the accuracy of effect evaluation and enhances the generalization ability of the evaluation scheme.

1. Introduction

Currently, artificial precipitation enhancement is an important approach for alleviating drought conditions and increasing water storage in many countries and regions. However, the evaluation method for the catalytic effect of cloud seeding has been questioned on occasion [1,2,3,4,5]. The evaluation method for the effect of artificial precipitation enhancement mainly focuses on the following four aspects:

• (1). Comparing the target cloud operation with contrast cloud radar echoes, satellite inversion products, etc., and conducting physical inspections [6,7,8,9,10,11];

• (2). Statistical evaluation of rainfall in the affected area and comparison area [12,13,14,15,16,17];

• (3). Adopting a comprehensive inspection method that combines numerical model simulation and live observation [18,19,20,21];

• (4). Random evaluation of rainfall increase rates [22,23,24,25,26].

In (1) and (2), due to the significant spatiotemporal variability of precipitation in China, it is challenging to identify an appropriate reference area for comparison. For (3), live observation data are usually collected at discrete stations with 1-h resolution, while the artificial rainfall process usually lasts for only 4–7 h or even shorter. Thus, it is difficult to distinguish the increase in precipitation due to artificial means from natural variation using limited data. As for (4), the randomization method requires a sufficiently long experiment period to test the effect of single or a few artificial precipitation enhancement operations. However, this method wastes half of the operation time and may not completely eliminate human influence, making non-randomization methods more commonly used in practice.

Deep learning, a promising branch of machine learning, has received extensive attention in recent years and has been widely used in pattern recognition, image processing, fault detection, classification, and prediction tasks [27,28,29,30,31,32,33,34,35]. Compared to shallow models, deep learning has three major characteristics: unsupervised feature learning, strong generalization ability, and big data training. Deep learning-based predictive models are generally accepted to exhibit attractive performance in terms of accuracy, stability, and effectiveness. As rainfall is a direct indicator to evaluate the effect of artificial precipitation, it is important to compare the accuracy of natural precipitation variation in the last 4–7 h under climatic conditions within half an hour before the implementation of artificial precipitation. Therefore, short-term prediction algorithms based on deep learning can be used to evaluate the effect of artificial precipitation enhancement. A local prediction system based on deep learning was proposed by [36] to predict short-term rainfall at 16 points in Japan individually, using various forms of massive data provided by the Japan Meteorological Agency (JMA). However, as there are only a few artificial precipitation enhancement operations in each region per year, data is limited. Work in [37] proposed a SmaAT-UNet architecture for short-term forecasts, which is applicable to cases with small amounts of data and calculation. However, the predicted precipitation maps of all models are quite blurry due to the use of MSE as the guiding loss function, which is biased towards blurry images.

In summary, while deep learning-based prediction algorithms have been widely applied in the meteorological field, they require a large and diverse amount of training data. For the evaluation of artificial precipitation enhancement, deep learning algorithms have faced challenges due to the scarcity and uneven distribution of training samples, which limits their predictive accuracy and impacts the credibility of the evaluation results. As a result, traditional methods based on physical models or statistical principles remain the mainstream approach. However, these methods often require the selection of a comparison area (as shown in Figure 1) and can only provide qualitative evaluations, which may not lead to convincing conclusions. Therefore, we propose a deep learning algorithm based on small sample training to invert the 1–7 h rainfall natural catalysis situation in the same area that meets the meteorological conditions of artificial precipitation enhancement. We evaluate the effectiveness of artificial precipitation enhancement by comparing the residual between the natural catalysis rainfall and the actual rainfall enhancement. The grid-based residual data can be verified through the collected data from rainfall monitoring stations, thus proving the feasibility of our evaluation plan.

This paper is organized as follows. In Section 2, we provide a brief overview of related research on evaluating the effect of artificial precipitation enhancement. Section 3 describes the proposed UNET-GRU architecture and other models used for effect evaluation. In Section 4, we present the experiments conducted and the results obtained. A discussion of the results is provided in Section 5. Finally, we conclude with some remarks in Section 6.

2. Related Work

2.1. Classical Evaluation Methods Model in Meteorological Field

The traditional approach for evaluating the effect of artificial precipitation enhancement relies on statistical methods (Figure 1). This involves determining the impact area and comparison area of the operation, and estimating the difference between natural precipitation and measured precipitation based on these two regions to determine the effect of the operation. However, in many cases, the impact area and comparison area of artificial precipitation enhancement are not fixed. In such cases, the floating comparison area method is usually employed to evaluate the effect, which involves dividing the comparison area based on principles such as similar weather systems, similar terrain, and good precipitation correlation. Therefore, a reasonable division of the comparison area is a prerequisite for the statistical evaluation of the effect of artificial precipitation enhancement.

2.2. Application of Deep Learning Model in Meteorological Field

Deep learning methods, including convolutional neural networks (CNNs) and recurrent neural networks (RNNs), have been widely applied in weather forecasting. For instance, Kevin Trebing et al. introduced the SMART-UNET algorithm with CBAM attention block for short-term precipitation forecasting [37]. Xingjian Shi et al. utilized radar CAPPI data to achieve regional precipitation forecasting with convolutional LSTM [38]. Jing J et al. introduced the MLC-LSTM algorithm to extrapolate echo sequences by leveraging the space time correlation between multi-level weather radar echoes [39]. The deep learning methods can handle the nonlinear problem of precipitation and radar echo that cannot be solved by statistical methods and numerical simulations. Furthermore, they can better simulate the natural evolution of natural precipitation and radar echo. However, these methods have rarely been used in evaluating the effect of artificial precipitation enhancement. The evaluation of this effect requires predicting the rainfall under natural conditions, which can also be achieved through deep learning methods. Therefore, introducing deep learning techniques to improve the existing evaluation methods of artificial precipitation enhancement can have significant theoretical and practical value.

2.3. UNET

In the field of computer vision, the FCN is a well-known image segmentation network. In the field of medical image processing, UNET is a popular network for semantic segmentation tasks, often used as a baseline [40]. The UNET architecture consists of two parts: feature extraction and upsampling. This structure is commonly referred to as an encoder decoder structure. The network resembles a letter U, hence its name. As shown in Figure 2, the input image is first convolved and pooled. In the original UNET paper, the image is pooled four times, resulting in features of sizes 144 × 144, 72 × 72, 36 × 36, and 18 × 18. The 18 × 18 feature map is then upsampled to obtain a 36 × 36 feature map, which is concatenated with the previous 36 × 36 feature map to preserve channel information. The concatenated feature map is then convolved and upsampled to obtain a 72 × 72 feature map, which is again concatenated with the previous 72 × 72 feature map. After four rounds of upsampling, a 288 × 288 prediction result with the same size as the input image can be obtained.

UNET has several advantages. Firstly, as the network layer becomes deeper, the feature map’s field of view increases. While shallow convolution focuses on texture features, deep networks focus on essential features, making both deep and shallow features grid meaningful. Secondly, larger-sized feature map edges obtained through deconvolution are often lack of information. This is because, during downsampling, some edge features are lost, and these cannot be retrieved during upsampling. Therefore, one edge feature can be retrieved through feature splicing.

UNET has found wide applications in medical image segmentation due to four characteristics of medical images. Firstly, medical images have simple semantics and fixed positions, making their semantic information relatively simple, and there is no need to filter out useless information. All features of medical images are important, including both low-level features and high-level semantic features, making the skip connection structure (feature splicing) of the U-shaped structure better used. Secondly, medical images are difficult to obtain, and the dataset available is usually very small. The amount of data may only be a few hundred, or even fewer than a hundred. Therefore, it is easy to over-fit large networks like DeepLabv3+. While large networks have stronger image representation ability, simpler and fewer medical images do not have much content to express. As a result, some people find that, in the decimal scale, the segmented SOTA model has no advantage over the lightweight UNET. Thirdly, medical images are often multimodal, such as in the ISLES, data on modalities such as CBF, MTT, and CBV can usually be obtained. In medical imaging tasks, we often need to design our network to extract different modal features, lightweight and simple UNET more operation space.

In the field of artificial precipitation enhancement effect inspection, the UNET algorithm has been found to be applicable for short-term rainfall prediction in the same region, because it has the ability to handle the above three characteristics. Firstly, the area of interest for precipitation enhancement (artificial rainfall area) is usually specific and localized (characteristic 1). Secondly, the number of opportunities for artificial precipitation enhancement in a specific area is limited each year (characteristic 2), with generally only tens of operations per year in the same area, each operation lasting for 4–7 h. Finally, the available data for precipitation enhancement evaluation include various modal data formats such as radar data (MCR, MTOP, CAPPI, etc.), satellite data, and real-time data (characteristic 3). Therefore, the UNET algorithm can be used to evaluate the effect of artificial precipitation enhancement.

3. Methods

3.1. Effect Evaluation Method Based on Deep Learning

The classical method involves determining a contrast area and an affected area, and comparing the rainfall in these two areas. However, the comparison and affected areas cannot be entirely consistent, which may result in less convincing evaluation results. To address this, we propose setting the affected area as the comparison area to ensure that both areas have the same meteorological conditions (e.g., wind speed, air temperature, and radar data) prior to the implementation of artificial precipitation enhancement. By inverting the deduction process under natural conditions (4–7 h), we can use it as a comparison area to compare with the affected area. Thus, we transform the evaluation method into Figure 3.

This paper proposes an evaluation method for artificial precipitation enhancement effects based on deep learning. The method focuses on designing a reasonable short-term rainfall forecasting algorithm and building a training model for natural evolution and artificial precipitation enhancement conditions separately using historical data. After implementing artificial precipitation enhancement in the region, the current meteorological data can be inputted into both models to obtain rainfall predictions under natural evolution and artificial precipitation enhancement conditions, allowing for the evaluation of artificial precipitation enhancement effects. To verify the model’s applicability, the predicted rainfall from the artificial precipitation enhancement model can be compared with actual rainfall. It should be noted that the general rainfall forecast model is not applicable in this method due to the small dataset under artificial precipitation enhancement conditions (dozens of datasets per year). Deep learning algorithms usually require tens of thousands of datasets for training, which can lead to poor model generalization ability or even overfitting. Therefore, it is necessary to propose a short-term rainfall forecasting algorithm that supports small dataset training.

3.2. UNET-GRU Algorithm

Therefore, we propose a convolution GRU algorithm that integrates UNET, as shown in Figure 4, for short-term rainfall forecasting. The algorithm utilizes the four modes of weather radar data with 6-min resolution (MTOP, MCR, MVIL, and CAPPI) to generate an image, which is then processed by the UNET network to extract modal characteristics. As different modes contain dense links, we introduce convolution GRU to better utilize the relationship between continuous radar data and explain the nonlinearity in multimodal data modeling. The output of the model is the grid rainfall at a specific time in a certain area. Due to the large data volume of CAPPI data with its 24 layers, the training sequence is not included in this experiment, as our laboratory computer is not capable of handling such large datasets. Nonetheless, the proposed algorithm still yields satisfactory results.

GRU (as shown in Figure 5) is a highly effective variation of the LSTM network, featuring a simpler structure with comparable performance. As a result, it has become a popular choice in many applications. In common with LSTM, GRU is capable of addressing the long dependency problem in RNNs. The GRU model has only two gates: a reset gate and an update gate. The reset gate functions similarly to LSTM’s forgetting gate, but it does not forget the information of the memory unit $C_{t-1}$ from the previous time step. Instead, it forgets the information of the hidden layer unit $h_{t-1}$ from the previous time step:

(1)$r_t=\sigma (W_r·[h_{t-1},x_t]+b_r)$

The function of update gate is different from that of LSTM. It controls the balance between the hidden layer state $h_{t-1}$ at the previous moment and the current input information.

(2)$Z_t=\sigma (W_z·[h_{t-1},x_t]+b_z)$

The input information is $r_t$ · $h_{t-1}$ after forgetting,

(3)${\tilde{h}}_t=\tanh (W_h·[r_t·h_{t-1},x_t]+b_h)$

$h_t$ after balance:

(4)$h_t=(1-Z_t)·h_{t-1}+z_t·{\tilde{h}}_t$

[] represents concat, · represents element-level multiplication.

3.3. Other Models

As a comparison, we trained both the UNET and CoGRU (GRU algorithm supporting convolution operation) architectures, resulting in a total of three models compared in this study. Table 1 presents a comparison of the models’ parameters. When examining the standard UNET architecture and our proposed UNET-GRU architecture, it is evident that the latter has slightly more parameters—approximately 21.6 million compared to approximately 17.3 million.

3.4. Training

All three models mentioned above were trained for a maximum of 200 epochs. An early stopping criterion was utilized, which stopped the training process if the validation loss did not improve in the last 15 epochs. This criterion was satisfied in all training iterations, so the maximum of 200 epochs was never reached. Additionally, a learning rate scheduler was employed, which decreased the learning rate to one-tenth of the previous rate when the validation loss did not improve for four epochs. The initial learning rate was set to 0.001, and the Adam [36] optimizer with default values was used. The training was conducted on a single NVIDIA GeForce RTX 3090 Super with 24 GB of VRAM.

3.5. Model Evaluation

The loss function used in this study is the mean squared error (MSE) between the output images and the ground truth images. The MSE is calculated as follows:

(5)$MSE=\frac{{{\displaystyle {\sum }_{i=1}^n\left(y_i-{\widehat{y}}_i\right)}}^2}{n}$

where n represents the number of samples, $y_i$ is the value of the ground truth, and ${\widehat{y}}_i$ is the value of the prediction. Apart from mean squared error (MSE), various performance evaluation metrics are calculated, including precision, recall (probability of detection), accuracy, F1-score, critical success index (CSI), false alarm rate (FAR), and Heidke skill score (HSS). For the precipitation map dataset, these metrics are computed for rainfall greater than a threshold of 0.5 mm/h. To achieve this, a Boolean mask is generated for each pixel of the predicted output and target images using the specified threshold. Based on this, the number of true positives (TP) (prediction = 1, target = 1), false positives (FP) (prediction = 1, target = 0), true negatives (TN) (prediction = 0, target = 0), and false negatives (FN) (prediction = 0, target = 1) can be calculated. Finally, the CSI, FAR, and HSS metrics can be computed as follows:

(6)$CSI=\frac{TP}{TP+FN+FP}$

(7)$FAR=\frac{FP}{TP+FP}$

and

(8)$HSS=\frac{TP\times TN-FN\times FP}{(TP+FN\left)\right(FN+TN)+(TP+FP\left)\right(FP+TN)}$

The selection of the threshold of 0.5 mm/h is consistent with the works by Shi et al. [41,42] and Xingjian et al. [43], and it can differentiate whether there is rain or not.

4. Experiments

Precipitation Map Dataset

We obtained three-dimensional radar from Swan radar from the Hubei Meteorological Service Center, which includes MCR, MTOP, MVIL, QPE30, QPE60, and MOSAIC maps at 6-min intervals from the last four years (2017–2020) in Hubei province. The dataset consists of approximately 1.8 million maps generated by 14 PD weather radars, with each raw map having dimension of 1200 × 800. During the period, 177 artificial rain-enhancement operations were conducted in Shiyan and 123 in Wuhan. The impact of artificial rainfall generally lasts for about 7 h, as shown in Figure 6. Therefore, to train the rainfall model under natural conditions, we excluded the data influenced by artificial precipitation. We extracted a map with a dimension of 288 × 288, with the longitude and latitude of the evaluation site for artificial precipitation enhancement as the center, as the training data.

To improve the model training, only data that meet the criteria for rainfall enhancement, as shown in Table 2, can be used as training data. One issue to address is determining the appropriate number of grid points for evaluating the rainfall enhancement criteria. Through the analysis of data from 2017 to 2020 (as shown in Figure 7), we found that selecting a 10 × 10 grid as the reference for rainfall enhancement criteria provides the most suitable data. As the grid size increases, the number of eligible training data decreases. Therefore, the optimal grid size is 10 × 10, which is consistent with the working area of most artificial precipitation enhancement operations.

In the data preparation stage, we normalized the values in the training and testing sets by dividing them by the maximum value in the training set. Additionally, we cropped the precipitation maps to use only a subset of the original image (see Figure 8). This was because many pixels in the original image contained no data values, which was due to the maximum range of the radar being smaller than the image size (as shown in the black edges on the left panel of Figure 8). The rectangular area within the radar range is 1200 pixels × 800 pixels, equivalent to 1200 km by 800 km. For ease of neural network training, we performed a central crop of size 288 pixels × 288 pixels (as shown in the right panel of Figure 8). The crop size was determined based on the movement speed of clouds. On average, the speed of cloud movement is 36 km per hour, with a maximum of 50–60 km per hour. The speed and direction of clouds are influenced by factors such as wind speed, cloud height, and cloud density. In our algorithm, we selected 12 maps that met the requirements for cloud seeding as training inputs, with a time interval of 6 min between each map. If we do not consider the direction of movement, the maximum radius of cloud movement in each map can be calculated as 72 km. Therefore, we chose a size of 144 pixels × 144 pixels for each map, which is sufficient for training. To ensure the generalizability of the training, we doubled the length and width of each map.

The output of the training is the rainfall of each grid point in each region. Actual grid rainfall data is actually difficult to obtain. We usually only obtain 1-h resolution rainfall data by radar retrieved quantitative precipitation estimation (QPE) and discrete monitoring station rainfall data for each region. Therefore, these two 1-h resolution data must be converted into 6-min resolution grid data for algorithm training.

At present, radar measurement of precipitation is mainly based on Z-I relationship [44,45,46]. That is $Z=A\times I^b$, where Z is the radar reflection factor (unit: mm³/m⁶). I is rainfall intensity (unit: mm/h). A and b are coefficients. The accuracy of quantitative precipitation estimation depends to a large extent on the determination of A and b parameters in the Z-I relationship. Because the precipitation properties are different in different seasons and locations, the Z-I relationship is also different. At present, many stations still only use the fixed Z-I relationship provided by the manufacturer to estimate ground precipitation. With the construction of a large number of encrypted automatic weather stations, the spatial and temporal density of precipitation observation has greatly increased. It has become a reality to make full use of the encrypted ground precipitation observation data and the intensity of radar echo to carry out high-precision Z-I relationship analysis. Many domestic scholars have also carried out relevant research. This paper proposes a specific technical scheme based on this problem. By integrating the radar network mosaic data of daily business applications and the precipitation observation data of the ground encrypted automatic weather station, based on the optimization method, the local dynamic Z-I relationship is established, and the quantitative precipitation inversion data with 6-min resolution is obtained in real time. The optimization algorithm is divided into the following three steps:

• (1). Based on the rainfall Z-I relationship, convert the 6-min radar real reflectivity factor Z in the past hour into the radar-estimated rainfall I, and then accumulate the 6-min radar-estimated rainfall I to obtain the hourly radar-estimated rainfall, so as to compare it with the precipitation observed by automatic ground stations.

• (2). In order to obtain the optimal parameters A and b for radar retrieval of precipitation, the hourly radar-estimated precipitation is R and the ground automatic station observed precipitation is G, and the error target discriminant function CTF is selected:

  (9)$CTF=\min \left\{{\displaystyle \sum _{i=1}^n\left[{\left(R_i-G_i\right)}^2+\left|R_i-G_i\right|\right]}\right\}$

  In Equation (9), R is the hourly radar-estimated precipitation; G is the precipitation observed by the automatic ground station; n is the total logarithm of radar automatic station data matching involved in rainfall Z-I relationship fitting.

  In practical business applications, in order to save calculation time and ensure that parameters A and b change within a reasonable range, the adjustment ranges of A and b are limited to [150.00, 400.00] and [0.80, 2.40] respectively, and the adjustment intervals are 0.10 and 0.01 respectively. For each group of A and b, a CTF can be obtained. By constantly adjusting the combination of A and b, it is determined that the Z-I relationship of precipitation determined by Equation (9) A and b whose error objective discriminant function CTF reaches the minimum is optimal.

• (3). Convert the precipitation Z-I relationship obtained in step (2) of the 6-min radar reflectivity factor prediction field within the current 1 hour into precipitation, and then accumulate it into hourly radar quantitative precipitation retrieval data to meet the needs of precipitation inspection.

In addition, in order to quantitatively analyze the precipitation inversion error of the dynamic Z-I relationship method, the mean error ($E_{ME}$), mean relative error ($E_{MRE}$) and other test parameters are calculated.

The error calculation equation is:

(10)$E_{ME}=\frac{1}{n}{\displaystyle \sum _{i=1}^n\left(R_i-G_i\right)}$

(11)$E_{MRE}=\frac{1}{n}{\displaystyle \sum _{i=1}^n\frac{\left|R_i-G_i\right|}{G_i}}$

Equations (10) and (11), $R_{\mathrm{i}}$ and $G_{\mathrm{i}}$ are the precipitation inversion value and real value of the automatic ground weather station respectively; n is the total number of sites.

5. Results and Discussion

After training the three discussed models, we selected the model with the lowest validation loss for each model. These best-performing models were then used to calculate several metrics introduced in Section 3 on the test set. The models were trained, evaluated, and tested on the dataset. In precipitation nowcasting, a common baseline is the persistence method. The persistence model predicts the last input image of a sequence as the prediction image, based on the assumption that the weather will not significantly change from time point t to t + 1. Especially in nowcasting, this baseline is not easy to outperform because the time differences between images are so short (e.g., 2 or 6 min) that weather conditions often remain the same [47].

5.1. Evaluation on Precipitation Map Dataset

The results obtained on the dataset are presented in Table 3. It is important to note that the MSE values were calculated after denormalizing the model predictions to the original rainfall (mm/6 min). The results demonstrate that every model we tested outperformed the common persistence baseline by a significant margin on the Precipitation map dataset. This is remarkable since, as mentioned earlier, it can be challenging to surpass this baseline in nowcasting due to the short time differences between the input and target.

Based on the figure, it is evident that the UNET-GRU model outperforms other models in terms of capturing the development of heavy rain clusters and accurately describing the vertical distribution of rain clusters. However, the hybrid model, which combines the strengths of multiple models, is even more superior in performance compared to any single model.

5.2. Evaluate the Effect of Rainfall Enhancement

We conducted a 7-h rainfall inversion based on two artificial rainfall cases implemented in Shiyan and Wuhan, both in Hubei Province. Information regarding the rain enhancement operation is presented in Table 4. The inversion results were compared with the actual rainfall, as shown in Figure 9 and Figure 10. The results indicated that the inversion accuracy in Shiyan was higher compared to that in Wuhan. This may be attributed to the fact that Shiyan is situated in a mountainous region and is less influenced by human activities. The radar data are relatively stable, leading to more accurate predictions.

Data from the rainfall monitoring station and the rain enhancement operation record file (Table 4) show that the meteorological conditions in Wuhan before artificial rain operation were light to moderate rain (0.20 mm/h to 0.7 mm/h), consistent with the rainfall predicted through deep learning in Figure 9a. Based on the natural catalytic inversion lasting for 7 h, starting from 5:58:10 a.m. on July 30, 2017, it can be inferred that if the artificial rain operation had not been carried out, the rainfall during the 7-h period would have been only 3.56 mm, whereas the actual rainfall was 18.91 mm, with a residual value of 15.35 mm (Table 5). This indicates a significant rain enhancement effect, which is consistent with the results recorded in historical files for this artificial rain operation (Table 4).

Similarly, the artificial rain operation carried out in Shiyan, China on 26 April 2018 at 12:32 a.m. also shows good results. Before the operation, the weather was cloudy and the rainfall changed only slightly. However, the rainfall began to increase in the fourth hour and reached its peak in the sixth hour. The artificial rain operation lasted for 7 h and produced a rainfall of 11.03 mm, while the natural evolution only produced 1.05 mm of rainfall. This fully demonstrates the success of this artificial rain operation.

From Figure 9a and Figure 10a, it can be observed that in the rainfall estimation algorithm, the rainfall estimated by the UNET algorithm is generally larger than that estimated by the UNET-GRU and GRU algorithms. The consistency of rainfall estimation by the UNET-GRU and GRU algorithms is relatively good, but in Figure 9a, the GRU algorithm shows prediction distortion in the first hour, indicating that the robustness of the GRU algorithm is inferior to that of the UNET-GRU algorithm. From Figure 9b and Figure 10b, it can also be observed that the UNET-GRU algorithm is superior to the other two algorithms. The rainfall maps predicted by the GRU and UNET algorithms are blurred, while the rainfall map predicted by the UNET-GRU algorithm has relatively high clarity.

6. Conclusions

In this study, we propose a novel evaluation method to assess the effectiveness of artificial precipitation enhancement. Our approach involves training a weather evolution and development model using artificial neural networks, which simulates the natural catalysis process under increasing rainfall conditions. This eliminates the need for a traditional evaluation method that requires selecting a contrast test area. Our experimental results demonstrate that the effect of artificial precipitation enhancement can be quantitatively evaluated by a grid, rather than relying solely on qualitative assessment, thereby making the evaluation method more reliable and scientifically sound. To demonstrate the universality of our approach, we tested our evaluation method using two sets of artificial rainfall data for 7-h rainfall inversion.

In some cases, the SWAN radar data used for algorithm training may be contaminated due to terrain effects, especially in densely populated urban areas, which can lead to an increase in prediction errors. Therefore, it is necessary to clean the radar data before algorithm training. Additionally, the model currently has approximately 21.6 million parameters, and training once on one piece of RTX3090 graphics card takes about 10 hours, which is unacceptable. Therefore, optimizing the model is an aspect worth researching.

Author Contributions

Conceptualization, R.L.; Data curation, D.L. and H.Z.; Formal analysis, R.L. and D.L.; Funding acquisition, L.Z.; Investigation, L.Z. and P.X.; Methodology, R.L., D.L. and P.X.; Project administration, R.L.; Software, R.L. and P.X.; Supervision, L.Z. and H.Z.; Writing—original draft, R.L.; Writing—review and editing, R.L. All authors have read and agreed to the published version of the manuscript.

Data Availability Statement

The data used to support the findings of this study are available from the corresponding author upon request. Due to the fact that meteorological radar data is classified as national confidential data, the data has not been made public.

Conflicts of Interest

The authors declare that they have no conflict of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Generate image description

Figure 1. The classical evaluation method of artificial precipitation enhancement effect.

Generate image description

Figure 2. UNET structure.

Generate image description

Figure 3. Evaluation scheme of artificial precipitation enhancement based on deep learning.

Generate image description

Figure 4. UNET-GRU algorithm block diagram.

Generate image description

Figure 5. GRU algorithm block diagram.

Generate image description

Figure 6. Duration of an artificial precipitation process.

Generate image description

Figure 7. The relationship between grid size selection and the amount of data eligible for rainfall enhancement.

Generate image description

Figure 8. Weather radar data map.

Generate image description

Figure 9. Comparison of 7-h grid and average rainfall inversion for a simulated artificial rainfall event in Wuhan region. (a) Comparison chart of 1-h rainfall forecast and actual measurement values for each model (lasts for 7 h). (b) Comparison of 1-h grid-based rainfall forecasts and observed values from different models (lasts for 7 h).

Generate image description

Figure 10. Comparison of 7-h grid and average rainfall inversion for a simulated artificial rainfall event in the Shiyan region. (a) Comparison chart of 1-h rainfall forecast and actual measurement values for each model (lasts for 7 h). (b) Comparison of 1-h grid-based rainfall forecasts and observed values from different models (lasts for 7 h).

Table 1

Number of parameters of the compared models.

Model	Parameters
UNET	17,272,577
CoGRU 2	210,701
UNET-GRU	21,555,009

Table 2

Rainfall enhancement criteria for stratiform and cumulus mixed clouds.

Season	Altitude(Unit: m)	MCR (Unit: dbz)	MTOP(Unit: km)	MVIL(Unit: kg/m3)
Spring (March to May)	<1000	>20	>4	>5
Summer (June to August)	<1000	>25	>5	>10
Autumn (September to November)	<1000	>20	>5	>5
Winter (December to February)	<1000	>15	>4	>5

Table 3

MSE and scores on rainfall bigger than 0.5 mm/h indicating rain or no rain. Best result for that score is in bold. A ↑ indicates that higher values for that score are good whereas a ↓ indicates that lower scores are better.

Model	MSE↓	Accuracy↑	Precision↑	Recall↑	F1↑	CSI↑	FAR↓	HSS↑
Persistence (baseline)	1.1697	0.7264	0.7315	0.8313	0.729	0.5735	0.2736	0.4039
UNet	0.1239	0.6615	0.8530	0.7913	0.5078	0.3403	0.3385	0.3951
CoGRU	0.1542	0.6294	0.6643	0.8042	0.5216	0.3529	0.3706	0.4238
UNet-GRU	0.1182	0.6311	0.874	0.8462	0.5192	0.3506	0.3689	0.4139

Table 4

Description of information related to artificial precipitation enhancement.

No.	Date	Rockets(pcs)	Start Time	End Time	Conditions before op.	Conditions after op.	Area (km2)	Effect	Region
1	30 July 2017	6	05:58:10	06:52:40	Light to moderate rain	Moderate to heavy rain	400	good	Wuhan
2	26 April 2018	4	00:06:32	00:48:22	overcast	light rain	360	good	Shiyan

Table 5

Effect estimates of artificial precipitation compared with historical records.

No.	Date	Start Time	Duration	Naturally Evolved Rainfall	Actual Rainfall	Residual Rainfall	Effect	Region
1	30 July 2017	05:58:10	7 h	3.56 mm	18.91 mm	15.35 mm	good	Wuhan
2	26 April 2018	00:06:32	7 h	1.05 mm	11.03 mm	9.98 mm	good	Shiyan

References

1. Silverman, B.A. A critical assessment of glaciogenic seeding of convective clouds for rainfall enhancement. Bull. Am. Meteorol. Soc.; 2001; 82, pp. 903-924. [DOI: https://dx.doi.org/10.1175/1520-0477(2001)082<0903:ACAOGS>2.3.CO;2]

2. National Research Council. Critical Issues in Weather Modification Research; National Academies Press: Washington, DC, USA, 2004; [DOI: https://dx.doi.org/10.17226/10829]

3. World Meteorological Organization. WMO Statement on the Status of the Global Climate in 2014. Environ. Policy Collect.; 2015; 40, pp. 144-151. Available online: https://library.wmo.int/doc_num.php?explnum_id=7907 (accessed on 23 March 2015).

4. Flossmann, A.I.; Manton, M.; Abshaev, A.; Bruintjes, R.; Murakami, M.; Prabhakaran, T.; Yao, Z. Review of advances in precipitation enhancement research. Bull. Am. Meteorol. Soc.; 2019; 100, pp. 1465-1480. [DOI: https://dx.doi.org/10.1175/BAMS-D-18-0160.1]

5. Gabriel, K.R. Ratio statistics for randomized experiments in precipitation stimulation. J. Appl. Meteorol. Climatol.; 1999; 38, pp. 290-301. [DOI: https://dx.doi.org/10.1175/1520-0450(1999)038<0290:RSFREI>2.0.CO;2]

6. Woodley, W.L.; Rosenfeld, D.; Silverman, B.A. Results of on-top glaciogenic cloud seeding in Thailand. Part I: The demonstration experiment. J. Appl. Meteorol.; 2003; 42, pp. 920-938. [DOI: https://dx.doi.org/10.1175/1520-0450(2003)042<0920:ROOGCS>2.0.CO;2]

7. Woodley, W.L.; Rosenfeld, D.; Silverman, B.A. Results of on-top glaciogenic cloud seeding in Thailand. Part II: Exploratory analyses. J. Appl. Meteorol. Climatol.; 2003; 42, pp. 939-951. [DOI: https://dx.doi.org/10.1175/1520-0450(2003)042<0939:ROOGCS>2.0.CO;2]

8. Griffith, D.A.; Yorty, D.P. A brief history of evaluations performed on the operational Kings River winter orographic cloud seeding program. J. Weather Modif.; 2014; 46, pp. 29-36. [DOI: https://dx.doi.org/10.54782/jwm.v46i1.200]

9. Wu, X.H.; Niu, S.J.; Jin, D.Z.; Sun, H. Influence of natural rainfall variability on the evaluation of artificial precipitation enhancement. Sci. China Earth Sci.; 2015; 58, pp. 906-914. [DOI: https://dx.doi.org/10.1007/s11430-015-5055-0]

10. Hsu, P.C.; Lee, J.Y.; Ha, K.J.; Tsou, C.-H. Influences of boreal summer intraseasonal oscillation on heat waves in monsoon Asia. J. Clim.; 2017; 30, pp. 7191-7211. [DOI: https://dx.doi.org/10.1175/JCLI-D-16-0505.1]

11. Wu, X.; Yan, N.; Yu, H.; Niu, S.; Meng, F.; Liu, W.; Sun, H. Advances in the evaluation of cloud seeding: Statistical evidence for the enhancement of precipitation. Earth Space Sci.; 2018; 5, pp. 425-439. [DOI: https://dx.doi.org/10.1029/2018EA000424]

12. Srinivas, V.V. Regionalization of precipitation in India—A review. J. Indian Inst. Sci.; 2013; 93, pp. 153-162.

13. Liu, L.; Xu, Z.X. Regionalization of precipitation and the spatiotemporal distribution of extreme precipitation in southwestern China. Nat. Hazards; 2016; 80, pp. 1195-1211. [DOI: https://dx.doi.org/10.1007/s11069-015-2018-x]

14. Fazel, N.; Berndtsson, R.; Uvo, C.B.; Madani, K.; Kløve, B. Regionalization of precipitation characteristics in Iran’s Lake Urmia basin. Theor. Appl. Climatol.; 2018; 132, pp. 363-373. [DOI: https://dx.doi.org/10.1007/s00704-017-2090-0]

15. Mannan, A.; Chaudhary, S.; Dhanya, C.T.; Swamy, A.K. Regionalization of rainfall characteristics in India incorporating climatic variables and using self-organizing maps. ISH J. Hydraul. Eng.; 2018; 24, pp. 147-156. [DOI: https://dx.doi.org/10.1080/09715010.2017.1400409]

16. Xueliang, G.; Chungang, F.; Guangxian, L.; Xiaofeng, L.; Zhengjun, S.; Ziping, Y.; Peiren, L.; Zehou, Y. Progresses of weather modification technologies and applications in China from 2008 to 2018. J. Appl. Meteorol.; 2019; 30, pp. 641-650. [DOI: https://dx.doi.org/10.11898/1001-7313.20190601]

17. Mazzetti, T.; Geerts, B.; Xue, L. A Numerical Evaluation of the Impact of Operational Ground-Based Glaciogenic Cloud Seeding on Precipitation over the Wind River Range, Wyoming. J. Appl. Meteorol. Climatol.; 2023; 62, pp. 489-510. [DOI: https://dx.doi.org/10.1175/JAMC-D-22-0132.1]

18. Axisa, D.; DeFelice, T.P. Modern and prospective technologies for weather modification activities: A look at integrating unmanned aircraft systems. Atmos. Res.; 2016; 178, pp. 114-124. [DOI: https://dx.doi.org/10.1016/j.atmosres.2016.03.005]

19. Rasmussen, R.M.; Tessendorf, S.A.; Xue, L.; Weeks, C.; Ikeda, K.; Landolt, S.; Breed, D.; Deshler, T.; Lawrence, B. Evaluation of the Wyoming Weather Modification Pilot Project (WWMPP) using two approaches: Traditional statistics and ensemble modeling. J. Appl. Meteorol. Climatol.; 2018; 57, pp. 2639-2660. [DOI: https://dx.doi.org/10.1175/JAMC-D-17-0335.1]

20. Mielke, P.W. Design and evaluation of weather modification experiments. Probability, Statistics, and Decision Making in the Atmospheric Sciences; CRC Press: Boca Raton, FL, USA, 2019; pp. 439-459. ISBN 9780429303081

21. Wehbe, Y.; Temimi, M.; Adler, R.F. Enhancing precipitation estimates through the fusion of weather radar, satellite retrievals, and surface parameters. Remote Sens.; 2020; 12, 1342. [DOI: https://dx.doi.org/10.3390/rs12081342]

22. Simpson, J.; Woodley, W.L.; Miller, A.H.; Cotton, G.F. Precipitation results of two randomized pyrotechnic cumulus seeding experiments. J. Appl. Meteorol. Climatol.; 1971; 10, pp. 526-544. [DOI: https://dx.doi.org/10.1175/1520-0450(1971)010<0526:PROTRP>2.0.CO;2]

23. Wang, M.; Chen, J.; Li, J.; Wang, J.; Xie, H. Statistic evaluation of low-frequency acoustic wave impact on rainfall stimulation. Stoch. Environ. Res. Risk Assess.; 2023; 37, pp. 611-623. [DOI: https://dx.doi.org/10.1007/s00477-022-02283-2]

24. Zheng, W.; Xue, F.; Zhang, M.; Wu, Q.; Yang, Z.; Ma, S.; Liang, H.; Wang, C.; Wang, Y.; Ai, X. et al. Charged particle (negative ion)-based cloud seeding and rain enhancement trial design and implementation. Water; 2020; 12, 1644. [DOI: https://dx.doi.org/10.3390/w12061644]

25. Zheng, W.; Ma, H.; Zhang, M.; Xue, F.; Yu, K.; Yang, Y.; Ma, S.; Wang, C.; Pan, Y.; Shu, Z. et al. Evaluation of the First Negative Ion-Based Cloud Seeding and Rain Enhancement Trial in China. Water; 2021; 13, 2473. [DOI: https://dx.doi.org/10.3390/w13182473]

26. Wei, X.; Shi, Y.; Qin, F.; Zhou, H.; Shao, X. Effects of experimental warming, precipitation increase and their interaction on AM fungal community in an alpine grassland of the Qinghai-Tibetan Plateau. Eur. J. Soil Biol.; 2021; 102, 103272. [DOI: https://dx.doi.org/10.1016/j.ejsobi.2020.103272]

27. Dou, H.; Ming, D.; Yang, Z.; Pan, Z.; Li, Y.; Tian, J. Object-based visual saliency via laplacian regularized kernel regression. IEEE Trans. Multimed.; 2017; 19, pp. 1718-1729. [DOI: https://dx.doi.org/10.1109/TMM.2017.2689327]

28. Ma, J.; Zhao, J.; Tian, J.; Yuille, A.L.; Tu, Z. Robust point matching via vector field consensus. IEEE Trans. Image Process.; 2014; 23, pp. 1706-1721. [DOI: https://dx.doi.org/10.1109/TIP.2014.2307478]

29. Sun, S.; Liu, R.; Wen, W. Unsupervised classification method for polarimetric synthetic aperture radar imagery based on Yamaguchi four-component decomposition model. J. Electr. Comput. Eng.; 2015; 2015, 39. [DOI: https://dx.doi.org/10.1155/2015/680715]

30. Yang, C.; Zhou, H.; Sun, S.; Liu, R.; Zhao, J.; Ma, J. Good match exploration for infrared face recognition. Infrared Phys. Technol.; 2014; 67, pp. 111-115. [DOI: https://dx.doi.org/10.1016/j.infrared.2014.07.011]

31. Heming, J.T.; Prates, F.; Bender, M.A.; Bowyer, R.; Cangialosi, J.; Caroff, P.; Coleman, T.; Doyle, J.D.; Dube, A.; Faure, G. et al. Review of recent progress in tropical cyclone track forecasting and expression of uncertainties. Trop. Cyclone Res. Rev.; 2019; 8, pp. 181-218. [DOI: https://dx.doi.org/10.1016/j.tcrr.2020.01.001]

32. Farzin, S.; Anaraki, M.V.; Naeimi, M.; Zandifar, S. Prediction of groundwater table and drought analysis; a new hybridization strategy based on bi-directional long short-term model and the Harris hawk optimization algorithm. J. Water Clim. Chang.; 2022; 13, pp. 2233-2254. [DOI: https://dx.doi.org/10.2166/wcc.2022.066]

33. Valikhan Anaraki, M.; Mahmoudian, F.; Nabizadeh Chianeh, F.; Farzin, S. Dye Pollutant Removal from Synthetic Wastewater: A New Modeling and Predicting Approach Based on Experimental Data Analysis, Kriging Interpolation Method, and Computational Intelligence Techniques. J. Environ. Inform.; 2022; 40, pp. 84-94. [DOI: https://dx.doi.org/10.3808/jei.202200473]

34. Xiao, Y.; Su, X.; Yuan, Q.; Liu, D.; Shen, H.; Zhang, L. Satellite video super-resolution via multiscale deformable convolution alignment and temporal grouping projection. IEEE Trans. Geosci. Remote Sens.; 2021; 60, 5610819. [DOI: https://dx.doi.org/10.1109/TGRS.2021.3107352]

35. Xiao, Y.; Yuan, Q.; Jiang, K.; He, J.; Wang, Y.; Zhang, L. From degrade to upgrade: Learning a self-supervised degradation guided adaptive network for blind remote sensing image super-resolution. Inf. Fusion; 2023; 96, pp. 297-311. [DOI: https://dx.doi.org/10.1016/j.inffus.2023.03.021]

36. Jiang, K.; Wang, Z.; Yi, P.; Chen, C.; Wang, Z.; Wang, X.; Jiang, J.; Lin, C.W. Rain-free and residue hand-in-hand: A progressive coupled network for real-time image deraining. IEEE Trans. Image Process.; 2021; 30, pp. 7404-7418. [DOI: https://dx.doi.org/10.1109/TIP.2021.3102504] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34403336]

37. Trebing, K.; Staǹczyk, T.; Mehrkanoon, S. SmaAt-UNet: Precipitation nowcasting using a small attention-UNet architecture. Pattern Recognit. Lett.; 2021; 145, pp. 178-186. [DOI: https://dx.doi.org/10.1016/j.patrec.2021.01.036]

38. Shi, X.; Chen, Z.; Wang, H.; Yeung, D.-Y.; Wong, W.-k.; Woo, W.-C. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. arXiv; 2015; arXiv: 1506.04214

39. Jing, J.; Li, Q.; Peng, X. MLC-LSTM: Exploiting the spatiotemporal correlation between multi-level weather radar echoes for echo sequence extrapolation. Sensors; 2019; 19, 3988. [DOI: https://dx.doi.org/10.3390/s19183988]

40. Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Proceedings, Part III 18; Munich, Germany, 5–9 October 2015; Springer International Publishing: Cham, Switzerland, 2015; pp. 234-241. [DOI: https://dx.doi.org/10.1007/978-3-319-24574-4_28]

41. Kingma, D.P.; Ba, J. A method for stochastic optimization. arXiv; 2014; arXiv: 1412.6980

42. Shi, X.; Gao, Z.; Lausen, L.; Wang, H.; Yeung, D.Y.; Wong, W.K.; Woo, W.C. Deep learning for precipitation nowcasting: A benchmark and a new model. arXiv; 2017; arXiv: 1706.03458

43. Agrawal, S.; Barrington, L.; Bromberg, C.; Burge, J.; Gazen, C.; Hickey, J. Machine learning for precipitation nowcasting from radar images. arXiv; 2019; arXiv: 1912.12132

44. Abbas, M.; Nazir, T.; Lampert, T.A.; Radenović, S. Common fixed points of set-valued F-contraction mappings on domain of sets endowed with directed graph. Comput. Appl. Math.; 2017; 36, pp. 1607-1622. [DOI: https://dx.doi.org/10.1007/s40314-016-0314-z]

45. Shukla, S.; Radenović, S.; Vetro, C. Graphical metric space: A generalized setting in fixed point theory. Revista de la Real Academia de Ciencias Exactas, Físicas y Naturales. Serie A. Matemáticas; 2017; 111, pp. 641-655. [DOI: https://dx.doi.org/10.1007/s13398-016-0316-0]

46. Todorčević, V. Harmonic Quasiconformal Mappings and Hyperbolic Type Metrics; Springer International Publishing: Berlin/Heidelberg, Germany, 2019; [DOI: https://dx.doi.org/10.1007/978-3-030-22591-9]

47. Tang, L.; Deng, Y.; Ma, Y.; Huang, J.; Ma, J. SuperFusion: A versatile image registration and fusion network with semantic awareness. IEEE/CAA J. Autom. Sin.; 2022; 9, pp. 2121-2137. [DOI: https://dx.doi.org/10.1109/JAS.2022.106082]