Quantitative Evaluation of Maize Emergence Using UAV Imagery and Deep Learning

Abstract

Accurate assessment of crop emergence helps breeders select appropriate crop genotypes, and farmers make timely field management decisions to increase maize yields. Crop emergence is conventionally quantified by manual calculations to quantify the number and size of seedlings, which is laborious, inefficient, and unreliable and fails to visualize the spatial distribution and uniformity of seedlings. Phenotyping technology based on remote sensing allows for high-throughput evaluation of crop emergence at the early growth stage. This study developed a system for the rapid estimation of maize seedling emergence based on a deep learning algorithm. The RGB images acquired from an unmanned aerial vehicle (UAV) were used to develop the optimal model for the recognition of seedling location, spacing, and size, and the prediction performance of the system was evaluated in three stations during 2021–2022. A case study was conducted to show the evaluation of the system for maize seedlings and combined with TOPSIS (Technique for Order of Preference by Similarity to Ideal Solution) analysis. The results show that the system has good prediction performance for maize seedling count with an average R² value of 0.96 and an accuracy of 92%; however, shadows and planting density influence its accuracy. The prediction accuracy reduces significantly when the planting density is above 90,000 plants/ha. The distribution characteristics of seedling emergence and growth were also calculated based on the average value and variation coefficient of seedling spacing, seedling area, and seedling length. The estimation accuracies for the average value of seedling spacing, the coefficient of variation of seedling spacing, the average value of the seedling area, the coefficient of variation of the seedling area, and the average value of the seedling length were 87.52, 87.55, 82.69, 84.51, and 90.32%, respectively. In conclusion, the proposed system can quickly analyze the maize seeding growth and uniformity characteristics of experimental plots and locate plots with poor maize emergence.

1. Introduction

The world population is expected to increase by 25% to 10 billion over the next 30 years [1], which means that global food consumption will increase significantly along with population growth [2]. Affected by global climate change, it is a great challenge to obtain increasing food production on limited land resources. This requires making full use of existing land resources to ensure sustainable food production. It is worth noting that crop emergence is a key factor affecting crop yield [3,4].

In the early management of crops, it is crucial to accurately detect the emergence of seedlings and their growth in real time [3]. Crop emergence has traditionally been mostly determined by manual evaluation, which is not only inefficient and costly but also lacks a standardized method [4]. In addition to the easily quantifiable number of seedlings per unit area or specific area, crop emergence also involves seedling size and uniformity of seedling distribution, which cannot be measured manually. Uniformity of crop emergence is crucial to maximize yield as it balances competition among plants for water, nutrients, and sunlight [5]. The uniformity represents more accurately the distribution of crop seedlings in a plot than an average condition [6] and allows for a more accurate evaluation of material quality [7]. Thus, it is necessary to integrate multiple emergence characteristics to realize the evaluation of crop emergence. As the most widely planted crop in the world, maize (Zea mays L.) plays an important role in ensuring food security [8]. Due to advances in cultivation techniques, maize is often grown in large areas, adding complexity and challenges to effective field management. It is crucial to assess maize emergence rapidly in a large area to ensure the quality of sowing, monitor pests and diseases, and make relevant protective measures in time to ensure maize yield [9].

Several researchers have carried out studies to monitor and quantify crop emergence. Several real-time crop growth monitoring systems (i.e., PhenoCams) have been introduced to monitor crop growth and phenological changes in real time [10,11]. However, this system is not mobile, so its range of measurement is limited. Studies have demonstrated that crop seedlings can be assessed using ground vehicle-based imaging systems, but these systems were less efficient at monitoring emergence over large spatial regions [9,12,13,14,15]. Recently, unmanned aerial vehicles (UAVs) equipped with various sensors have become the most effective means of tracking the development of crop seedlings with high-resolution, real-time imaging, efficient acquisition, and easy operation [16]. The established techniques have been demonstrated in a variety of crops and research, such as detecting rice and wheat phenology with an accuracy rate of 83.9% [17] and up to 93% accuracy [18], the monitoring of cotton seedling number and size with R² of 95 and 93% [19], the rapid counting of maize and sunflower seedlings with a relative root mean square error of 4.44 and 4.29% [4], seedling counting of maize at different seedling stages with a root mean square error of 2.0–7.7 [20], seedling counting of wheat with an average accuracy rate of 89% [21] and the uniformity of cotton seedling emergence with an accuracy of 84.1% [22]. Vong et al. [23] used planting density and the standard deviation of planting spacing to evaluate the uniformity of emergence. However, no systematic study on crop emergence has been reported.

Image processing and analysis are critical to images captured by UAV systems. Several challenges currently exist for seedling identification. First, images of crop seedlings captured by drones, displayed in a few pixels, are often very small, making them difficult to identify [19]. Second, UAV-based imaging can collect large amounts of data, which makes image analysis difficult and increases computational complexity [24]. Furthermore, there may be obvious differences in soil types, planting techniques, plant material, and field management between plots, which has a significant impact on image processing [19,25]. The Random Forest classifier was used by Li et al. [26] to estimate the number of potato plants at the emergence stage from six morphological features extracted from UAV RGB images, but this method may not be robust in the case of noise or outliers in the image data. A combination of three machine learning regression methods (regression trees, support vector regression, and Gaussian process regression) combining spectral and morphological information was also applied to estimate the number of wheat seedlings; however, these methods are sensitive to noise and difficult to adapt to diverse environments [27]. Thus, an efficient, robust, adaptable to diverse environments, and real-time image processing tool is urgently needed. Liu et al. [28] compared the prediction performance of the core detection model (RMSE = 3.78), the linear regression model (RMSE = 2.11), and the deep learning model (Faster R-CNN, RMSE = 1.38) in maize seedling counting and found the deep learning model to have the lowest RMSE. Deep learning (DL) is an advanced data processing technique that has been increasingly employed to analyze image data obtained from agricultural applications, and research has also shown that plant information can be extracted from image data efficiently and robustly using deep learning algorithms [29]. This technique is regarded as a successful approach to overcome the difficulties of UAV image analysis [30]. In deep learning, many algorithms have demonstrated high prediction performance in crop seedling counting, such as ResNet-18, which achieves 0.95 and 0.93 R² for cotton counting and canopy size prediction [19], and AlexNet in seedling classification can be 94% accurate [31], while a better version of Yolov5 can achieve 98% accuracy [32]. However, these algorithms are primarily concerned with counting seedlings and ignore a number of important attributes of the seedlings.

Our study aims to develop a framework for image data processing and analysis using UAV-based image processing and deep learning technologies. Compared to previous research, this study considered traits including seedling spacing, growth, and uniformity in addition to the number of maize seedlings per plot. The specific objectives include: (1) developing a new maize seedling analysis system by integrating image processing and deep learning and (2) evaluating the performance of several indicators characterizing crop emergence.

2. Materials and Methods

2.1. Study Sites

Three stations were considered during 2021–2022: Zhuozhou Experimental Station of China Agricultural University in Hebei province, Gongzhuling Experimental Station of the Chinese Academy of Agricultural Science in Jilin province, and Zhangye Experimental Station of Gansu Academy of Agricultural Science in Gansu province. Location information and crop planting settings are shown in Table 1. A multi-station experiment was designed to evaluate the system’s adaptability to diverse ecological areas and understand how that adaptability changes through repeated years of the experiment.

The study focuses on maize, but different stations are treated differently. The planting density experiment was carried out in Zhuozhou Station from 2021–2022, which included 45,000, 63,000, and 81,000 plants/ha. In 2021, materials included Xianyu 335, Zhengdan 958, and ND108. In 2022, 32 inbred lines were planted along with the hybrids that were planted in 2021. In Gongzhuling, 93 transgenic lines were planted in 2021 without treatment, and the planting density experiment was conducted in 2022 with 32 hybrid materials and 31 maize inbred lines. The planting density includes 45,000, 63,000, 81,000, 90,000, and 120,000 plants/ha. In Zhangye, a deficit irrigation experiment based on Genome-Wide Association Studies (GWAS) materials was carried out with 150 inbred lines in 2021 and 108 inbred lines in 2022. Irrigation treatments included full irrigation and deficit irrigation, with the deficit irrigation amount being half that of full irrigation.

2.2. UAV Data Collection

Three UAV-based RGB cameras were utilized to capture high-resolution images at three stations during 2021–2022. The detailed parameters of the UAV are summarized in Table 2. The UAV aerial image data were collected when the plants were at vegetative growth stages V3 to V5. The image data collection time was between 10 a.m. and 2 p.m. local time, coinciding with minimal changes in the solar zenith angle. The UAV’s trajectory was determined prior to flight with a flight speed of 2 m/s. For better image splicing, adjacent images’ footprints must overlap by more than 80% along and across the trajectory. The global positioning system (GPS) and barometer were used to keep horizontal position and altitude inaccuracies to within 2 and 0.5 m, respectively. During the last two years, we maintained the same resolution (0.5 cm/pixel) at three stations, and varying flight altitudes ensured that cameras with different pixels could achieve similar resolutions. The image’s resolution was set to 5472 by 3648 pixels, and the storage format was set to JPEG. The shutter speed of the camera was set to 2 s, and the exposure mode, shutter speed, ISO, and white balance settings were all set to automatic.

2.3. Image Pre-Processing

In our system, the input data for the model are a complete aerial image of the site (Figure 1C). Therefore, before model analysis, the images taken by the UAV needed to be spliced together to form a complete site map. The Pix4Dmapper software [33] was used for the splicing of UAV images (Figure 1B). All UAV images were imported into Pix4D software, and the image coordinate system, geographical positioning, camera model, and other information were set according to the UAV and camera parameters. It is sometimes difficult to ensure the vertical distribution of plots during maize planting, so rows of maize may be inclined. To avoid influencing the segmentation of experimental plots, plots with an obvious inclination were manually corrected using Adobe Photoshop software [34].

2.4. Evaluation for Maize Emergence

2.4.1. Model Training Data Creation and Model Verification Data Collection

In order to create a training data set covering a variety of ecological areas, UAV images of maize seedlings were collected from six experimental stations in 2018–2019, including Gongzhuling in Jilin Province, Zhuozhou City in Hebei Province, Zhangye City in Gansu Province, Anyang City in Henan Province, Sanya in Hainan Province, and Xinxiang in Henan Province. In addition to the main varieties of maize grown in the region, there were also several inbred lines of maize with extensive blood relations. A total of 11,840 UAV images were collected, and each image has a size of 500 × 500 pixels. All seedlings in each image were identified by manually marking their locations using the Python library labelImg [35]. A training dataset was constructed based on images and seedling location information.

A model verification set, which was independent of the training set and based on data from three stations from 2021–2022, was prepared in order to evaluate the predictive accuracy of the training model. The validation dataset was collected and recorded manually. The manually obtained data include data from seedling counts performed manually in the field and data from seedlings marked manually in the UAV images. Seedling counts can easily be performed under the field, thus the actual number of seedlings in all experimental plots at three stations was manually counted by hand in both years as a verification of the model’s ability to predict seedling counts. However, the average value of seedling spacing, coefficient of variation of seedling spacing, the average value of seedling area, and coefficient of variation of seedling area are difficult to measure manually in the field. In order to obtain the real data, we used a UAV to obtain the orthophoto data of seedlings at low altitudes and then used the labelImg tool to manually label the seedlings so that the marker box would just fit the seedlings, thus obtaining the location and scale information of the seedlings. Based on this location and scale information, we then calculated the two-by-two spacing and seedling area information of all seedlings to calculate the average value of seedling spacing, coefficient of variation of seedling spacing, the average value of seedling area, and coefficient of variation of seedling area.

2.4.2. Model Training

The YOLO (You Only Look Once) model, a typical target identification approach, was employed in this work to detect seedlings and uses machine learning to transform the target detection issue into a regression problem [36]. Among other well-known pre-trained CNN models (e.g., Faster RCNN Resnet50 [37], Faster RCNN VGG16 [38], YOLOV5 [39]), Yolo v3 [40] was chosen as the main analysis algorithm because it achieved similar or higher accuracy (Support Information 2). The network structure of seedling detection model is shown in Figure 2. The model’s parameters were trained and established using this training set, and they were then preserved as the foundation for the model’s prediction (Figure 1E). A cross-validation process was used during the training process of the model to ensure its performance. A ratio of 7:2:1 (training/validation/test) was used to divide the training data.

2.4.3. Model Prediction

A complete spliced image of the site was provided as input data for the trained model (Figure 1C). It is, however, difficult to analyze a spliced image directly because it usually takes up more than 1 GB of memory, so the spliced image was first cut into small images (608 × 608 pixels) using sliding window. The sliding window had 480 pixels per step, and the excess pixels were filled in black. The model was fed the small images, and based on the training parameters (Figure 1F,G), it predicted where the maize seedlings are in each small image (Yolo v3 returns the coordinates of the lower-left corner of each maize seedling prediction box, as well as its width and height). It is important to remember that the seedling coordinate parameter in the small image is not the final result. This is because the cutting windows overlap, so there are overlapped areas in the small images next to each other. This helps keep the seedling image from getting damaged while it is being cut. During the cutting process, all of the coordinates for the small images were recorded. Based on these recorded coordinates, the maize seedlings in the small image were mapped to the coordinate system in the large image (spliced image). So, the final result of the prediction was the coordinate information in the spliced image, and each seedling’s information was figured out based on the data (Figure 1H,I). The red rectangular box was used to mark maize seedlings. However, the red rectangular box in the output image is not the actual output coordinates. Instead, it is a box with a specified size determined based on the coordinates of the points in the upper left corner, which were used to indicate the position of the seedlings.

2.4.4. Calculation for Parameters of Seedling Growth and Distribution of Maize

Our objective here is to examine the growth and distribution of maize seedlings within an experimental plot. As a result, the coordinate information generated by the model cannot be applied directly to calculations. By determining plot coordinates, one can separate the plot from the overall site and determine seedling growth and distribution based on the plot’s location. The following characteristics were used to describe the growth and distribution of seedlings in each plot: seedling count, seedling spacing, seedling area, and seedling length. In addition to the seedling count, the average value and coefficient of variation of other characteristics in each plot were computed. The number of seedling prediction windows in each plot was used to evaluate the seedling count. The distance between the center points of adjacent prediction windows was regarded as seedling spacing, and the mean and coefficient of variation of seedling spacing were calculated. The area of the prediction window of the model was regarded as the seedling area, the longest edge was regarded as seedling length, and the average value and coefficient of variation were evaluated.

All these traits were finally output as a CSV data file, including the following variables: Row (the row number of the experimental plot), Col (the column number of the experimental plot), SSm (the mean value of seedling spacing in the experimental plot), SScv (the coefficient of variation of seedling spacing in the experimental plot), SAm (the mean values of seedling area in the experimental plot), SAcv (the coefficient of variation of seedling area in the experimental plot), SLm (the mean values of seedling length in the experimental plot), and SLcv (the coefficient of variation of seedling length in the experimental plot).

2.5. Statistical Analysis

A predictive performance evaluation of the model was conducted using manual data and model prediction data for maize seedlings in different plots at each station in 2021 and 2022. An evaluation of the model’s prediction of maize seedling numbers was conducted by comparing the model’s seedling count data with manual seedling counts collected in the field. For the average value and coefficient of variation of seedling spacing, seedling area, and seedling length, a comparison was made between the data obtained by the model and the data obtained by manual markers. Four statistical parameters, accuracy, mean error (ME), mean absolute error (MAE), and root mean square error (RMSE), were applied to evaluate errors. A simple linear regression was used to fit the relationship between measured and simulated values, and R² was calculated and plotted. The following equations were employed to obtain accuracy, ME, MAE, and RMSE by comparing measured (M) and corresponding predictive (P) values as follows:

(1)$\mathrm{Accuracy}=\frac{1}{\mathrm{N}}{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{N}}\left(100 - \left(\frac{{\mathrm{P}}_{\mathrm{i}}{ - \mathrm{M}}_{\mathrm{i}}}{{\mathrm{M}}_{\mathrm{i}}}\times 100\right)\right)}$

(2)$\mathrm{ME}=\frac{1}{\mathrm{N}}{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{N}}\left({\mathrm{M}}_{\mathrm{i}}{ - \mathrm{P}}_{\mathrm{i}}\right)}$

(3)$\mathrm{MAE}=\frac{1}{\mathrm{N}}{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{N}}\left|{\mathrm{M}}_{\mathrm{i}}{ - \mathrm{P}}_{\mathrm{i}}\right|}$

(4)$\mathrm{RMSE}=\sqrt{\frac{1}{\mathrm{N}}{\displaystyle \sum _{\mathrm{i}=1}^{\mathrm{N}}{\left({\mathrm{M}}_{\mathrm{i}}{ - \mathrm{P}}_{\mathrm{i}}\right)}^2}}$

The Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) is a multiple-criteria decision making (MCDM) method that can integrate multiple traits into a scoring indicator. Here, we use the TOPSIS method based on topsis package in R [41] to synthesize the count, growth, and distribution characteristics of multiple maize seedlings into crop emergence scores to evaluate the emergence of different experimental plots.

3. Results

3.1. Quantitative Maize Emergence

This study proposes a new measurement system for maize emergence that includes not only the seedling count and size but also parameters relating to distribution and uniformity. The measuring system generates two files: a CSV file containing many emergence indicators from various plots and an image of the UAV with a prediction box (Figure 3A,B). Red rectangular boxes (detected boxes) indicate all identified maize seedlings in the output image, while blue rectangular boxes separate experimental plots, allowing us to manually identify seedlings. Since experimental plots are arranged in a regular and uniform size form (Figure 3A), rows and columns are used to identify plots in this system. The system, however, did not export seedling coordinates but rather calculated seedling count, seedling size, seedling distribution, uniformity, and other relevant qualities and exported them as CSV files. The seedling count (seedling number) was used to assess the maize emergence rate, and seedling area, seedling length, and seedling spacing were used to assess the growth and distribution of seedlings. An experimental plot’s seedling spacing indirectly reflects its number of seedlings, while its coefficient of variation indicates how evenly the seedlings are distributed. The average value of seedling size (seedling area and seedling length) reflects the average growth of seedlings in the experimental plot, while the variation coefficient of seedling size indicates the uniformity of seedling growth in the plot.

3.2. Evaluation of Seedling Measurement System for Predicting the Seedling Count of Maize

In our system, seedling detection information was used to generate emergence characteristics in various experimental plots. Therefore, the prediction performance of the seedling count and the factors that influence it were analyzed. Overall, the measurement system performs well in predicting the number of maize seedlings. In three stations during 2021–2022, the R² value is between 0.89 and 0.99, and the accuracy value can reach up to 96.99%; the RMSE is between 28.82 and 214.02; the ME is between 0.78 and 3.77; and the MAE is between 0.88 and 12.36, except at Zhangye Station in 2022 (the accuracy is only 43.96%) (Table 3). The effect of planting density on the seedling count measured by the seedling measurement system was also investigated. Different planting densities were applied at Zhuozhou Station in 2021 and Gongzhuling Station in 2022. In 2021, three planting density treatments were used at Zhuozhou Station (45,000–81,000 plants/ha), and two planting densities were increased at Gongzhuling Station, with the highest density increasing to 120,000 plants/ha in 2022. The results show that planting densities ranging from 45,000 to 81,000 plants/ha could maintain high prediction accuracy. Although prediction accuracy decreased as planting density increased, the average accuracy was greater than 91% (Figure 4A,B). However, prediction accuracy reduces significantly when the planting density is above 90,000 plants/ha.

3.3. Evaluation of the Seedling Measurement System on Prediction of Maize Seedling Distribution

Our system pays attention not only to seedling counts but also to their growth and distribution characteristics. The distribution characteristics of seedlings were evaluated using the coefficients of variation in seedling spacing and seedling size, while growth characteristics were evaluated using the average seedling size. Field measurements, however, cannot provide information about the seedling spacing or seedling size. Accordingly, UAV aerial images were used to determine the coordinates and location of maize seedlings in the field with manual markers. This information was used to calculate the characteristics of the actual seedling spacing, seedling area, and seedling length and to compare the model evaluation with the model prediction. According to Figure 5, the growth and distribution characteristics of seedlings have an R² value between 0.95 and 0.99, an accuracy value between 82 and 91%, an RMSE of up to 4.95, an MAE of up to 0.52, and an ME of −0.05 to 0.002. Therefore, these traits also have excellent prediction performance.

3.4. Application of the Seedling Measurement System

A demonstration of maize emergence in various plots using the seedling measurement system was conducted in 2021 at the experimental site in Zhangye. Heat maps were generated by rebuilding the distribution of plots based on row and column numbers, and different colors were assigned based on the values of different characteristics (Figure 6). Seedling count characteristics indicated that some plots at the site had poor seedling emergence, primarily in the middle and upper regions. The differences in average seedling spacing were found to be relatively small. However, the coefficient of variation of the seedling spacing was quite different, which correlated with the lower seedling counting characteristics (Figure 6 and Figure 7). Seedling size, on the other hand, showed a wide range of variation in size. There were significant differences among materials at the station because most of the materials were maize GWAS materials with different genetic backgrounds. To analyze these parameters and compare maize emergence across plots, the TOPSIS analysis approach [42] was applied to synthesize multiple characteristics into a score index and rank the emergence of all plots based on this score index. The comprehensive score of the emergence of all plots is presented in Table S1 of Support Information 1. The first 10 plots and the last 10 plots in the order of emergence are summarized in Table 4, respectively. The distribution characteristics of the seedling emergence scores of all plots are shown in Figure 8; the result shows that the emergence score of most plots is about 0.1. Consequently, we will be able to identify the plot with the poor seedlings of maize more effectively. In this way, we can easily understand the emergence of all plots in the experimental area and find plots with poor maize seedlings.

4. Discussion

A critical aspect of early field management is crop emergence, which may be assessed by the seedling count, seedling size, and seedling uniformity [19,43]. However, the current crop seedling evaluation is based on the number of seedlings [3,19,44,45,46], which does not account for seedling uniformity. Presently, seedling uniformity is primarily determined through manual qualitative evaluation, which greatly affects the uniformity index [6]. In this measurement system, the growth and distribution characteristics of seedlings are incorporated. Seedling coordinates as well as experimental plot coordinates are used to calculate the seedling count in each experimental plot, and the mean and coefficient of variation of the seedling spacing, as well as seedling size (seedling area and seedling length). A coefficient of variation based on seedling spacing and seedling size in the experimental area is calculated to reflect the characteristics of distribution uniformity and growth uniformity. Despite some research having been conducted on seedling uniformity analysis, this method of evaluating seedling distribution in plots using seedling coordinate information has significant advantages when compared to other evaluation methods that use spectrum [22] or complex algorithms (such as pixel distribution features) [6] in terms of efficiency, cost, and accuracy.

According to our results, the measurement system in this study was able to accurately predict the seedlings count for most scenarios, similar to Barreto et al. [47] and Bai et al. [4]. However, a high error rate was observed at Zhangye Station in 2022, with an accuracy of only 43.96%. Aerial images of this station showed that seedlings in the field were clearly visible and in shadow, which affected the accuracy of the measurement system (Figure S1). Furthermore, maize GWAS inbred line materials were the primary planting materials at Zhangye Station in 2021 and the three stations in 2022. Maize inbred lines tend to have more complex morphological characteristics [48], which can impact accuracy. The impact of planting density on measurement accuracy was also studied. Plant density will not significantly affect the prediction performance of the measurement system in the appropriate range (45,000–81,000 plants/ha). However, when planting density exceeds 81,000 plants/ha, the measurement system will significantly lower its prediction performance. The same phenomenon was observed by Liu et al. [49] and has been attributed to the overlapping of leaves under high density. Recent decades have seen an increase in planting density as an effective way to improve maize yield [50], and to a substantial extent, this has contributed to the rise in yield [51]. Our measuring system is well suited to adapt to the influence of planting density in the future.

A seedling measurement system’s prediction accuracy can be affected by the choice of models and the quality of UAV image acquisition [22,32,52]. Yolov3 was used as the target detection model in this measurement system. Our previous work reported that several algorithms and backbone networks (Faster_RCNN_resnet50, Faster_RCNN_VGG16, Yolov3, and Yolov5) had differences in model accuracy, and Yolo v3 and v5 had better seedling detection performance than Fast RNN (Figures S3 and S4). The Yolo v3 model performed better than Yolov5 in detecting maize seedlings at different sites. This study uses Yolo v3 as a seedling detection algorithm (Table S2). In subsequent iterations, the updated model will be further enhanced. Second, we have selected relatively cheap UAVs, which will reduce the promotion and application costs. In spite of the fact that cameras with higher resolution and UAV platforms with stronger performance can achieve better performance, this inevitably increases the cost of usage. As part of this system, relatively low-performance UAVs were utilized to collect UAV images at a low cost. Compared with other more powerful devices for seedling detection research [3,14,53], this device was more practical.

5. Conclusions

In this study, a maize seedling measurement system was developed that considered the characteristics of the seedling count, size, uniformity, and distribution comprehensively. Images were captured by unmanned aerial vehicles. The measurement system for maize emergence is accurate, but shadows and planting density influence its accuracy.

The TOPSIS method was used to synthesize multiple maize emergence indicators into a single emergence score. The emergence of all experimental plots can be efficiently sorted based on this score so that the best and worst plots for emergence can be identified. The distribution characteristics of the emergence of all plots can also be easily determined based on this score. The emergence of seedlings on the experimental plot is analyzed in this way efficiently. Therefore, our system allows us to easily analyze the seeding quality and seedling emergence of various experimental plots and locate those with poor seedling emergence.

Author Contributions

Conceptualization, overall design, experiment arrangement and execution, X.-Q.W.; formal analysis, M.L.; writing—original draft preparation, M.L.; writing—review and editing, W.-H.S.; visualization, M.L. All authors have read and agreed to the published version of the manuscript.

Data Availability Statement

Data are available on request due to restrictions, e.g., privacy or ethical.

Acknowledgments

The authors wish to thank Xiaoming Zhao for help in the data collection under field, and Wei Li and Yue Zhang for their help with image processing.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. Workflow of image preprocessing and model prediction. (A) Images captured by an unmanned aerial vehicle (UAV); (B) Pix4d software was used to splice images captured by the UAV into a large image; (C) calibrated image; (D) labeled image; (E) model training process; (F) split the large image into small patches (608 × 608 pixels) for model input; (G) load model parameters for model prediction; (H) result image of model output; (I) enlarged view of the output marked image; (J) numerical results of model output.

Figure 2. The architecture of maize seedling detection model, as applied to the estimation of seedling count and seedling size.

Figure 3. Output of the seedling measurement system. (A) Image with rectangular prediction box from the output of the seedling measurement system; (B) enlarged view of part of the output image; (C) the relationship between the output indicators of the measurement system.

Figure 4. Evaluation of the seedling measurement system for predicting the seedling count of maize under different planting densities. (A) Prediction performance of the seedling measurement system in different planting density treatments at Zhuozhou Station in 2021; (B) prediction performance of the seedling measurement system in different planting density treatments at Gongzhuling Station in 2022.

Figure 5. The relationship between the model output and the manually measured data of seedling spacing and seedling size-related indicators. SSm: the average value of seedling spacing in the experimental plot; SScv: the coefficient of variation of seedling spacing in the experimental plot; SAm: the average value of seedling area in the experimental plot; SScv: the coefficient of variation of seedling area in the experimental plot; SLm: the average value of seedling spacing in the experimental plot.

Figure 6. The growth and distribution characteristics of maize seedlings at Zhangye Station in 2021. CV: the coefficient of variation.

Figure 7. Correlation analysis between growth and distribution characteristics of seedlings at Zhangye Station in 2021. Count: seedling count; SSm: the average value of seedling spacing in the experimental plot; SScv: the coefficient of variation of seedling spacing in the experimental plot; SAm: the average value of seedling area in the experimental plot; SScv: the coefficient of variation of seedling area in the experimental plot; SLm: the average value of seedling spacing in the experimental plot.

Figure 8. Probability distribution diagram of the TOPSIS score value of seedling emergence in all plots of Zhangye Experimental Station in 2021.

Table 1

Location and crop planting settings in Zhuozhou, Gongzhuling, and Zhangye during 2021–2022.

Year	2021			2022
Station	Zhuozhou	Gongzhuling	Zhangye	Zhuozhou	Gongzhuling	Zhangye
Latitude	N39.5°	N43.5°	N38.8°	N39.5°	N43.5°	N38.8°
Longitude	E115.8°	E124.8°	E100.37°	E115.8°	E124.8°	E100.37°
Altitude (m)	45	200	1548	45	200	1548
Soil	Sandy loam	Chernozem	Irrigated desert soil	Sandy loam	Chernozem	Irrigated desert soil
Crop	Hybrid line	Transgenic line	Inbred line	Hybrid and inbred line	Hybrid and inbred line	Inbred line
Row spacing (m)	0.6	0.6	0.6	0.6	0.6	0.6
Plant spacing (m)	0.206–0.37	0.25	0.25	0.206–0.37	0.206–0.37	0.25
Number of plots	217	641	301	771	1024	300

Table 2

The detailed parameters of the unmanned aerial vehicle system (UAV) at different stations during 2021–2022.

Year	2021			2022
Site	Zhuozhou	Gongzhuling	Zhangye	Zhuozhou	Gongzhuling	Zhangye
Stage	4–5 leaves	4 leaves	5 leaves	5 leaves	5 leaves	5 leaves
Date	5/24	6/15	5/24	6/1	6/14	6/11
UAV	MAVIC PRO2	MAVIC PRO2	MAVIC PRO1	MAVIC PRO1	MAVIC PRO2	MAVIC PRO1
Height above ground (m)	20	20	15	15	20	15
Stitching accuracy (RMS Error: x, y, z, %)	0.50, 0.23, 0.37	0.56, 0.65, 1.43	0.70, 0.78, 0.76	0.44, 0.37, 0.37	0.38, 0.43, 1.10	0.16, 0.28, 0.27

Table 3

Performance of the seedling measurement system in predicting maize seedling count.

Year	Station	R2	Accuracy	RMSE	ME	MAE
2021	Gongzhuling	0.9937	92.00	28.82	1.14	1.15
2021	Zhangye	0.9641	83.93	32.50	1.88	3.20
2021	Zhuozhou	0.9984	96.99	11.50	0.78	0.88
2022	Gongzhuling	0.8897	77.76	120.19	3.77	3.81
2022	Zhangye	0.9358	43.96	214.02	12.36	12.36
2022	Zhuozhou	0.9753	89.00	68.52	2.53	2.57

Table 4

Top 10 plots and last 10 plots of information based on TOPSIS for maize seedling emergence at Zhangye Station in 2021.

The Type	ID	Row	Col	Score	Rank
Top 10 plots	4_34	4	34	0.907732	1
	4_35	4	35	0.737265	2
	4_43	4	43	0.663084	3
	4_36	4	36	0.602543	4
	4_39	4	39	0.487979	5
	4_9	4	9	0.450847	6
	4_22	4	22	0.408759	7
	4_41	4	41	0.401846	8
	4_61	4	61	0.344274	9
	4_52	4	52	0.328818	10
The last 10 plots	4_25	4	25	0.079492	207
	2_73	2	73	0.078795	208
	2_6	2	6	0.078173	209
	3_30	3	30	0.077785	210
	4_6	4	6	0.077404	211
	2_44	2	44	0.07678	212
	3_49	3	49	0.075722	213
	3_74	3	74	0.075224	214
	4_63	4	63	0.071147	215
	4_47	4	47	0.061903	216

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/rs15081979/s1, Figure S1: Aerial image of maize seedlings at Zhangye Experimental Station in 2022; Figure S2: The measurement system removes the influence of weeds outside the row through the detection of maize rows; Figure S3: Performance comparison of four algorithms (Faster_RCNN_resnet50, Faster_RCNN_VGG16, YOLO_v3, and YOLO_v5) in maize seedling prediction; Figure S4: Model performance comparison of different models (YOLO_v3 and YOLO_v5). Table S1: Ranking of maize emergence at experimental plots of Zhangye Station in 2021 based on TOPSIS; Table S2: Comparison of accuracy between YOLO_v3 and YOLO_v5 in the prediction of seedling count of maize.

References

1. Searchinger, T.; Waite, R.; Hanson, C.; Ranganathan, J.; Dumas, P.; Matthews, E.; Klirs, C. Creating a Sustainable Food Future: A Menu of Solutions to Feed Nearly 10 Billion People by 2050. Final Report; World Resources Institute: Washington, DC, USA, 2019.

2. FAO. The State of Food and Agriculture: Climate Change, Agriculture and Food Security; Food and Agriculture Organization of the United Nations: Rome, Italy, 2016.

3. Shirzadifar, A.; Maharlooei, M.; Bajwa, S.G.; Oduor, P.G.; Nowatzki, J.F. Mapping Crop Stand Count and Planting Uniformity Using High Resolution Imagery in a Maize Crop. Biosyst. Eng.; 2020; 200, pp. 377-390. [DOI: https://dx.doi.org/10.1016/j.biosystemseng.2020.10.013]

4. Bai, Y.; Nie, C.; Wang, H.; Cheng, M.; Liu, S.; Yu, X.; Shao, M.; Wang, Z.; Wang, S.; Tuohuti, N. et al. A Fast and Robust Method for Plant Count in Sunflower and Maize at Different Seedling Stages Using High-Resolution UAV RGB Imagery. Precis. Agric.; 2022; 23, pp. 1720-1742. [DOI: https://dx.doi.org/10.1007/s11119-022-09907-1]

5. Karayel, D.; Özmerzi, A. Evaluation of Three Depth-Control Components on Seed Placement Accuracy and Emergence for a Precision Planter. Appl. Eng. Agric.; 2008; 24, pp. 271-276. [DOI: https://dx.doi.org/10.13031/2013.24494]

6. Liu, T.; Li, R.; Jin, X.; Ding, J.; Zhu, X.; Sun, C.; Guo, W. Evaluation of Seed Emergence Uniformity of Mechanically Sown Wheat with UAV RGB Imagery. Remote Sens.; 2017; 9, 1241. [DOI: https://dx.doi.org/10.3390/rs9121241]

7. Egli, D. Relationship of Uniformity of Soybean Seedling Emergence to Yield. J. Seed Technol.; 1993; 17, pp. 22-28.

8. Ranum, P.; Peña-Rosas, J.P.; Garcia-Casal, M.N. Global Maize Production, Utilization, and Consumption. Ann. N. Y. Acad. Sci.; 2014; 1312, pp. 105-112. [DOI: https://dx.doi.org/10.1111/nyas.12396]

9. Liu, S.; Baret, F.; Allard, D.; Jin, X.; Andrieu, B.; Burger, P.; Hemmerlé, M.; Comar, A. A Method to Estimate Plant Density and Plant Spacing Heterogeneity: Application to Wheat Crops. Plant Methods; 2017; 13, 38. [DOI: https://dx.doi.org/10.1186/s13007-017-0187-1]

10. Liu, Y.; Bachofen, C.; Wittwer, R.; Duarte, G.S.; Sun, Q.; Klaus, V.H.; Buchmann, N. Using PhenoCams to Track Crop Phenology and Explain the Effects of Different Cropping Systems on Yield. Agric. Syst.; 2022; 195, 103306. [DOI: https://dx.doi.org/10.1016/j.agsy.2021.103306]

11. Aasen, H.; Kirchgessner, N.; Walter, A.; Liebisch, F. PhenoCams for Field Phenotyping: Using Very High Temporal Resolution Digital Repeated Photography to Investigate Interactions of Growth, Phenology, and Harvest Traits. Front. Plant Sci.; 2020; 11, 593. [DOI: https://dx.doi.org/10.3389/fpls.2020.00593]

12. Jin, X.; Liu, S.; Baret, F.; Hemerlé, M.; Comar, A. Estimates of Plant Density of Wheat Crops at Emergence from Very Low Altitude UAV Imagery. Remote Sens. Environ.; 2017; 198, pp. 105-114. [DOI: https://dx.doi.org/10.1016/j.rse.2017.06.007]

13. Maertens, K.; Reyns, P.; De Clippel, J.; De Baerdemaeker, J. First Experiments on Ultrasonic Crop Density Measurement. J. Sound Vib.; 2003; 266, pp. 655-665. [DOI: https://dx.doi.org/10.1016/S0022-460X(03)00591-1]

14. Shi, Y.; Wang, N.; Taylor, R.; Raun, W. Improvement of a Ground-LiDAR-Based Corn Plant Population and Spacing Measurement System. Comput. Electron. Agric.; 2015; 112, pp. 92-101. [DOI: https://dx.doi.org/10.1016/j.compag.2014.11.026]

15. Bai, J.; Li, J.; Li, S. Monitoring the Plant Density of Cotton with Remotely Sensed Data. Proceedings of the International Conference on Computer and Computing Technologies in Agriculture; Springer: Berlin/Heidelberg, Germany, 2010; pp. 90-101.

16. Ren, H.; Zhao, Y.; Xiao, W.; Hu, Z. A Review of UAV Monitoring in Mining Areas: Current Status and Future Perspectives. Int. J. Coal Sci. Technol.; 2019; 6, pp. 320-333. [DOI: https://dx.doi.org/10.1007/s40789-019-00264-5]

17. Yang, Q.; Shi, L.; Han, J.; Yu, J.; Huang, K. A near Real-Time Deep Learning Approach for Detecting Rice Phenology Based on UAV Images. Agric. For. Meteorol.; 2020; 287, 107938. [DOI: https://dx.doi.org/10.1016/j.agrformet.2020.107938]

18. Zhou, M.; Zheng, H.; He, C.; Liu, P.; Awan, G.M.; Wang, X.; Cheng, T.; Zhu, Y.; Cao, W.; Yao, X. Wheat Phenology Detection with the Methodology of Classification Based on the Time-Series UAV Images. Field Crops Res.; 2023; 292, 108798. [DOI: https://dx.doi.org/10.1016/j.fcr.2022.108798]

19. Feng, A.; Zhou, J.; Vories, E.; Sudduth, K.A. Evaluation of Cotton Emergence Using UAV-Based Imagery and Deep Learning. Comput. Electron. Agric.; 2020; 177, 105711. [DOI: https://dx.doi.org/10.1016/j.compag.2020.105711]

20. Che, Y.; Wang, Q.; Zhou, L.; Wang, X.; Li, B.; Ma, Y. The Effect of Growth Stage and Plant Counting Accuracy of Maize Inbred Lines on LAI and Biomass Prediction. Precis. Agric.; 2022; 23, pp. 2159-2185. [DOI: https://dx.doi.org/10.1007/s11119-022-09915-1]

21. Liu, T.; Wu, W.; Chen, W.; Sun, C.; Zhu, X.; Guo, W. Automated Image-Processing for Counting Seedlings in a Wheat Field. Precis. Agric.; 2016; 17, pp. 392-406. [DOI: https://dx.doi.org/10.1007/s11119-015-9425-6]

22. Feng, A.; Zhou, J.; Vories, E.; Sudduth, K.A. Evaluation of Cotton Emergence Using UAV-Based Narrow-Band Spectral Imagery with Customized Image Alignment and Stitching Algorithms. Remote Sens.; 2020; 12, 1764. [DOI: https://dx.doi.org/10.3390/rs12111764]

23. Vong, C.N.; Conway, L.S.; Feng, A.; Zhou, J.; Kitchen, N.R.; Sudduth, K.A. Corn Emergence Uniformity Estimation and Mapping Using UAV Imagery and Deep Learning. Comput. Electron. Agric.; 2022; 198, 107008. [DOI: https://dx.doi.org/10.1016/j.compag.2022.107008]

24. Forcella, F.; Arnold, R.L.B.; Sanchez, R.; Ghersa, C.M. Modeling Seedling Emergence. Field Crops Res.; 2000; 67, pp. 123-139. [DOI: https://dx.doi.org/10.1016/S0378-4290(00)00088-5]

25. Egli, D.; Rucker, M. Seed Vigor and the Uniformity of Emergence of Corn Seedlings. Crop Sci.; 2012; 52, pp. 2774-2782. [DOI: https://dx.doi.org/10.2135/cropsci2012.01.0064]

26. Li, B.; Xu, X.; Han, J.; Zhang, L.; Bian, C.; Jin, L.; Liu, J. The Estimation of Crop Emergence in Potatoes by UAV RGB Imagery. Plant Methods; 2019; 15, 15. [DOI: https://dx.doi.org/10.1186/s13007-019-0399-7]

27. Banerjee, B.P.; Sharma, V.; Spangenberg, G.; Kant, S. Machine Learning Regression Analysis for Estimation of Crop Emergence Using Multispectral UAV Imagery. Remote Sens.; 2021; 13, 2918. [DOI: https://dx.doi.org/10.3390/rs13152918]

28. Liu, S.; Yin, D.; Feng, H.; Li, Z.; Xu, X.; Shi, L.; Jin, X. Estimating Maize Seedling Number with UAV RGB Images and Advanced Image Processing Methods. Precis. Agric.; 2022; 23, pp. 1604-1632. [DOI: https://dx.doi.org/10.1007/s11119-022-09899-y]

29. Valente, J.; Sari, B.; Kooistra, L.; Kramer, H.; Mücher, S. Automated Crop Plant Counting from Very High-Resolution Aerial Imagery. Precis. Agric.; 2020; 21, pp. 1366-1384. [DOI: https://dx.doi.org/10.1007/s11119-020-09725-3]

30. Kamilaris, A.; Prenafeta-Boldú, F.X. Deep Learning in Agriculture: A Survey. Comput. Electron. Agric.; 2018; 147, pp. 70-90. [DOI: https://dx.doi.org/10.1016/j.compag.2018.02.016]

31. Perugachi-Diaz, Y.; Tomczak, J.M.; Bhulai, S. Deep Learning for White Cabbage Seedling Prediction. Comput. Electron. Agric.; 2021; 184, 106059. [DOI: https://dx.doi.org/10.1016/j.compag.2021.106059]

32. Lin, Y.; Chen, T.; Liu, S.; Cai, Y.; Shi, H.; Zheng, D.; Lan, Y.; Yue, X.; Zhang, L. Quick and Accurate Monitoring Peanut Seedlings Emergence Rate through UAV Video and Deep Learning. Comput. Electron. Agric.; 2022; 197, 106938. [DOI: https://dx.doi.org/10.1016/j.compag.2022.106938]

33. Pix4D SA. Pix4Dmapper. Available online: https://www.pix4d.com/product/pix4dmapper-photogrammetry-software (accessed on 15 November 2021).

34. Adobe Photoshop, Adobe. Available online: https://www.adobe.com/products/photoshop.html (accessed on 20 April 2019).

35. Lin, T. LabelImg. 2015; Available online: https://github.com/heartexlabs/labelImg (accessed on 13 September 2022).

36. Huang, Y.Q.; Zheng, J.C.; Sun, S.D.; Yang, C.F.; Liu, J. Optimized YOLOv3 Algorithm and Its Application in Traffic Flow Detections. Appl. Sci.; 2020; 10, 3079. [DOI: https://dx.doi.org/10.3390/app10093079]

37. Koonce, B.; Koonce, B. ResNet 50. Convolutional Neural Networks with Swift for Tensorflow: Image Recognition and Dataset Categorization; APress: New York, NY, USA, 2021; pp. 63-72.

38. Barhoom, A.M.; Al-Hiealy, M.R.J.; Abu-Naser, S.S. Bone Abnormalities Detection and Classification Using Deep Learning-Vgg16 Algorithm. J. Theor. Appl. Inf. Technol.; 2022; 100, pp. 6173-6184.

39. YOLO-V5. Available online: https://github.com/ultralytics/yolov5 (accessed on 13 August 2020).

40. Redmon, J.; Farhadi, A. Yolov3: An Incremental Improvement. arXiv; 2018; arXiv: 1804.02767

41. Yazdi, M.M. Topsis: TOPSIS Method for Multiple-Criteria Decision Making (MCDM). 2013; Available online: https://cran.r-project.org/web/packages/topsis/ (accessed on 6 October 2022).

42. Olson, D.L. Comparison of Weights in TOPSIS Models. Math. Comput. Model.; 2004; 40, pp. 721-727. [DOI: https://dx.doi.org/10.1016/j.mcm.2004.10.003]

43. Sansone, C.; Isakeit, T.; Lemon, R.; Warrick, B. Texas Cotton Production: Emphasizing Integrated Pest Management; TCE Magazine: College Station, TX, USA, 2002.

44. Sun, C.; Bian, Y.; Zhou, T.; Pan, J. Using of Multi-Source and Multi-Temporal Remote Sensing Data Improves Crop-Type Mapping in the Subtropical Agriculture Region. Sensors; 2019; 19, 2401. [DOI: https://dx.doi.org/10.3390/s19102401]

45. Jiang, Y.; Li, C.; Paterson, A.H.; Robertson, J.S. DeepSeedling: Deep Convolutional Network and Kalman Filter for Plant Seedling Detection and Counting in the Field. Plant Methods; 2019; 15, 141. [DOI: https://dx.doi.org/10.1186/s13007-019-0528-3]

46. Wu, W.; Liu, T.; Zhou, P.; Yang, T.; Li, C.; Zhong, X.; Sun, C.; Liu, S.; Guo, W. Image Analysis-Based Recognition and Quantification of Grain Number per Panicle in Rice. Plant Methods; 2019; 15, 122. [DOI: https://dx.doi.org/10.1186/s13007-019-0510-0]

47. Barreto, A.; Lottes, P.; Yamati, F.R.I.; Baumgarten, S.; Wolf, N.A.; Stachniss, C.; Mahlein, A.-K.; Paulus, S. Automatic UAV-Based Counting of Seedlings in Sugar-Beet Field and Extension to Maize and Strawberry. Comput. Electron. Agric.; 2021; 191, 106493. [DOI: https://dx.doi.org/10.1016/j.compag.2021.106493]

48. Al-Naggar, A.; Soliman, A.; Hussien, M.; Mohamed, A. Genetic Diversity of Maize Inbred Lines Based on Morphological Traits and Its Association with Heterosis. SABRAO J. Breed. Genet.; 2022; 54, pp. 589-597. [DOI: https://dx.doi.org/10.54910/sabrao2022.54.3.11]

49. Liu, S.; Baret, F.; Andrieu, B.; Burger, P.; Hemmerlé, M. Estimation of Wheat Plant Density at Early Stages Using High Resolution Imagery. Front. Plant Sci.; 2017; 8, 739.

50. Zhang, Y.; Wang, R.; Wang, S.; Ning, F.; Wang, H.; Wen, P.; Li, A.; Dong, Z.; Xu, Z.; Zhang, Y. et al. Effect of Planting Density on Deep Soil Water and Maize Yield on the Loess Plateau of China. Agric. Water Manag.; 2019; 223, 105655. [DOI: https://dx.doi.org/10.1016/j.agwat.2019.05.039]

51. Zhang, M.; Tao, C.; Latifmanesh, H.; Feng, X.; Cao, T.; Qian, C.; Deng, A.; Song, Z.; Zhang, W. How Plant Density Affects Maize Spike Differentiation, Kernel Set, and Grain Yield Formation in Northeast China?. J. Integr. Agric.; 2018; 17, pp. 1745-1757. [DOI: https://dx.doi.org/10.1016/S2095-3119(17)61877-X]

52. Armalivia, S.; Zainuddin, Z.; Achmad, A.; Wicaksono, M.A. Automatic Counting Shrimp Larvae Based You Only Look Once (YOLO). Proceedings of the 2021 International Conference on Artificial Intelligence and Mechatronics Systems (AIMS); Bandung, Indonesia, 28–30 April 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 1-4.

53. Gao, M.; Yang, F.; Wei, H.; Liu, X. Individual Maize Location and Height Estimation in Field from UAV-Borne LiDAR and RGB Images. Remote Sens.; 2022; 14, 2292. [DOI: https://dx.doi.org/10.3390/rs14102292]