Efficient Semantic Segmentation for Large-Scale Agricultural Nursery Managements via Point Cloud-Based Neural Network

Abstract

Remote sensing technology has found extensive application in agriculture, providing critical data for analysis. The advancement of semantic segmentation models significantly enhances the utilization of point cloud data, offering innovative technical support for modern horticulture in nursery environments, particularly in the area of plant cultivation. Semantic segmentation results aid in obtaining tree components, like canopies and trunks, and detailed data on tree growth environments. However, obtaining precise semantic segmentation results from large-scale areas can be challenging due to the vast number of points involved. Therefore, this paper introduces an improved model aimed at achieving superior performance for large-scale points. The model incorporates direction angles between points to improve local feature extraction and ensure rotational invariance. It also uses geometric and relative distance information for better adjustment of different neighboring point features. An external attention module extracts global spatial features, and an upsampling feature adjustment strategy integrates features from the encoder and decoder. A specialized dataset was created from real nursery environments for experiments. Results show that the improved model surpasses several point-based models, achieving a Mean Intersection over Union ($mIoU$) of 87.18%. This enhances the precision of nursery environment analysis and supports the advancement of autonomous nursery managements.

1. Introduction

Nurseries are special spaces for cultivating and nurturing seedlings, offering a diverse range of seedlings to local urban forests to meet both local environmental requirements and market demands [1,2]. Apart from contributions to the ecological development, environmental preservation, and urban beautification, nurseries also play a key role in promoting agricultural economic growth. Therefore, it is essential to establish effective management and monitoring practices for nursery environments.

Actually, the majority of nurseries occupy large areas, making manual management impractical without adequate allocation of human resources. Recently, significant progress in the field of science and technology has facilitated the employment of agricultural robots into nursery management and monitoring practices [3,4,5]. These technological developments have resulted in substantial savings in terms of human resources and costs. Particularly, the simultaneous localization and mapping (SLAM) related to remote sensing technology [6,7,8,9,10,11] can facilitate the localization tasks for robots and establishing maps of nurseries. Despite these advancements, SLAM techniques primarily focus on spatial locations of objects, often neglecting specific details such as category, tree crowns, trunks, and so on. This limitation obstructs agricultural robots from performing effective detection. Information regarding the crowns or trunks of trees can not only reflect the growth situation, but also help agricultural robots in implementing various management operations. For instance, the crown volume can be used to determine variable spray operations, while the trunk information can realize navigation operations without human interventions [12,13,14,15,16]. Implementing segmentation tasks can resolve the aforementioned problems. Point cloud semantics facilitate the identification of specific points corresponding to various common components within nurseries, including tree canopies, trunks, supporting poles, the ground, and so on.

In recent years, deep learning technologies have gained significant attention in the field of semantic segmentation, particularly in the agricultural digital domain. Image-based semantic segmentation has emerged as one of the pioneering and increasingly mature techniques, showcasing successful implementations in tasks such as crown detection [17,18] and navigation path recognition [15]. However, it is important to acknowledge that images are more susceptible to variations in lighting conditions compared to point clouds. Changes in illumination can cause fluctuations in pixel values, ultimately impacting the accuracy of the final segmentation results. In contrast, point clouds are primarily composed of three-dimensional coordinates, making them more robust to different lighting conditions. Given the fact that nurseries are predominantly situated in outdoor environments, employing point cloud data for semantic segmentation tasks is more suitable.

The maintenance of trees in nurseries is of primary importance. Various methods based on point clouds have been developed to perform segmentation tasks and extract more detailed information about trees. Ref. [19] designs an unsupervised pipeline based on a superpoint graph to segment individual trees and classify leaves or wood. A coarse-to-fine tree segmentation methodology is implemented in [20] for individual landscape trees. Additionally, Ref. [21] proposes the scaled PointNet++ for the segmentation tasks of landscape trees in urban environments, eliminating the need for voxelization of point clouds. Ref. [22] analyzes the vertical distribution of LiDAR point clouds and divides them into an overstory and multiple understory tree canopy layers, realizing the segmentation of tree crowns. However, it is important to note that the aforementioned models and methods primarily focus on trees in forests or street trees in urban areas, which differ significantly from trees grown in nurseries. Firstly, trees in forests tend to be taller compared to those in nurseries. And the nurturing environments are also different. Apart from vegetation, nurseries incorporate other objects like planting pots and supporting poles, which play the crucial role during the growth stages of seedling and are typically absent in forest settings. In urban environments, landscape trees usually grow over many years and reach adulthood. In nurseries, most of trees are in the seedling stage and the extracted features are also different. Therefore, the data or methods from the above studies can not be used in the nursery environments directly. In [18], a lightweight model is proposed for the segmentation of fruit tree crowns. The approach combines an attention mechanism, a focal loss function, and U-Net. Although the characteristics and growth environments of fruit trees in orchards are similar to those of seedlings in nurseries, the method devised in [18] focuses on segmenting tasks based on point cloud data collected by unmanned aerial vehicles. However, due to occlusion issues, it becomes difficult to obtain some information related to trunks and ground conditions from the air, which are highly relevant for ground operations of agricultural robots. Ref. [23] employs a 3D sensor system to segment tree canopy and support poles by eliminating the trellis wires. However, this study is aimed at the semantic segmentation task for each individual tree. Therefore, limited research currently exists on the segmentation of detailed information about trees from large-scale groups of point clouds due to the need for excessive computing resources. The large-scale nursery point cloud semantic maps support details of each tree in the nursery, introducing novel technical support for contemporary horticulture, particularly in the realm of plant cultivation.

This proposed paper has primarily two contributions. Firstly, a special point cloud dataset from a real nursery has been established. Currently, most large-scale point cloud datasets are focused on indoor or urban street scenes (e.g., S3DIS, ScanNet, KITTI, DublinCity, and SensatUrban), and those specifically targeting nursery environments are relatively scarce. As a result, models trained on existing datasets cannot be directly applied to nursery environments. Thus, the creation of a specialized nursery dataset is essential for training related neural networks, helping agricultural robots to perform perception tasks within nurseries. To avoid the problem of occlusion, we collected point clouds from the real nursery scene using a ground robot. According to the actual situation, each point was classified into five categories: crown, trunk, planting pot, supporting pole, and ground. The models trained on this dataset empower agricultural robots to perform many tasks. Consequently, the establishment of a specialized point cloud dataset tailored to nurseries holds significant importance in agricultural applications.

The second contribution of this paper is the introduction of a specialized semantic segmentation model tailored for large-scale point cloud tasks. At present, only a few models are able to efficiently perform semantic segmentation tasks on large-scale point clouds. The proposed model, building upon the classical semantic segmentation framework of RandLA-Net, incorporates significant improvements in three crucial aspects. Firstly, in the local feature extraction module, the relative directional angles between the neighbor points and the center point around XYZ-axis in each local area are incorporated, reflecting the rotational invariance feature of the point clouds. And the geometric distance and relative distance features are utilized to significantly improve the ability to extract local features. Secondly, an external attention module is integrated into the model. This module enables the extraction of global spatial features from the entire set of point clouds. Finally, the improved model involves leveraging the features obtained from the encoder to adjust the corresponding features in the decoder before concatenating them. This operation aims to reduce the differences between encoder features and decoder features, thereby facilitating the fusion of these two types of features. The enhanced model is capable of achieving higher-precision semantic segmentation results. When applied to nursery scenarios, it provides more accurate environmental information, thereby aiding agricultural robots in precisely perceiving their operational environments.

2. Materials and Methods

2.1. Data Acquisition and Processing

The experimental data collection site was a nursery located in Zhenjiang, Jiangsu Province, China. The nursery, spanning a vast area, specializes in cultivating various common ornamental trees. It serves as a reliable source of high-quality seedlings for landscaping purposes. All seedlings were planted in pots filled with biochar soil and yellow soil. Additionally, each seedling was provided with supporting poles to ensure their healthy growth during the seedling stage. Figure 1 is the photo of the nursery from the air and the ground. The distance between trees varied from 3 to 5 m, ensuring that the machine antenna avoided unobstructed occlusion during the data acquisition process to capture enough data.

The machine used to collect data is shown in Figure 2. It was equipped with an internal computer, a VLP-16 LiDAR, a Xsense 680MTi IMU sensor, a Si Nan K823 high precision positioning board card, and a SCOUT MINI Experimental by-wire Chassis. The lidar was mounted on an aluminum alloy base, positioned at a height of 185 mm from the top of the machine. Figure 3 shows the collection process.

During the movement of the machine, the lidar continuously captured environmental information and organized the sampled data into a frame of point clouds. The point cloud data of each frame was collected based on the coordinates of the LiDAR in the moment of capture. Since the data collection machine was in constant motion during the process, the central coordinates of each point cloud frame continuously changed. Therefore, directly splicing the each frame of point clouds without any processing would have been inappropriate. To construct the point cloud map for the entire scene, the method proposed in [24] was employed to register and align each frame of point cloud data. This process ultimately produced the complete point cloud map corresponding to the entire scene. After processing the noise points, four sets of point clouds could be obtained, as illustrated in Figure 4a. Each point contains coordinates and reflectivity information.

Based on the actual situation, all points were divided into five categories: crown, trunk, pot, pole, and ground. An example of annotation is shown in Figure 5. The visualization ground truths of the four scenes after labeling are shown in Figure 4b. Table 1 provides additional details about the dataset, encompassing the number and size of point clouds for different categories across scenes, the actual area occupied by each scene, and the varying density ranges of point clouds.

2.2. Related Work

Based on the feature representation and processing flow, methods for semantic segmentation tasks can be categorized into two types: traditional methods and deep learning-based methods. Traditional methods employ manual feature extraction, followed by the use of some algorithms (like support vector machines, random forest, and so on) to assign a label to each point [25,26,27]. Although these methods are simple and fast, their applications in semantic segmentation tasks are limited due to the constraints of manual feature extraction. With the progress in computer science, deep learning-based methods have gained significant attention in recent years. These methods primarily obtain features through neural network models and assign a label to each point in an end-to-end manner. Currently, deep learning-based methods can be broadly classified into three categories: (1) image-based methods, (2) voxel-based methods, and (3) point-based methods. In this section, we will provide some detailed descriptions on these three approaches.

2.2.1. Image-Based Methods

Point clouds, unlike images, are unstructured data and their corresponding features cannot be directly extracted using two-dimensional convolution operations. In order to solve the above problem, image-based methods typically involve projecting point clouds from different angles or positions onto multiple images. Subsequently, image-based neural network models are employed to extract features from the projected images. These features are then reprojected onto three-dimensional point clouds, enabling the labeling of individual points and achieving accurate semantic segmentation results [28,29]. In [28], the proposed method utilizes a multi-stream network architecture to further investigate the impact of various features, including color and depth information, on the effectiveness of semantic segmentation. Ref. [30] presents a lightweight model, referred to as ECA-MVRes2net. The model employs Res2net within the MVCNN framework to extract point cloud features and integrates an ECA attention mechanism into the group convolution module for optimization. Experimental results demonstrate that the proposed model can quickly achieve effective recognition and detection of crops. Ref. [31] innovatively developed a view Transformer module to fully leverage the information between multiple views, thereby achieving effective fusion of the view information.

While image-based neural network models are relatively mature and yield excellent results in some specific or small-scale point cloud semantic segmentation tasks, they do have some limitations. First of all, selecting appropriate projection angles and positions poses a significant challenge, and the projection operations will limit the effective utilization of local feature information. Furthermore, the projection process disrupts the original geometric relationship of point clouds, resulting in the loss of three-dimensional spatial context feature information.

2.2.2. Voxel-Based Methods

Considering the structured organization of voxels, researchers have proposed voxel-based methods to obtain features from point clouds. The main idea of this kind of method is to convert original point clouds into regular voxel representations, and then use the three-dimensional convolutional neural network models to extract relevant features [32]. Ref. [33] integrates the advantages of neural networks (NNs), trilinear interpolation (TI), and fully connected conditional random fields (FC-CRF) to propose an end-to-end voxel-based model, SEGCloud, designed specifically for semantic segmentation tasks. The VPCNet [34] model is proposed to perform object detection tasks by leveraging the voxel information of point clouds to obtain bounding boxes for the detected objects. To reduce the computational cost for sparse point clouds, some methods [35,36,37] propose strategies accordingly.

However, similar to the image-based methods, voxel-based methods also require converting the original point cloud data into other structures. This conversion process may lead to loss of original data features and important information. Additionally, the size of voxels directly impacts the performances of the models. Smaller voxel values demand extensive computational resources, while larger voxel values may result in inaccurate outcomes. Consequently, the practicality of voxel-based methods, especially for large-scale point cloud data, is relatively low due to significant computation costs involved.

2.2.3. Point-Based Methods

In recent years, to reduce the computational complexity of point cloud data processing and extract features directly from unstructured point clouds as much as possible, researchers have proposed a series of point-based methods [38,39,40,41,42,43]. One of the pioneering methods in this field is PointNet, which uses point clouds as input to the model. It addresses the disorder of point clouds by using the max pooling function for aggregation and incorporates T-Net modules to handle rotation invariance. However, PointNet [38] primarily focuses on the global features of the point clouds and overlooks the importance of local features, which negatively affects classification and segmentation performances. To address this limitation, PointNet++ [39] introduces a hierarchical structure and constructs local regions by ball query. This approach applies the PointNet layer to each local region to extract local features. DGCNN [40] takes a different approach by utilizing the K-nearest neighbors (KNN) algorithm to dynamically divide local regions. It then uses edge convolution operations to obtain local features. These enhanced models have found wide applications in the field of agriculture [44,45,46,47,48].

However, when it comes to handling large-scale point cloud semantic segmentation tasks, these models tend to be relatively inefficient. To address this issue, RandLA-Net [43] has emerged as a classical method. Instead of the commonly used farthest distance sampling method, it uses the random sampling method, which significantly reduces computational costs. Additionally, RandLA-Net incorporates the local spatial encoding technique and dilated residual block to extract local features and expand receptive fields. Experiments show that RandLA-Net has achieved good segmentation results on multiple large-scale point cloud data sets. Currently, RandLA-Net has been applied for the semantic understanding of large-scale agricultural environments [49,50]. In this paper, based on RandLA-Net, we make some improvements to enhance its segmentation performance.

2.3. Main Structure of the Proposed Model

The semantic segmentation model proposed in this paper follows a similar architecture to the U-Net model. The structure of the model is illustrated in Figure 6. In this section, more details will be introduced.

Input: The model takes a set of point clouds $P\in {\mathbb{R}}^{N\times d_{in}}$ as input, where N and $d_{in}$ represent the number and dimensionality of input points, respectively. For simplicity, the batch dimension is omitted. The set of point clouds introduced in Section 2.1 is used to conduct experiments, where each point contains coordinates and reflectivity information. That is, $d_{in}=4$. For the proposed model, the input point clouds are initially fed through a fully connected layer to enable point-wise feature learning. This process expands the dimension of the point cloud from the original 4 to 8.

Encoder: The encoder primarily consists of sampling layers and the feature extraction modules proposed in Section 2.4. Considering the amount of points present in large-scale point clouds, like the RandLA-Net model, the random sampling approach is employed to downsample the points within each sampling layer. Each layer retains either 25% or 50% of the points from the previous layer ($N\to N/\phantom{N4}4\to N/\phantom{N16}16\to N/\phantom{N64}64\to N/\phantom{N256}256\to N/\phantom{N512}512$). Simultaneously, the feature extraction module is continuously utilized to extract high-dimensional features from local regions obtained through the KNN algorithm $(32\to 32\to 128\to 256\to 512\to 1024)$.

Decoder: In the decoder part, the nearest neighboring interpolation method is used to propagate features of original points from sampling points. This involves finding the nearest point in the original point clouds and assigning its features directly to the corresponding sampling point. After that, both the interpolated features from the decoder part and the features from the encoder part are fed into the upsampling features adjustment module introduced in Section 2.6 to adjust features.

Output: The output of the model is the category of each point. The proposed global feature extraction module in Section 2.5 is followed by the decoder to obtain spatial feature information for all points. Finally, two fully connected layers ($(N,32)\to (N,32)\to (N,d_{out})$) are utilized to obtain the prediction score for each point, thereby enabling the accomplishment of the point cloud semantic segmentation task. $d_{out}$ represents the number of output categories. According to the data set introduced in Section 2.1, the value of $d_{out}$ is set as 5.

2.4. Local Feature Extraction Module

Invariance under transformations is one of the notable characteristics of point clouds. This means that certain transformations, such as rotations and translations, can be applied to the entire set of point clouds while it still represents the same object. But for most models without special operations, when input point clouds are disturbed by rotation, their prediction results and performances will be affected.

In reference to [51], the relative positions of neighboring points and the central point around the axis in each local area are taken into account during the process of obtaining the local features. Suppose the coordinates of the center point of the i-th local region $G_i$ is denoted by $p_i=\left(x_i,y_i,z_i\right)$, and the coordinate of the corresponding j-th neighboring point is denoted by $p_{ij}=\left(x_{ij},y_{ij},z_{ij}\right)$. The relative coordinates of the normalized neighboring point can be expressed as ${\widehat{p}}_{ij}=\left({\widehat{x}}_{ij},{\widehat{y}}_{ij},{\widehat{z}}_{ij}\right)$, where ${\widehat{x}}_{ij}=\left(x_i-x_{ij}\right)$, ${\widehat{y}}_{ij}=\left(y_i-y_{ij}\right)$, and ${\widehat{z}}_{ij}=\left(z_i-z_{ij}\right)$. For the RandLA-Net model, the local region features are obtained using the multi-layer perceptron (MLP):

(1)$F_i=MLP\left(concat\left(p_i,p_{ij},{\widehat{p}}_{ij},d_{ij}\right)\right),$

where $d_{ij}$ denotes the distance from the neighboring point to the center point:

(2)$d_{ij}=\sqrt{{\widehat{x}}_{ij}^2+{\widehat{y}}_{ij}^2+{\widehat{z}}_{ij}^2}.$

For the proposed model, the direction angle information formed by the line connecting the center point and the neighboring point around the two axes is added at the local feature extraction module. The obtained angles are shown in Figure 7, which can be expressed as follows:

(3)${\alpha }_{ij}=arctan\left({\widehat{z}}_{ij}/\sqrt{{\widehat{x}}_{ij}^2+{\widehat{y}}_{ij}^2}\right),{\beta }_{ij}=arctan\left({\widehat{y}}_{ij}/\sqrt{{\widehat{x}}_{ij}^2+{\widehat{z}}_{ij}^2}\right),{\gamma }_{ij}=arctan\left({\widehat{x}}_{ij}/\sqrt{{\widehat{y}}_{ij}^2+{\widehat{z}}_{ij}^2}\right).$

The main idea of local features is the differences between neighboring points and the central points within the local areas. Experimental results demonstrate that enhancing the local feature extraction capability of models can effectively increase their segmentation performances. However, most improved models assume that each neighboring point has an equal influence on the center point. Actually, the influence of each neighboring point on the center point can vary due to factors like distance. However, many existing models may overlook this variation in influence. Therefore, adjusting the influence of different neighboring points on the center point is a valid approach to address the aforementioned problem and improve model performance.

Distance is an important factor in determining the closeness of the relationship between two points within the local area. Specifically, the distances between the center point and adjacent points are directly proportional to the significance of the corresponding local information. The local features extracted from neighboring points that are closer to the center point in proximity can effectively represent the information of the current region. In addition to the geometric distance, the relative distances of features also hold significance. Therefore, during the extraction of local features, both geometric distances between the center point and its neighboring points and relative feature distances are considered to calculate the attention coefficient. This allows us to adjust features obtained from different neighboring points accordingly. Similar to PointConv, the local feature extraction module generates the weight function by dealing with the relative coordinates of neighboring points within the local region. This weight function aids in extracting morphological features. More details on the proposed local feature extraction module are introduced in Figure 8.

In Figure 8, for the i-th local region $G_i$, assuming that the corresponding features of the center point are $f_i\in {\mathbb{R}}^{N_L\times C}$, and the corresponding features of the j-th neighbor point are $f_{ij}\in {\mathbb{R}}^{N_L\times C}$ ($j=1\cdots k$). k represents the number of neighbor points, $N_L$ is the total number of generated local areas, and C denotes the dimensionality of the features. Geometric features and relative feature distances can be expressed as ${\widehat{p}}_{ij}$ and ${\widehat{f}}_{ij}=\mathrm{Mean}\left({\displaystyle \sum _{j=1}^k}\left|f_{ij}-f_i\right|\right)$, where Mean is the mean operation and the objective of this operation is to obtain the relative average features of the k-nearest neighbors to the center point in each local region. ${\widehat{p}}_{ij}\in {\mathbb{R}}^{N_L\times 3}$, and ${\widehat{f}}_{ij}\in {\mathbb{R}}^{N_L\times C}$. Two multi-layer perceptrons are utilized to generate different coefficients ${\lambda }_1$ and ${\lambda }_2$ according to ${\widehat{p}}_{ij}$ and ${\widehat{f}}_{ij}$. In localized regions, the relative features of neighboring points in relation to central points provide more meaningful local information compared to pure distance metrics. This is because distance alone fails to capture directional or shape-related information, which are essential local characteristics of point clouds. Therefore, we utilize relative features to mitigate the limitations of distance information. $d_{ij}\in {\mathbb{R}}^{N_L\times 1}$ and ${\widehat{f}}_{ij}$ are adjusted using generated coefficients, and the concatenation operator is applied to obtain $f_{id}$:

(4)${\lambda }_1=MLP\left({\widehat{p}}_{ij}\right),{\lambda }_2=MLP\left({\widehat{f}}_{ij}\right),$

(5)$f_{id}=Concat({\lambda }_1{d}_{ij},{\lambda }_2{\widehat{f}}_{ij}),$

Then, a shared MLP and the softmax operation are used to obtain the corresponding coefficients for local information ${\omega }_i$:

(6)$f_{ij}^m=Concat({\alpha }_{ij},{\beta }_{ij},{\gamma }_{ij},{\widehat{p}}_{ij}),$

(7)${\omega }_i=softmax\left(MLP\right(Concat(f_{id},{\widehat{f}}_{ij},f_{ij}^m\left)\right)).$

Additionally, the shared MLP is employed to generate weights based on the relative coordinates of neighboring points, thereby obtaining coefficients for each point ${\mu }_i$:

(8)${\mu }_i=MLP\left({\widehat{p}}_{ij}\right),$

These coefficients are then multiplied by the obtained weight coefficients and features of neighboring points. Finally, the corresponding local features $F_{G_i}$ can be obtained:

(9)$F_{G_i}={\mu }_i·\sum _{j=1}^k{\omega }_{ij}·f_{ij}^m.$

2.5. Global Feature Extraction Module

Currently, the majority of models focus on enhancing the capability of local feature extraction, while overlooking the significance of global spatial features. It is also important to consider the importance of global spatial features as remote points in space, which may have the same semantic information. However, most models adopt the hierarchical structure, where downsampling operations are employed to sample points from the preceding layers. Therefore, despite the continuous expansion of the receptive field, the final output only remains a part of the point cloud information. In this paper, our goal is to extract global spatial features from all point clouds.

Self-attention is the fundamental component of the Transformer model and has been extensively applied in the field of natural language processing. This module updates the features of each word by calculating the weighted sum and realizes the self-association of information. This mechanism can also be applied to capture relationship information between points. Figure 9a illustrates the architecture of self-attention. Let $F_{input}\in {\mathbb{R}}^{N\times d}$ represent the input features, and N and d denote the number of point in point clouds and the dimension of features, respectively. Then, $F_{input}$ is input into the query matrix and the key matrix to obtain $Q\in {\mathbb{R}}^{N\times d^{\prime }}$ and $K\in {\mathbb{R}}^{N\times d^{\prime }}$. And the softmax operation is applied to obtain the corresponding attention matrix $A\in {\mathbb{R}}^{N\times N}$:

(10)$A=softmax(Q{K}^T/\sqrt{d}).$

Next, $F_{input}$ is multiplied by the value matrix to get $V\in {\mathbb{R}}^{N\times d}$. The result is obtained by multiplying V and A:

(11)$F_{output}=AV.$

To minimize computational complexity, as shown in Figure 9b, the simplified self-attention module obtains its attention matrix directly, without employing query matrix, key matrix, and value matrix processing. To enhance the robustness and representational capacity of the self-attention module, the original architecture incorporates multiple independent attention heads, as shown in Figure 9c. The utilization of multiple distinct query, key, and value matrices enables the original model to obtain more different information from multiple representation spaces. However, one major drawback of self-attention is its high computational complexity ($O\left(d{N}^2\right)$), which arises from the need to compute attention scores for every element relative to all other elements. Especially for large-scale point clouds, the extensive number of points poses a significant challenge to the computer hardware.

In 2021, the concept of external attention was introduced. This module employs two linear layers and two normalization layers to replace the self-attention module. The experiments demonstrate that external attention requires significantly fewer parameters and multiply-accumulate operations (MACs) compared to the standard self-attention and its multiple variables. Furthermore, it effectively improves the feature extraction ability of models and achieves good results in classification and segmentation tasks based on point clouds. The reduction in computational requirements and good performance make the external attention module a more efficient alternative. As shown in Figure 9d, this module contains two external memory units $M_k\in {\mathbb{R}}^{S\times d}$ and $F_{output}=A{M}_v$. And the output is obtained by the following formula:

(12)$F_{output}=Norm\left(F_{input}{M}_k^T\right)M_v,$

where Norm represents double normalization operation, which can be expressed as follows:

(13)${\left(\tilde{\alpha }\right)}_{i,j}=F_{input}{M}_k^T,$

(14)${\widehat{\alpha }}_{i,j}=exp\left({\tilde{\alpha }}_{i,j}\right)/\sum _{i=1}^{k}exp({\tilde{\alpha }}_{k,j}),$

(15)${\alpha }_{i,j}={\widehat{\alpha }}_{i,j}/\sum _{j=1}^k{\widehat{\alpha }}_{i,k}.$

${\alpha }_{i,j}$ denotes the similarity of i-th and j-th point. The computational complexity of external attention is $O\left(dSN\right)$ and the value of parameter S can be adjusted. When dealing with a large number of point clouds, that is, when the value of N is large, the computational complexity of the model using external attention will be significantly lower compared to the self-attention module. Furthermore, multi-head external attention can focus on the different information from unknown representation sub-spaces, as shown in Figure 9e. Therefore, we adopt this module to extract the global features of the whole set of point clouds. The multi-head external attention can be expressed as:

(16)$h_i=ExternalAttention(F_i,M_k,M_v),$

(17)$\begin{array}{cc}\hfill F_{out}& =MultiHead(F_{input},M_k,M_v)\hfill \\ \hfill & =Concat(h_1,\cdots ,h_H)W_0.\hfill \end{array}$

where $h_i$ and H represent the i-th head and number of heads.

2.6. Upsampling Feature Adjustment Module

The U-Net [52] is a U-shaped network architecture primarily used for medical image segmentation tasks. It has also gained popularity in point cloud segmentation tasks due to its consideration of contextual and positional information. The model includes two parts: encoder and decoder. Generally speaking, the encoder includes sampling layers and local feature extraction layers. In the sampling layers, various sampling methods are employed to sample points from the previous layers. The local features are then extracted in the feature extraction layers. The receptive field expands as the number of layers increase, capturing higher-dimensional global information. In the decoder, there are primarily three main operations: upsampling, skip connections, and convolution. The distance-based interpolation methods are utilized to achieve upsampling and feature propagation, enabling the acquisition of features for each individual point in the point clouds. Skip connections concatenate encoder features to the decoder, recovering some lost information caused by downsampling.

In recent years, the U-Net architecture has been enhanced by incorporating various network design concepts, where attention U-Net [53] enables end-to-end learning without significantly increasing the number of parameters and computational requirements. In this paper, we introduce this idea to adjust the features in the decoder using the corresponding features in the encoder and introduce more details in this section.

The majority of U-Net models related to point clouds usually employ the nearest neighbor or weighted average interpolation to propagate point cloud features during the process of upsampling. However, both of the above interpolation methods only take into account the distance relationship between points, while disregarding the semantic label information. Therefore, it is crucial to fuse features from the encoder with their corresponding counterparts in the decoder in order to enrich the information. Presently, the most widely utilized approach is the implementation of skip connections, where features are copied and concatenated. Although this approach is simple and effectively enhances the feature information of interpolation points, it overlooks the discrepancies between encoder and decoder features. To address the above problem, some researchers propose utilizing high-level encoder features, which are derived from the low-level features in the encoder, to adjust and minimize differences between features at the low-level. An example of this strategy is the backward attentive fusing encoder-decoder (BAF-ED) [54] approach. In BAF-ED, high-dimensional features are first converted into an attention map, which is then used to modify the low-dimensional features in the encoder before concatenation. However, when fusing features from both the encoder and decoder, it is generally more appropriate to refine the decoder features by using the corresponding encoder features. In line with this idea, we propose the upsampling features adjustment module, as depicted in Figure 10.

Suppose that the inputs of the i-th encoder and the i-th decoder layer are $F_{ie}\in {\mathbb{R}}^{N_i\times f_{ie}}$ and $F_{id}\in {\mathbb{R}}^{N_i\times f_{id}}$. $N_i$ represents the number of input point clouds, and $f_{ie}$ and $f_{id}$ represent the dimensions of the encoder and decoder features, respectively. To facilitate the subsequent operations, features from the encoder and decoder are aligned through a convolution operation to match their dimensions. Then, the aligned features are added together, as shown in the following formula:

(18)$F_{ie}^{\prime }=Add(F_{id},W_{ie}^T{F}_{ie}+b_{ie}).$

Then processed features $F_{ie}^{\prime }$ undergo further processing to obtain the attention coefficients $\theta$:

(19)$\theta =\sigma (W_i\left(F_{ie}^{\prime }\right)+b_i).$

where $W_{ie}\in {\mathbb{R}}^{f_{ie}\times f_{id}}$ and $W_i\in {\mathbb{R}}^{f_{id}\times 1}$ are linear transformations, $b_{ie}\in {\mathbb{R}}^{F_{id}}$ and $b_i\in \mathbb{R}$ are bias terms, and $\sigma$ is the Sigmoid function. The ReLU operation introduces nonlinearity to enhance the expressive capacity of the model.

For attention coefficients, larger values typically correspond to more important information, whereas smaller values correspond to less relevant or unimportant information. Therefore, multiplying attention coefficients with the $F_{id}$ allows for the extraction of important information. After that, modified features, after being multiplied by the attention coefficients, are concatenated with $F_{ie}$. Then, the final output features $F_{id}^{\prime }$ from the upsampling feature adjustment module will go through a convolution operation:

(20)$F_{id}^{\prime }=W_{ia}^T\left(Concat(F_{ie},\theta F_{id},F_{id})\right)+b_{ia}.$

where $W_{ia}$ and $b_{ia}$ are the linear transformation and bias term of the convolution operation, respectively.

3. Results and Discussion

3.1. Evaluation Metrics and Experimental Equipments

To assess the performances of the models, this paper utilizes several commonly used evaluation metrics for semantic segmentation tasks. Among them, the intersection over union ($IoU$) and mean intersection over union ($mIoU$) are widely employed. $IoU$ serves as a measure to evaluate the degree of overlap between predicted results and the ground truth, with a higher value indicating better segmentation performance. $mIoU$ is derived by computing the average $IoU$ for each category, providing a more comprehensive evaluation of performances across different categories. The above evaluation metrics can be expressed as follows:

(21)$IoU=TP/(FP+TP+FN),$

(22)$mIoU=\mathrm{mean}\left(\sum _{i=1}^{m}Io{U}_i\right).$

where $TP$, $FP$, and $FN$ represent the number of points of true positive, false positive, and false negative. $Io{U}_i$ indicates the value of $IoU$ for the i-th category. m is the number of categories. Unlike classification tasks, semantic segmentation tasks require the completion of point-wise classification for each point of the point clouds, which is more complex. Relying solely on the aforementioned two evaluation indicators is insufficient. Therefore, in this paper, $precision$, $recall$, and $F1$-$scores$ are used as additional evaluation metrics. $Precision$ refers to the ratio between the number of true positive samples and the total number of samples predicted as positive categories. $Recall$, also known as true positive rate, quantifies the ability of model to correctly identify positive instances from the total number of positive samples. And F1-$scores$ is a commonly used evaluation metric that combines both $precision$ and $recall$.

(23)$Precision=TP/(TP+FP),$

(24)$Recall=TP/(TP+FN),$

(25)$F1-scores=(2\times Recall\times Precision)/(Recall+Precision).$

Similar to most of the other relevant work, our models were trained on GPUs. To fully gain access to the required libraries, the Ubuntu operation system rather than Windows was used to conduct all the experiments.

3.2. Experiments on Self-Made Data

This section presents a comparison of the segmentation performance of the proposed model with several commonly used point-based neural network models, including PointNet [38], PointNet++ [39], DGCNN [40], and RandLA-Net [43]. These four typical models are widely used as comparative models in semantic segmentation tasks on both public datasets and self-made datasets [55,56,57,58,59,60]. Therefore, in this paper, we compare the experimental results with these four models. Experiments were conducted using the self-made point cloud set described in Section 2.1. Area1 was utilized as the testing set, while Area2, Area3, and Area4 were used as the training sets. Figure 11 displays the loss and accuracy curves of all models during the training process, indicating that all models eventually reached steady states. Table 2 presents the value of $mIoU$ of all models, as well as the $IoU$ for each category.

According to results shown in Table 2, the proposed method demonstrates superior segmentation performance compared to other models. Specifically, the proposed model achieved a 4.71% improvement in $mIoU$ compared to the RandLA-Net model. Notably, it excelled in segmenting the crown, trunk, pot, and pole categories, with respective $IoU$ values of 98.26%, 80.55%, 86.00%, and 74.92%. Although the segmentation performance for pot and ground categories may be slightly lower than that of the original RandLA-Net, accurately capturing tree-related information holds greater significance for tree monitoring tasks. For instance, precise trunk information facilitates the monitoring of the growth status of trees and aids agricultural robots in performing plant protection tasks, including navigation and spraying operations. Our model surpassed the RandLA-Net model in trunk segmentation, achieving a 6.56% higher $IoU$ score. The number of points associated with the crown, pot, and ground was relatively higher. Consequently, the segmentation accuracy of each model tended to be higher for these three categories. Conversely, the number of corresponding points for the trunk and supporting pole was comparatively lower. As a result, it became challenging for PointNet, PointNet++, DGCNN, and RandLA-Net to accurately segment these two types of point clouds. On the other hand, the proposed model exhibited higher segmentation accuracy specifically for the trunk and supporting pole categories, indicating its effectiveness in accurately predicting and segmenting categories with relatively fewer points. In addition, the results of other semantic segmentation evaluation metrics are presented in Table 3, specifically Precision, Recall, and F1-scores. These metrics provide further insights into the segmentation performance of the proposed model. Table 3 clearly demonstrates that the model proposed in this paper has achieved excellent results across all above indicators.

In order to facilitate a more intuitive comparison of the segmentation performances, the following visualizations are presented (Figure 12a) to depict the ground truth as well as the outputs of both the RandLA-Net and the proposed model. Noteworthy details of the prediction results are highlighted, while the white boxes indicate areas where RandLA-Net had obvious prediction errors. According to visualization results (Figure 12b–e), it can be observed that the RandLA-Net model showed a relatively lower accuracy in distinguishing between point clouds belonging to the trunk and supporting pole categories. This challenge can be attributed to the similarity in shapes between these two categories, making accurate segmentation more difficult. However, the model proposed in this paper addressed this challenge, resulting in improved accuracy when predicting both types of point clouds. Furthermore, RandLA-Net model may misclassify certain points that belong to the pot category as supporting pole, and it may also encounter confusion when dealing with point clouds of some crowns and trunks. In contrast, the model proposed in this study demonstrated higher accuracy in the classification of various categories within the point clouds. Overall, the presented model outperformed RandLA-Net.

3.3. Ablation Experiments

To prove the effectiveness of the proposed local feature extraction (LFE), global feature extraction (GFE), and upsampling feature adjustment (UFA) modules, a series of ablation experiments were conducted. These experiments involved systematically removing each module from the architecture and evaluating the segmentation performances using metrics such as $mIoU$, Precision, Recall, and F1-scores. Similarly, Area1 was utilized as the testing set while the remaining three areas were used as the training sets. Figure 13 shows the training processes of all modified models, where the RandLA-Net model was the base model. The testing results are presented in Table 4 and Table 5.

The results in Table 4 indicate that the modified model with the local feature extraction module exhibited an improvement of 2.38% in terms of $mIoU$, as well as an approximate increase of 1.5% for the other metrics. This shows that the introduction of the local feature extraction module had a positive role in improving the segmentation performance. Based on the aforementioned model, the addition of the global feature extraction module resulted in an increase of 1.19% in $mIoU$, as well as improvements of 1.65% in Precision, 0.35% in Recall, and 0.85% in F1-scores. These improvements substantiate the efficacy of the global feature extraction module. Ultimately, the proposed model, which incorporates all the modules, achieved the highest values in all metrics, indicating its superior segmentation performance.

In addition, we present the Intersection over Union ($IoU$) values for each category, as shown in Table 5. The results indicate that the introduction of the proposed improvement modules led to a continuous enhancement in the recognition accuracy of trunks and supporting poles. Due to their similar external characteristics and relatively few points, distinguishing between these two categories posed a challenge. However, thanks to the continuous optimization of the enhanced model, the system effectively integrated both local and global features of the point clouds. This improvement significantly increased the recognition precision of these two categories, thereby enhancing the overall accuracy of the model. It is noteworthy that the original model had already demonstrated high predictive accuracy for other categories containing larger number of points and the addition of proposed modules did not result in a significant change in segmentation accuracy.

4. Conclusions

This study introduces a semantic segmentation model specifically designed for point clouds of large-scale nurseries to perform semantic segmentation tasks. The generated semantic segmentation results of the nursery are capable of capturing more information about seedlings, including the crown, trunk, and other details, thereby enabling more accurate nursery management. To evaluate the performance of the proposed model, a special dataset was created for conducting semantic segmentation experiments. This dataset consisted of point clouds of seedlings collected from nursery environments, with each point annotated with attributes such as crown, trunk, pot, supporting pole, and ground according to the actual conditions. The experimental results show that the proposed model had outstanding segmentation accuracy, achieving an impressive accuracy rate of 87.18%.

In future research, we aim to utilize the extracted crown, trunk, and other relevant information to help agricultural robots to perform some tasks in real nursery environments such as plant protection, navigation, and other operations. Additionally, we also intend to extend our research to implement large-scale instance segmentation techniques. This will allow us to monitor and manage each tree individually, enhancing the level and efficiency of nursery management.

Author Contributions

Conceptualization, H.L. and J.X.; methodology, J.X.; software, J.X.; validation, J.X.; formal analysis, H.L. and Y.S.; investigation, H.L. and Y.S.; resources, H.L.; data curation, J.X.; writing—original draft preparation, J.X.; writing—review and editing, W.-H.C., H.L., Y.S. and J.K.; visualization, J.X. and H.L.; supervision, H.L.; project administration, H.L. and Y.S.; funding acquisition, H.L. All authors have read and agreed to the published version of the manuscript.

Data Availability Statement

The original data presented in the study are openly available at https://github.com/KID-star (accessed on 27 October 2024).

Acknowledgments

The authors gratefully acknowledge the financial support provided by the National Natural Science Foundation of China General Project: (32171908). We thank Lixuan Xiao and Xuan Zhang for helping to collect the data. We also thank Yuesong Liu for assisting in the completion of the multi-frame point cloud stitching task, and to Xinrong Pu and Huimei Sun for their contributions in completing the point cloud annotation. Finally, we thank the China Scholarship Council (CSC) for providing funding.

Conflicts of Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. Some photos of the experimental data collection site.

Figure 2. The machine used to collect point clouds of seedlings in the nursery.

Figure 3. Some images of collection process. The red line represents the motion trajectory of the robot during the data collection process.

Figure 4. (a) is the point cloud data generated from four scenes within the nursery, with each point containing coordinate and reflectivity information. (b) is the annotated ground truth.

Figure 5. An example of labeled data. Different colors are used to represent different categories.

Figure 6. The main structure of the proposed model for implementing semantic segmentation tasks. Three improved modules are emphasized using dashed rectangular boxes. The local feature extraction module aims to effectively extract features from local areas, while the global feature extraction module is designed to capture the global features. The upsampling feature adjustment module replaces the traditional skip connections to facilitate the efficient fusion of encoder and decoder features.

Figure 7. The relative directional angles between the neighbor points and the center point around the XYZ axes.

Figure 8. The proposed local feature extraction module, where [Forumla omitted. See PDF.] represents the number of generated local areas, and C and [Forumla omitted. See PDF.] are the dimensionality of the input and out features of the proposed local feature extraction module.

Figure 9. (a–c) are the structures of the self-attention module and its variables. (d,e) represent the structures of the external attention and multi-head external attention modules.

Figure 10. The structure of the proposed upsampling features adjustment module.

Figure 11. The accuracy and loss curves of different models with the self-made dataset during the training processes.

Figure 12. Visualization segmentation results on the testing set. (a) represents the ground truth and segmentation results of RandLA-Net and the proposed method. (b–e) show some details of the segmentation results. White boxes indicate that RandLA-Net predicts error points.

Figure 13. The accuracy and loss curves of different models in the ablation experiments.

Table 1

More details about our self-made nursery point cloud dataset.

	Area (m2)	Density Range (Points/m3)	Crown		Trunk		Pot		Pole		Ground
			Num	Size (mb)	Num	Size (mb)	Num	Size (mb)	Num	Size (mb)	Num	Size (mb)
Area1	2988.15	205.91–28,416.14	646,630	35.28	55,541	3.05	80,776	4.48	14,769	0.81	247,169	13.66
Area2	3800.07	310.54–19,874.92	580,791	32.03	2813	0.15	90,859	5.03	16,810	0.93	327,488	18.12
Area3	3245.49	370.50–42,237.41	805,011	44.22	33,695	1.86	86,767	4.82	46,073	2.54	286,790	15.91
Area4	2438.09	359.93–33,114.27	635,954	35.59	25,483	1.44	64,417	3.66	62,105	3.50	221,914	12.55

Table 2

The values of $mIoU$ and $Iou$ of different categories of different models on the self-made dataset.

Method	mIoU	Crown	Trunk	Pot	Pole	Ground
PointNet [38]	43.70%	92.21%	0.87%	41.97%	8.31%	75.13%
PointNet++ [39]	60.05%	95.13%	30.96%	66.07%	18.42%	89.69%
DGCNN [40]	56.50%	95.09%	28.77%	58.38%	14.39%	85.86%
RandLA-Net [43]	82.47%	98.12%	73.99%	85.39%	58.44%	96.43%
Ours	87.18%	98.26%	80.55%	86.00%	74.92%	96.15%

Table 3

The values of Precision, Recall, and F1-scores of different models.

Method	Precision	Recall	F1-Scores
PointNet [38]	65.09%	53.53%	51.59%
PointNet++ [39]	75.21%	69.93%	70.00%
DGCNN [40]	72.42%	66.55%	66.69%
RandLA-Net [43]	87.76%	91.89%	89.63%
Ours	92.49%	93.44%	92.90%

Table 4

The segmentation results of the overall performance of the proposed modules in the ablation experiments.

Method	mIoU	Precision	Recall	F1-Scores
Base	82.47%	87.76%	91.89%	89.63%
Base + LFE	84.85%	89.89%	92.90%	91.33%
Base + LFE + GFE	86.04%	91.54%	93.25%	92.18%
All	87.18%	92.49%	93.44%	92.90%

Table 5

The segmentation results ($IoU$) of the different categories of the proposed modules in the ablation experiments.

Method	Crown	Trunk	Pot	Pole	Ground
Base	98.12%	73.99%	85.39%	58.44%	96.43%
Base + LFE	98.05%	77.58%	85.66%	66.53%	96.44%
Base + LFE + GFE	97.57%	78.61%	86.01%	71.46%	96.54%
All	98.26%	80.55%	86.00%	74.92%	96.15%

References

1. Marco, A.; Charlie, M.S. The potential influence of commercial plant nurseries in shaping the urban forest in South Africa. Urban Urban Green.; 2021; 64, 127254.

2. Justin, M.; Frank, A. La Sorte and Jehane Samaha. A multiscale assessment of the diversity of New Zealand’s nursery trees. Urban Urban Green.; 2022; 68, 127468.

3. Yai, U.A.; Paul, R.F.; Alan, W.H. Economic evaluation of transplant robots for plant cuttings. Sci. Hortic.; 2019; 246, pp. 237-243.

4. Liu, Y.; Wang, C.; Wu, H.; Wei, Y. An Effective LiDAR-Inertial SLAM-Based Map Construction Method for Outdoor Environments. Remote Sens.; 2024; 16, 3099. [DOI: https://dx.doi.org/10.3390/rs16163099]

5. Roberto, O.; Massimo, M.; Paolo, T.; Aldo, C.; Marcello, I.; Emanuele, T.; Marko, H.; Joerg, B.; Julian, P.; Christoph, S. et al. Selective spraying of grapevines for disease control using a modular agricultural robot. Biosyst. Eng.; 2016; 146, pp. 203-215.

6. Fan, Y.; Feng, Z.; Shen, C.; Tauheed, U.K.; Abdul, M.; Gao, X.; Chen, P.; Sajjad, S. A trunk-based SLAM backend for smartphones with online SLAM in large-scale forest inventories. ISPRS J. Photogramm. Remote Sens.; 2020; 162, pp. 41-49. [DOI: https://dx.doi.org/10.1016/j.isprsjprs.2020.02.006]

7. Karam, S.; Lehtola, V.; Vosselman, G. Simple loop closing for continuous 6DOF LIDAR & IMU graph SLAM with planar features for indoor environments. ISPRS J. Photogramm. Remote Sens.; 2021; 181, pp. 413-426.

8. Wu, Y.; Wang, L.; Zhang, L.; Bai, Y.; Cai, Y.; Wang, S.; Li, Y. Improving autonomous detection in dynamic environments with robust monocular thermal SLAM system. ISPRS J. Photogramm. Remote Sens.; 2023; 203, pp. 265-284. [DOI: https://dx.doi.org/10.1016/j.isprsjprs.2023.08.002]

9. Ma, L.; Kong, L.; Peng, X.; Wang, K.; Geng, N. PSTNet: Transformer for aggregating neighborhood features in 3D point cloud semantic segmentation of eggplant plants. Sci. Hortic.; 2024; 331, 113158. [DOI: https://dx.doi.org/10.1016/j.scienta.2024.113158]

10. Cai, D.; Li, R.; Hu, Z.; Lu, J.; Li, S.; Zhao, Y. A comprehensive overview of core modules in visual SLAM framework. Neurocomputing; 2024; 590, 127760. [DOI: https://dx.doi.org/10.1016/j.neucom.2024.127760]

11. Francisco, J.R.; Rafael, M.; Manuel, J.M.J.; Angel, C.P.; Rafael, M.C. ReSLAM: Reusable SLAM with heterogeneous cameras. Neurocomputing; 2024; 563, 126940.

12. Tarsha, K.F.; Lewandowicz, E.; Gharineiat, Z.; Shan, J. Accurate Calculation of Upper Biomass Volume of Single Trees Using Matrixial Representation of LiDAR Data. Remote Sens.; 2024; 16, 2220. [DOI: https://dx.doi.org/10.3390/rs16122220]

13. Guo, C.; Hu, Y.; Qin, J.; Xu, L.; Chu, M.; Wang, H. Image-based estimation of crown volume of individual street trees by plane calculation of angle disparity. Urban Urban Green.; 2023; 86, 128029. [DOI: https://dx.doi.org/10.1016/j.ufug.2023.128029]

14. Xu, J.; Liu, H.; Shen, Y.; Zeng, X.; Zheng, X. Individual nursery trees classification and segmentation using a point cloud-based neural network with dense connection pattern. Sci. Hortic.; 2024; 328, 112945. [DOI: https://dx.doi.org/10.1016/j.scienta.2024.112945]

15. Zhang, L.; Li, M.; Zhu, X.; Chen, Y.; Huang, J.; Wang, Z.; Hu, T.; Wang, Z.; Fang, K. Navigation path recognition between rows of fruit trees based on semantic segmentation. Comput. Electron. Agric.; 2024; 216, 108511. [DOI: https://dx.doi.org/10.1016/j.compag.2023.108511]

16. Tang, H.; Xu, Z.; Chen, S.; Yang, S.; Liu, W.; Zhou, L.; Wang, Y.; Kang, F.; Omayma, I.; Zheng, Y. Light efficacy estimation for fruit trees based on LIDAR point clouds: A case study on pear trees. Sci. Hortic.; 2024; 324, 112590. [DOI: https://dx.doi.org/10.1016/j.scienta.2023.112590]

17. Gong, H.; Sun, Q.; Fang, C.; Sun, L.; Su, R. TreeDetector: Using Deep Learning for the Localization and Reconstruction of Urban Trees from High-Resolution Remote Sensing Images. Remote Sens.; 2024; 16, 524. [DOI: https://dx.doi.org/10.3390/rs16030524]

18. Li, Q.; Xue, Y. Real-time detection of street tree crowns using mobile laser scanning based on pointwise classification. Biosyst. Eng.; 2023; 231, pp. 20-35. [DOI: https://dx.doi.org/10.1016/j.biosystemseng.2023.05.009]

19. Wang, D. Unsupervised semantic and instance segmentation of forest point clouds. ISPRS J. Photogramm. Remote Sens.; 2020; 165, pp. 86-97. [DOI: https://dx.doi.org/10.1016/j.isprsjprs.2020.04.020]

20. Li, Q.; Yan, Y.; Li, W. Coarse-to-fine segmentation of individual street trees from side-view point clouds. Urban Urban Green.; 2023; 89, 128097. [DOI: https://dx.doi.org/10.1016/j.ufug.2023.128097]

21. Gupta, A.; Byrne, J.; Moloney, D.; Watson, S.; Yin, H. Tree Annotations in LiDAR Data Using Point Densities and Convolutional Neural Networks. IEEE Trans. Geosci. Remote Sens.; 2020; 58, pp. 971-981. [DOI: https://dx.doi.org/10.1109/TGRS.2019.2942201]

22. Hamid, H.; Marco, A.C.; Zhang, J. Vertical stratification of forest canopy for segmentation of understory trees within small-footprint airborne LiDAR point clouds. ISPRS J. Photogramm. Remote Sens.; 2017; 130, pp. 385-392.

23. Zeng, L.; Feng, J.; He, L. Semantic segmentation of sparse 3D point cloud based on geometrical features for trellis-structured apple orchard. Biosyst. Eng.; 2020; 196, pp. 46-55. [DOI: https://dx.doi.org/10.1016/j.biosystemseng.2020.05.015]

24. Shen, Y.; Xiao, X.; Liu, H.; Zhang, X. Real-time Localization and Mapping Method for Agricultural Robotin Orchards Based on LiDAR/IMU Tight-coupling. Trans. Chin. Soc. Agric. Mach.; 2023; 54, pp. 20–28+48.

25. Triebel, R.; Kersting, K.; Burgard, W. Robust 3D scan point classification using associative Markov networks. Proceedings of the 2006 IEEE International Conference on Robotics and Automation; Orlando, FL, USA, 15–19 May 2006; 15–19 May pp. 2603-2608.

26. Abhishek, A.; Hema, S.K.; Thorsten, J.; Ashutosh, S. Contextually guided semantic labeling and search for three-dimensional point clouds. Int. J. Rob. Res.; 2013; 32, pp. 19-34.

27. Hackel, T.; Wegner, J.D.; Schindler, K. Fast semantic segmentation of 3D point clouds with strongly varying density. ISPRS Ann.; 2016; 3, pp. 177-184.

28. Lawin, F.J.; Danelljan, M.; Tosteberg, P.; Bhat, G.; Khan, F.S.; Felsberg, M. Deep Projective 3D Semantic Segmentation. Computer Analysis of Images and Patterns; Springer: Berlin/Heidelberg, Germany, 2017; pp. 95-107.

29. Ma, L.; Stückler, J.; Kerl, C.; Cremers, D. Multi-view deep learning for consistent semantic mapping with RGB-D cameras. Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS); Vancouver, BC, Canada, 24–28 September 2017; pp. 598-605.

30. Ma, X.; Xu, W.; Guan, H.; Zhang, X. Three-dimensional image recognition of soybean canopy based on improved multi-view network. Ind. Crops Prod.; 2024; 222, 119544. [DOI: https://dx.doi.org/10.1016/j.indcrop.2024.119544]

31. Jiang, J.; Liu, Z.; Li, J.; Tu, J.; Li, L.; Yao, J. iMVS: Integrating multi-view information on multiple scales for 3D object recognition. J. Vis. Commun. Image Represent.; 2024; 101, 104175. [DOI: https://dx.doi.org/10.1016/j.jvcir.2024.104175]

32. Huang, J.; You, S. Point cloud labeling using 3D Convolutional Neural Network. Proceedings of the 2016 23rd International Conference on Pattern Recognition (ICPR); Cancun, Mexico, 4–8 December 2016; pp. 2670-2675.

33. Tchapmi, L.; Choy, C.; Armeni, I.; Gwak, J.; Silvio, S. SEGCloud: Semantic Segmentation of 3D Point Clouds. Proceedings of the 2017 International Conference on 3D Vision (3DV); Qingdao, China, 10–12 October 2017; pp. 537-547.

34. Zhang, K.; Fan, B.; Wang, K.; Sun, H. VPCNet: Voxel-Point Cascade for 3D Object Detection. Proceedings of the 2022 China Automation Congress (CAC); Xiamen, China, 25–27 November 2022; pp. 6427-6432.

35. Meng, H.Y.; Gao, L.; Lai, Y.K.; Manocha, D. VV-Net: Voxel VAE Net with Group Convolutions for Point Cloud Segmentation. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV); Seoul, Republic of Korea, 27 October–2 November 2019; pp. 8499-8507.

36. Wang, L.; Huang, Y.; Hou, Y.; Zhang, S.; Shan, J. Graph Attention Convolution for Point Cloud Semantic Segmentation. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); Long Beach, CA, USA, 15–20 June 2019; pp. 10288-10297.

37. Zhou, H.; Zhu, X.; Song, X.; Ma, Y.; Wang, Z.; Li, H.; Lin, D. Cylinder3D: An Effective 3D Framework for Driving-scene LiDAR Semantic Segmentation. arXiv; 2020; arXiv: abs/2008.01550

38. Qi, C.R.; Su, H.; Kaichun, M.; Guibas, L.J. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); Honolulu, HI, USA, 21–26 July 2017; pp. 77-85.

39. Qi, C.R.; Yi, L.; Su, H.; Guibas, L.J. PointNet++: Deep hierarchical feature learning on point sets in a metric space. Proceedings of the 2017 the 31st International Conference on Neural Information Processing Systems; Long Beach, CA, USA, 4–9 December 2017; pp. 5105-5114.

40. Wang, Y.; Sun, Y.; Liu, Z.; Sarma, S.E.; Bronstein, M.M.; Solomon, J.M. Dynamic Graph CNN for Learning on Point Clouds. ACM Trans. Graph.; 2019; 38, 146. [DOI: https://dx.doi.org/10.1145/3326362]

41. Li, Y.; Bu, R.; Sun, M.; Wu, W.; Di, X.; Chen, B. PointCNN: Convolution On X-Transformed Points. Adv. Neural Inf. Process. Syst.; 2018; 31, pp. 828-838.

42. Thomas, H.; Qi, C.R.; Deschaud, J.E.; Marcotegui, B.; Goulette, F.; Guibas, L.J. KPConv: Flexible and Deformable Convolution for Point Clouds. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV); Seoul, Republic of Korea, 27 October–2 November 2019; pp. 6410-6419.

43. Hu, Q.; Yang, B.; Xie, L.; Rosa, S.; Guo, Y.; Wang, Z.; Trigoni, N.; Markham, A. RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); Seattle, WA, USA, 13–19 June 2020; pp. 11105-11114.

44. Kaya, T.; Helin, D.; David, R. RoseSegNet: An attention-based deep learning architecture for organ segmentation of plants. Biosyst. Eng.; 2022; 221, pp. 138-153.

45. Hao, H.; Wu, S.; Li, Y.; Wen, W.; Fan, J.; Zhang, Y.; Zhuang, L.; Xu, L.; Li, H.; Guo, X. et al. Automatic acquisition, analysis and wilting measurement of cotton 3D phenotype based on point cloud. Biosyst. Eng.; 2024; 239, pp. 173-189. [DOI: https://dx.doi.org/10.1016/j.biosystemseng.2024.02.010]

46. Hou, Z.; Huang, L.; Zhang, Q.; Miao, Y. Body weight estimation of beef cattle with 3D deep learning model: PointNet++. Comput. Electron. Agric.; 2023; 213, 108184. [DOI: https://dx.doi.org/10.1016/j.compag.2023.108184]

47. Hu, H.; Yu, J.; Yin, L.; Cai, G.; Zhang, S.; Zhang, H. An improved PointNet++ point cloud segmentation model applied to automatic measurement method of pig body size. Comput. Electron. Agric.; 2023; 205, 107560.

48. Frans, P.B.; Eldert, J.V.; Gert, K. The added value of 3D point clouds for digital plant phenotyping – A case study on internode length measurements in cucumber. Biosyst. Eng.; 2023; 234, pp. 1-12.

49. Wei, H.; Xu, E.; Zhang, J.; Meng, Y.; Wei, J.; Dong, Z.; Li, Z. BushNet: Effective semantic segmentation of bush in large-scale point clouds. Comput. Electron. Agric.; 2022; 193, 106653. [DOI: https://dx.doi.org/10.1016/j.compag.2021.106653]

50. Chen, Y.; Xiong, Y.; Zhang, B.; Zhou, J.; Zhang, Q. 3D point cloud semantic segmentation toward large-scale unstructured agricultural scene classification. Comput. Electron. Agric.; 2021; 190, 106445. [DOI: https://dx.doi.org/10.1016/j.compag.2021.106445]

51. Li, W.; Zhan, L.; Min, W.; Zou, Y.; Huang, Z.; Wen, C. BushNet: Effective Semantic Segmentation of Point Cloud with Novel Neural Radiation Field Convolution. IEEE Geosci. Remote. Sens. Lett.; 2023; 20, pp. 1-5.

52. Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015; Munich, Germany, 5–9 October 2015; pp. 234-241.

53. Huang, H.; Lin, L.; Tong, R.; Hu, H.; Zhang, Q.; Iwamoto, Y.; Han, X.; Chen, Y.; Wu, J. Attention U-Net: Learning Where to Look for the Pancreas. arXiv; 2018; arXiv: 1804.03999

54. Shuai, H.; Xu, X.; Liu, Q. Backward Attentive Fusing Network with Local Aggregation Classifier for 3D Point Cloud Semantic Segmentation. IEEE Trans. Image Process.; 2021; 30, pp. 4973-4984. [DOI: https://dx.doi.org/10.1109/TIP.2021.3073660] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33881995]

55. Han, J.; Liu, K.; Li, W.; Chen, G.; Wang, W.; Zhang, F. A Large-Scale Network Construction and Lightweighting Method for Point Cloud Semantic Segmentation. IEEE Trans. Image Process.; 2024; 33, pp. 2004-2017. [DOI: https://dx.doi.org/10.1109/TIP.2024.3372446] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/38451762]

56. Luo, J.; Zhang, D.; Luo, L.; Yi, T. PointResNet: A grape bunches point cloud semantic segmentation model based on feature enhancement and improved PointNet++. Comput. Electron. Agric.; 2024; 224, 109132. [DOI: https://dx.doi.org/10.1016/j.compag.2024.109132]

57. Li, L.; Chen, J.; Su, X.; Han, H.; Fan, C. Deep learning network for indoor point cloud semantic segmentation with transferability. Autom. Constr.; 2024; 168, 105806. [DOI: https://dx.doi.org/10.1016/j.autcon.2024.105806]

58. Zhang, M.; Kadam, P.; Liu, S.; Kuo, C.-C.J. GSIP: Green Semantic Segmentation of Large-Scale Indoor Point Clouds. Pattern Recognit. Lett.; 2024; 164, pp. 9-15. [DOI: https://dx.doi.org/10.1016/j.patrec.2022.10.014]

59. Duan, Y.; Wu, D.; LMeng, M.Y.; Zhu, J.; Zhang, J.; Firkat, E.; Liu, H.; Wei, H. LESA-Net: Semantic segmentation of multi-type road point clouds in complex agroforestry environment. Heliyon; 2024; 10, e36814. [DOI: https://dx.doi.org/10.1016/j.heliyon.2024.e36814]

60. Yin, C.; Wang, B.; Gan, V.J.L.; Wang, M.; Cheng, J.C.P. Automated semantic segmentation of industrial point clouds using ResPointNet++. Autom. Constr.; 2021; 130, 103874. [DOI: https://dx.doi.org/10.1016/j.autcon.2021.103874]