Deep Learning-Based Point Cloud Classification of Obstacles for Intelligent Vehicles

Abstract

Intelligent driving research has focused much attention on point cloud obstacles since they are a class of high-dimensional data that can adequately depict the shape and placement of obstacles, unlike picture data. Currently, deep learning technology is primarily employed for vehicle autonomy point cloud obstacle classification tasks. These techniques typically struggle with low classification accuracy, processing efficiency, and model stability. To tackle the abovementioned issues, this paper suggests a novel random forest algorithm that integrates the out-of-bag error theory and can consistently and accurately evaluate the influence of point cloud properties. Then, building on the novel algorithm, this paper suggests a modified PointNet network that incorporates the effects of both global and local features on the classification task, therefore increasing the conventional network’s classification accuracy. To assess the effectiveness of this novel approach in the experimental portion, we set up an evaluation system based on the metrics for average accuracy, overall accuracy, and a confusion matrix. According to the simulation results, the overall accuracy of the proposed network in terms of classification accuracy is 94.4% and the average accuracy is 84.9%, which are then compared to the prototype PointNet and its variants. The classification accuracies for the four types of obstacles are 97.6%, 63.6%, 92.5%, and 86.1%. In addition, the proposed method is effective at improving both the computational complexity and stability of the network.

1. Introduction

The exponential increase in urbanization and the corresponding surge in private vehicle ownership rates have given birth to a global issue of traffic congestion and accidents. Several countries have prioritized developing a proficient, sustainable, environmentally friendly, and secure Intelligent Transportation System (ITS) to tackle these concerns [1]. Figure 1 systematically demonstrates the complete hierarchical structure of the ITS.

According to Figure 1, point cloud and image processing technology are crucial for intelligent driving. This technology directly impacts the interaction between vehicles and roads. It indirectly influences the performance of motion planning and trajectory tracking. Hence, investigating the classification technology of road obstacles has immense theoretical significance and practical importance [2,3].

Recent advancements in deep learning technology have proven to be an effective solution for classifying road obstacles [4,5,6]. Depending on the research objects, these techniques can be broadly classified into image-based [5,7,8], such as ImvoteNet in [9] and the Real-Time Fusion Network in [10], and point cloud-based deep learning classification techniques [6,11], such as PointNet in [12] and the Geometry Sharing Network in [13]. However, when compared with images, which usually have regularly arranged pixels, it is more challenging to analyze point clouds with irregular structures [14,15].

With the great success of deep learning methods in image processing, pioneer works have been proposed to design network architecture for point cloud data. Multiple deep-learning models have been suggested [12,16], with PointNet as one of the most prominent networks [4,15,17,18]. PointNet, initially introduced by the researcher Charles R. Qi in 2017, is a robust deep-learning network specializing in processing point cloud data [12]. The network distinguishes itself from other deep learning networks by efficiently mitigating the impact of the point cloud data’s disorganized and inflexible rotational properties on classification accuracy [17,19,20,21]. Since its proposal, several different network versions have been created and utilized to classify point clouds. The most notable of these include PointNet++ proposed in the literature [20], PointMLP in the literature [19], and PointNeXt based on PointNet++ in the literature [22].

Most studies primarily concentrate on accomplishing a highly accurate point cloud classification by optimizing the conventional PointNet or the related variant networks [19,20,21,22,23]. Nevertheless, real-world driving environments constantly change and contain high-dimensional feature information, complicating current research [24,25]. Therefore, when optimizing the conventional PointNet, in addition to considering the improvement of its accuracy, it is essential to consider computational efficiency, stability, robustness, and the ability of the network to handle point cloud data containing high-dimensional features.

Random Forest (RF) is a typical machine-learning algorithm that utilizes decision trees to generate classification rules. It is particularly effective at addressing high-dimensional features with a few training samples and preventing overfitting [26,27,28]. The RF algorithm has been shown to help address classification issues in several domains due to its advantages of excellent classification performance, minimal human intervention, robustness, and rapid computation [28,29]. The differences in the research related to the literature are listed in Table 1.

This research aims to provide an innovative strategy for classifying point cloud obstacles. The strategy is to integrate the strengths of the conventional RF algorithm and PointNet network to build an exceptionally stable and robust lightweight network to achieve the high-accuracy point cloud classification task. The main contributions of this paper are as follows.

• (1). The first contribution of this study is to propose an improved Random Forest algorithm, the Random Forest Out-of-Bag Error (RF-OOB) algorithm, for filtering the point cloud features that are important for classification tasks. This innovative algorithm introduces the theory of out-of-bag error based on the RF algorithm, which can improve the stability of the standard algorithm while retaining the advantages of the RF algorithm in efficiently solving high-dimensional feature information.

• (2). Secondly, we propose a way to introduce the RF-OOB algorithm into the input channel of the traditional PointNet network to screen the features with a high impact on the point cloud classification task in order to improve the accuracy of the point cloud classification task.

• (3). Thirdly, an optimization strategy integrating the RF-OOB algorithm with the converted T-Net is adopted, and an associated network model, named Out-of-Bag PointNet (OOB-PointNet), is developed. This integration aims to achieve stable, accurate, and fast point cloud classifications.

This paper’s organization is as follows. The Section 1 is the Introduction Section; this section is used to introduce the background and the influence of the research. The Section 2 is the Materials and Methods Section; this section is used to expound the principles of the methodology described. The Section 3 is the Theory and Calculation Section; this section is used to describe the database and the evaluation system of the experiments. The Section 4 is the Results and Discussion Section; this section is used to present the experimental results and related analysis. The Section 5 is the Conclusions Section; this section is used to summarize and look ahead using the research in this paper.

2. Materials and Methods

2.1. The Principle of the RF-OOB Algorithm

The RF-OOB algorithm divides the feature filtering process into two steps. In the first step, the algorithm generates the feature decision trees. In the second step, the algorithm focuses on assessing the significance of the features. The subsequent content provides a concise explanation of the procedure and fundamental concepts of the algorithm to generate a feature decision tree.

The metrics used to generate a decision tree are coefficient, information gain, and information entropy. Calculating the $\mathrm{Gini}$ coefficient is more flexible and straightforward than the information gain and information entropy. It generates the decision trees with variety, effectively preventing the model from overfitting and increasing the algorithmic accuracy. Therefore, the $\mathrm{Gini}$ coefficient is used to create the decision trees in this step, calculated as shown in Equation (1). During the decision tree generation stage, for a given dataset $\mathrm{D}$, the number of features to be evaluated determines the maximum number of iterations of the algorithm, which, in turn, determines the total number of the generated decision trees.

In Equation (1), the expression $\mathrm{Gini}\left(\mathrm{D}\right)$ means the prescribed threshold, in which the notation $\mathrm{k}$ and ${\mathrm{P}}_{\mathrm{k}},$ respectively, represent the number of categories and the probability of being classified into category $\mathrm{k}$. Meanwhile, the expression ${\mathrm{Gini}}_{\mathrm{x}}\left(\mathrm{D}\right)$ represents the $\mathrm{Gini}$ coefficient based on the feature $\mathrm{x}$ and the datasets ${\mathrm{D}}_1$ and ${\mathrm{D}}_2$ represent the sub-datasets of the dataset $\mathrm{D}$.

(1)$\left\{\begin{array}{c}\mathrm{G}\mathrm{i}\mathrm{n}\mathrm{i}\left(\mathrm{D}\right)=1-{\displaystyle \sum _{\mathrm{k}=1}^{\mathrm{k}}}{\mathrm{P}}_{\mathrm{k}}^2\\ {\mathrm{Gini}}_{\mathrm{x}}\left(\mathrm{D}\right)={\displaystyle \frac{{\mathrm{D}}_1}{\mathrm{D}}}\mathrm{G}\mathrm{i}\mathrm{n}\mathrm{i}\left({\mathrm{D}}_1\right)+{\displaystyle \frac{{\mathrm{D}}_2}{\mathrm{D}}}\mathrm{G}\mathrm{i}\mathrm{n}\mathrm{i}\left({\mathrm{D}}_2\right)\end{array}\right.$

The iterative process of the algorithm is as follows. The RF-OOB algorithm randomly selects some samples from the dataset $\mathrm{D}$, defined as in-bag data samples. In contrast, the remaining unselected samples are defined as out-of-bag data. Then, the algorithm calculates the $\mathrm{Gini}$ coefficients of all the features based on the in-bag data and selects the feature with the smallest coefficient as the intermediate optimal feature of a decision tree. Following this, the algorithm randomly reselects a subset of samples from the dataset, repeats the above operation, and expands the feature decision tree by selecting the feature with the smallest $\mathrm{Gini}$ coefficient until the coefficient is less than the set threshold $\mathrm{Gini}\left(\mathrm{D}\right)$. Finally, a decision tree is generated. Repeat the above operations for several rounds until the maximum number of iterations is reached. Then, the decision tree clusters are formed. Figure 2 illustrates how the RF-OOB algorithm generates the feature decision trees.

During the feature importance assessment stage, we create an out-of-bag error model to assess the influence that a feature has in decision tree clusters. The model is illustrated in Equation (2).

(2)$OOB\left(x\right)={\displaystyle \frac{\left({\sum }_{i=1}^N\left|errOOB1\left(i\right)-errOOB2\left(i\right)\right|\right)}{N}}$

In Equation (2), the expression $OOB\left(x\right)$ represents the influence value of the feature $x$, the expression $errOOB1\left(i\right)$ represents the proportion of misclassified samples in the out-of-bag data after training the model based on the feature $x$, and the expression $errOOB2\left(i\right)$ means if the feature was perturbed, then this is the corresponding proportion of the misclassified samples. The number $N$ denotes the number of decision trees. We define the amplitude value $\left|errOOB1\left(i\right)-errOOB2\left(i\right)\right|$ to assess the influence value of the features. As the error model shows, if the feature $x$ has a more negligible effect on the classification result, the corresponding change in amplitude is smaller after increasing the perturbation. Conversely, if the feature $x$ has a more significant impact, the change in amplitude will be more remarkable. Figure 3 illustrates the entire schematic diagram of the RF-OOB algorithm.

2.2. The Principle of the OOB-PointNet Network Model

During this session, Figure 4 illustrates the model architecture of the conventional PointNet. The conventional PointNet maintains a typical configuration with a three-channel input that only captures the global spatial characteristics of the point cloud data. However, when facing point cloud obstacles that contain high-dimensional features, the conventional PointNet cannot achieve high-performance classification tasks.

Given the above description, this research proposes an enhancement strategy for PointNet based on the RF-OOB algorithm. The specific improvement procedures are as follows.

• (1). Before the point cloud data are inputted into the network, the RF-OOB algorithm is introduced for a feature importance assessment, in which the top three local features are defined as the features that have an essential impact on point cloud classification.

• (2). This study utilizes the scalability of the conventional PointNet input channels to enhance its capacity to extract local features and balance the global and local features in classification tasks. It expands the network to six input channels. This enables the integration of the three local features selected using Equation (1) with the point cloud 3D spatial coordinate features from the original channels. The combined features are input into the network for the subsequent classification operation.

• (3). Points (1) and (2) address the poor local extraction capability of the conventional PointNet. This point elaborates on the previous two points to discuss this optimized network’s mathematical efficiency and robustness in handling point cloud data containing high-dimensional feature information. During the feature importance evaluation stage, this paper proposes an algorithm to assess the importance of features. The optimized RF-OOB algorithm retains the ability of the RF algorithm to process high-dimensional data. It uses the $\mathrm{Gini}$ coefficient, which is computationally efficient, to generate a decision tree. This simplifies the computational process for colleges and universities. Additionally, the algorithm introduces the out-of-the-bag error as a perturbation, making it resistant to disturbances. During the data processing step, the first mapping layer output of T-Net in the conventional PointNet is extended to a 6 × 6 matrix while retaining the rotational orthogonality capability of the original T-Net for further data processing. Table 2 demonstrates the pseudocode of the RF-OOB algorithm. Figure 5 demonstrates the structural diagram of the OOB-PointNet model.

3. Theory and Calculation

This study’s theoretical implementation and experimental calculations are based on the hardware environment shown in Table 3. To ensure the reliability of the point cloud data, the experimental data were sourced from the KITTI dataset, which was collected in a real-world environment by the Karlsruhe Institute of Technology and Toyota Technological Institute in Chicago [30]. After the preprocessing operations, such as noise reduction and segmentation, are performed on the raw data, a point cloud dataset that meets the experimental requirements is generated. This dataset includes the point clouds of four types of obstacles: vehicles, bicycles, trees, and pedestrians, as illustrated in Figure 6.

Based on this dataset, a total of twenty-four point cloud features are extracted in this study. The features include eight global features, such as length, height, width, and others, and sixteen local features, such as surface normal, surface curvature, and fast point feature histogram. The influence value of the extracted point cloud features is calculated using the proposed RF-OOB algorithm. The calculation results are presented in Figure 7. It is evident that among the local features, the arithmetic mean of the surface curvature (curvature_ave), the minimum value of the surface average in the point cloud cluster (curvature_min), and the maximum value of the surface average in the z-direction (normal_z_max) are significantly more important than the other local features. Therefore, the RF-OOB algorithm identifies the curvature_ave, curvature_min, and normal_z_max as the top three critical local features significantly influencing the classification task. These three features are selected to be fused with the 3D coordinate information from the original channel to form the six-dimensional input features of the OOB-PointNet.

To ensure the rationality and validity of the results, the subsequent experiments consist of three parts.

• (1). In the RF-OOB algorithm effectiveness experiments, the evaluation system is established by relying on the confusion matrix, which includes the evaluation metrics of the Overall Accuracy (OA), the Accuracy of each sample (Accuracy), and the Correctness (Precision). Comparison experiments are carried out using the Support Vector Machine (SVM) algorithm and Decision Tree (DT) algorithm within this evaluation system.

• (2). In the network classification accuracy and arithmetic performance experiments, 200 rounds of network model training are first organized to compare each network’s training loss and training accuracy. Subsequently, evaluation metrics such as the OA, mean classification accuracy (mAcc), training time, and accuracy of each classification are used to verify the classification accuracy of the optimized networks. The calculation of the evaluation metrics in the above experiments is shown in Equation (3). Additionally, in this study, the time statistics experiments of each network classifying each frame of the point cloud are conducted to validate further the arithmetic performance of the proposed network in handling high-dimensional point cloud data.

  (3)$\left\{\begin{array}{c}Accuracy\left(O.A.\right)={\displaystyle \frac{TP+TN}{TP+FP+FN+TN}}\\ Precision={\displaystyle \frac{TP}{TP+FP}}\\ mAcc={\displaystyle \frac{\left(\frac{TP}{\left(TP+FN\right)}+\frac{TN}{\left(FP+TN\right)}\right)}{4}}\end{array}\right.$

• (3). In the network stability and robustness experiments, point cloud datasets with densities of 256, 512, 1024, 2048, and 4096 for each cluster are established while keeping all other conditions constant to validate the stability and robustness of the proposed OOB-PointNet by comparing the classification accuracies of each network under different densities of point cloud data.

4. Results and Discussion

4.1. Results of the Effectiveness Experiment of the RF-OOB Algorithm

Based on the confusion matrix calculation results shown in Figure 8, the experiment results of the effectiveness of the RF-OOB algorithm are presented in Figure 9, Table 4, and Table 5. Figure 9 illustrates the graph of the OA results of the three algorithms. According to Figure 9, the OA rate of the RF-OOB and SVM algorithms exceeds 95%, outperforming the DT algorithm. Therefore, the subsequent analysis will focus on the experimental results of the SVM and RF-OOB algorithms.

Table 4 and Table 5 display the classification accuracy and correctness rate of the three algorithms for each sample, respectively. After combining the experimental results in Table 4 and Table 5, for the category of vehicles, the classification performance of the RF-OOB algorithm is not significantly outstanding when compared with the other two algorithms. For the category of pedestrians, the classification accuracy of the RF-OOB algorithm is lower than that of the SVM algorithm by 1.1%. However, the correct rate is higher than that of the SVM algorithm by 2.8%. For the categories of trees and bicycle obstacles, the RF-OOB performs outstandingly. Its classification accuracy reaches 92.4% and 74%, which is higher than the SVM algorithm by 4.6% and 7.3%. Additionally, its classification correct rate is higher than that of the SVM algorithm by 11.3%.

The RF-OOB algorithm demonstrates slightly better results than the other algorithms, as shown in Table 4 and Table 5. The experiment’s findings support the effectiveness of incorporating the out-of-bag error theory into the conventional RF algorithm. This optimization effectively reduces overfitting in the classification results. In summary, the RF-OOB algorithm offers distinct advantages in evaluating point cloud features and effectively identifies important local features for classification.

4.2. Results on Accuracy and Arithmetic Performance of the OOB-PointNet

In the network effectiveness experiments, the proposed network is firstly trained with PointNet, PointNet++, PointMLP, and PointNeXt in the same experimental environment for 200 epochs of repetitive training to compare the training loss with the training accuracy. The results are shown in Figure 10. It can be observed from Figure 10 that all five networks exhibit a consistent convergence trend after 200 epochs of repeated training. However, OOB-PointNet achieves the lowest loss value and the highest accuracy after convergence.

Based on the experimental results demonstrated by Figure 10, the conventional PointNet, its variant PointNet++, and the lightweight PointMLP are chosen to experiment with the OOB-PointNet under the metrics of OA, mAcc, and training time to fully validate the classification accuracy and arithmetic performance of the proposed OOB-PointNet. Table 6 presents the results of the experiment. Based on the training time in Table 6, Table 7 lists the training time for the four networks in batches of 20 epochs for a total of 200 epochs of training.

Regarding the OA metrics, the OOB-PointNet network performs slightly worse than the PointMLP network by 0.3%. However, it shows a 3.4% and 6.3% improvement over the PointNet++ and conventional PointNet networks, respectively. Regarding the mAcc metrics, the OOB-PointNet performs well, surpassing the conventional PointNet, PointNet++, and PointMLP by 14.3%, 6.7%, and 0.4%, respectively. Regarding the training time metrics, the conventional network requires 22 min to complete training. In contrast, the training time for the PointNet++ network extends up to 3 h, and the training time for the PointMLP network is nearly 2 h. In contrast, the OOB-PointNet proposed in this paper has a larger input data size but requires only 26 min for training. This duration is only 4 min longer than the prototype network and significantly faster than the other two optimized networks.

In addition to the overall categorization accuracy experiments, various obstacle classification accuracy experiments are conducted in this study. The findings of these experiments are shown in Table 8. Regarding vehicle obstacles, the classification accuracy of all four types of networks can be maintained above 96%, among which the OOB-PointNet network has a classification accuracy of 97.6%, which is only 0.1% lower than the highest PointMLP. Regarding bicycle and tree obstacles, the OOB-PointNet network outperforms the other three networks. Regarding pedestrian obstacles, OOB-PointNet outperforms the conventional network by 14.3%. However, it falls behind the PointNet++ and PointMLP networks by 6.6% and 4%, respectively.

It should be noted that none of the four types of networks can highly accurately classify bicycle obstacles. This is due to this study’s obstacles, including stationary and moving bicycles. Additionally, the point clouds of moving bicycles and pedestrians have overlapping areas, which make it more challenging for the networks to differentiate between bicycles and pedestrians. Hence, future research should prioritize enhancing the categorization of bicycle point clouds in both stationary and moving scenarios.

Based on the experimental results presented, it is evident that the proposed network has certain advantages in training time and classification accuracy compared to the other three types of networks. However, in practical applications, it is not adequate to thoroughly validate the arithmetic performance of the network if only the training time is considered. Thus, to further evaluate the computational efficiency of the proposed network in point cloud classification, this study selects thirty-two sets of valid data, each containing ten frames of point cloud data. The time taken by the four networks to classify each frame of the input point cloud is measured, and the results are displayed in Figure 11.

Based on Figure 11 and the analysis of the experimental results in Table 6 and Table 7, it can be concluded that the training period of the proposed OOB-PointNet is moderate. Additionally, the time required to classify each point cloud frame in the classification task is shorter, which is somewhat better than the other networks. This indicates that the OOB-PointNet performs better in regards to classification accuracy and computational efficiency.

4.3. Results on the Stability and Robustness Performance of the OOB-PointNet

Furthermore, the classification performance of the network is influenced by the variation in point cloud density. To assess the stability and robustness of the OOB-PointNet, the experiment uses the OA and mAcc as metrics and plots the accuracy curves of the network for different point cloud densities: 256, 512, 1024, 2048, and 4096 per cluster. The specific results of the experiment are displayed in Figure 12 and summarized in Table 9.

By analyzing the fluctuation amplitude of the vertical coordinate values in Figure 12, it becomes evident that OOB-PointNet’s classification accuracy performance is less affected than the other two networks. Based on the statistics in Table 9, the OOB-PointNet network consistently achieves a classification accuracy of 92% to 94%, even when there are variations in point cloud density. These results substantiate two assertions. Even in a low-density point cloud data environment, the proposed OOB-PointNet network can still accomplish a high degree of accuracy, demonstrating a certain degree of stability. Moreover, despite variations in data density, the OOB-PointNet consistently maintains a high classification performance, presenting a notable degree of robustness.

To comprehensively validate the stability and robustness of the proposed OOB-PointNet, the classification accuracy experiments of the three networks under four types of obstacles are respectively conducted. The experimental results are given in Figure 13, and the results shown in Figure 13 are arranged and presented in Table 10.

According to the curves in Figure 13 and the data in Table 10, the classification accuracy of OOB-PointNet for the four types of obstacles using data with different point cloud densities can be maintained in a smooth interval range. Even when using low-density point cloud data, the classification accuracy does not produce significant changes. It always maintains a range of higher value intervals, which is better than the other two types of networks.

4.4. Summary of the Experimental Results and Discussions

This section summarizes the research results and contributions. After 200 rounds of training, OOB-PointNet shows moderate training peaks and performs well in terms of training loss and accuracy compared to PointNet, PointNet++, PointMLP, and PointNeXt due to the introduction of the RF-OOB algorithm and the effect of local features. It also achieves high accuracy point cloud classification to some extent. In addition, OOB-PointNet can maintain high classification accuracy more smoothly when dealing with point cloud data of different densities. The experiments confirm that the proposed network outperforms the existing PointNet variants to some extent in terms of classification accuracy, computational performance, stability, and robustness. However, the network still has some shortcomings, such as lower accuracy in bicycle obstacle classification, and the training time is still slightly longer than that of the standard PointNet network due to the increased influence of local features on the classification task, which therefore increases the computational complexity of the network.

5. Conclusions

The RF-OOB algorithm and OOB-PointNet have achieved good results. The RF-OOB algorithm effectively identifies the important local features in high-dimensional images while expanding the input channel, and enhances the ability of the original network to extract local features, thus improving the classification accuracy. In addition, the RF-OOB algorithm and T-Net extension reduce computational complexity and improve computational performance. The proposed method has high accuracy and efficiency, and can achieve higher classification accuracy in a shorter time with less computational loss in real environment applications. However, OOB-PointNet needs to be further optimized and improved. According to the above experimental results, the classification accuracy of the existing point cloud classification networks, including the proposed network, for bicycle obstacles needs to be improved, and future research on point cloud obstacle classification should focus on the analysis of bicycle obstacles to improve the OOB-PointNet’s classification accuracy for bicycle-related point cloud data in both static and dynamic environments. In addition, this paper only utilizes the RF algorithm to improve the network, and subsequent research should consider the intelligent element heuristic algorithm, edge detection, and the model-free adaptive algorithm to improve the network model in order to construct a network model with high classification accuracy, a fast classification time, and high stability.

Author Contributions

Conceptualization, Y.X. and M.Z.; methodology, M.Z.; software, M.Z.; validation, Y.X., M.Z. and J.Y.; formal analysis, Y.X.; investigation, Y.X.; resources, Y.X.; data curation, Y.X.; writing—original draft preparation, Y.X.; writing—review and editing, D.W.; visualization, D.W.; supervision, Y.X.; project administration, Y.X.; funding acquisition, Y.X. All authors have read and agreed to the published version of the manuscript.

Data Availability Statement

The original contributions presented in this study are included in the article. Further inquiries can be directed to the corresponding author.

Conflicts of Interest

Author Mengfei Zhou was employed by the company Nanjing Xiaomi East China Headquarters, but no funding is from this company. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as potential conflicts of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. Technology Roadmap for the ITS.

Figure 2. The schematic diagram of the algorithm to generate a decision tree forest.

Figure 3. Schematic diagram of the RF-OOB algorithm.

Figure 4. The model architecture diagram of the conventional PointNet.

Figure 5. The structural model of the optimized OOB-PointNet.

Figure 6. Point cloud of four types of obstacles.

Figure 7. Histogram of feature importance assessment.

Figure 8. Confusion matrix plots for the three classifiers.

Figure 9. Plot of the experimental results of the overall accuracy of classification for the three classifiers.

Figure 10. Experimental diagram of 200 epochs of training for five types of networks.

Figure 11. Diagram of each network categorizing each frame of the point cloud.

Figure 12. The diagram of classification accuracy of the networks for different point cloud densities.

Figure 13. The diagram of classification accuracy of the networks for each type of obstacle for different point cloud densities.

Table 1

Classification table of differences between this method and those in the other literature.

Model	Literature	Advantages	Disadvantages	Applicable Datasets
PointNet	[12]	Simple and efficient, directly processes point clouds	Lacks local feature extraction, struggles with complex point clouds	ModelNet40, ShapeNet, KITTI
PointNet++	[20]	Enhanced local feature extraction, suitable for dense point clouds	High computational complexity, relies on sampling and grouping strategies	ScanNet, S3DIS, KITTI
PointMLP	[19]	Fast inference, fewer parameters, simple structure	Average local feature extraction, limited for complex geometries	ModelNet40, ShapeNet, KITTI
PointNeXt	[22]	Efficient optimization, easily extendable, straightforward parameter tuning	Dependent on parameter tuning, inherits some complexity	ScanNet, Waymo
OOB-PointNet	This research	Strong feature extraction, robust, noise-resistant	Higher complexity, additional overhead for feature importance evaluation	ScanNet, KITTI, Waymo

Table 2

The pseudocode of the RF-OOB algorithm.

Pseudocode: RF-OOB Algorithm

1. Start2. Define function Train_RF_OOB(data, labels, n_trees):3.  forest = []     # Initialize the forest4.  oob_errors = []   # Store OOB errors5.  For i from 0 to n_trees-1:6.   bootstrap_data, oob_data = Bootstrap_Sample(data) 7.   tree = Train_Decision_Tree(bootstrap_data, labels) 8.   forest.append(tree)9.   oob_error = Evaluate_OOB_Error(tree, oob_data) 10.   oob_errors.append(oob_error)11.  End for12.  Return forest, oob_errors13. End function14. Define function Calculate_Feature_Importance(forest, data):15.  feature_importances = [0] * data.feature_count() 16.  For each tree in forest:17.   tree_importance = Compute_Tree_Feature_Importance(tree, data) 18.   feature_importances += tree_importance # Accumulate the scores19.  End for20.  Return Normalize(feature_importances)21. End function22. Define function Select_Top_Features(feature_importances, k):23.  Return Top_k_Indices(feature_importances, k)24. End function25. Define function RF_OOB_Feature_Selection(data, labels, n_trees, k):26.  forest, _ = Train_RF_OOB(data, labels, n_trees) 27.  feature_importances = Calculate_Feature_Importance(forest, data) 28.  Return Select_Top_Features(feature_importances, k) # Select top-k features29. End function37. End

Table 3

Workstation hardware environment.

Operating System	CPU Model	GraphicsConfiguration	Memory Environment
Ubuntu 20.04	Inter® Core™ i9-9900K CPU @ 3.60GHz	GeForce RTX 2080 SUPER	Fixed memory 64 GDynamic Hard Disk 2 T.B.

Table 4

Classification accuracy of three classification algorithms for different obstacles.

	SVM(% Accuracy)	DT(% Accuracy)	RF-OOB(% Accuracy)
Vehicles	98.6%	96.5%	98.2%
Bicycles	66.7%	70.3%	74.0%
Trees	87.8%	87.8%	92.4%
Pedestrians	90.3%	78.4%	89.2%

Table 5

Classification correctness of different classifiers for different obstacles.

	SVM(% Precision)	DT(% Precision)	RF-OOB(% Precision)
Vehicles	97.6%	95.9%	97.2%
Bicycles	72.0%	70.3%	83.3%
Trees	92.0%	80.5%	89.7%
Pedestrians	88.4%	86.9%	91.2%

Table 6

Overall performance of the four network models.

Mold	OA (%)	mAcc (%)	Training TimeAverage (min)
PointNet	88.1%	70.6%	22
PointNet++	91.0%	78.2%	194
PointMLP	94.7%	84.5%	111
OOB-PointNet	94.4%	84.9%	26

Table 7

The comparison of network training times in terms of per-batch metrics.

	Mold	PointNet	PointNet++	PointMLP	OOB-PointNet
Epochs
1~20		6.52	58.94	33.82	7.30
21~40		6.52	57.78	33.10	7.29
41~60		6.53	57.86	33.21	7.28
61~80		6.59	58.02	33.26	7.27
81~100		6.53	57.80	33.98	7.25
101~120		6.52	57.79	33.09	7.26
121~140		6.52	60.14	33.65	7.28
141~160		6.57	60.64	33.88	7.28
161~180		6.52	60.64	33.98	7.31
181~200		6.55	60.62	33.01	7.24

Table 8

Classification accuracy of different obstacle types.

	Vehicles(Acc%)	Bicycles(Acc%)	Trees(Acc%)	Pedestrian(Acc%)
PointNet	96.9%	37.5%	76.4%	71.8%
PointNet++	97.4%	42.2%	80.5%	92.7%
PointMLP	97.7%	62.2%	87.9%	90.1%
OOB-PointNet	97.6%	63.6%	92.5%	86.1%

Table 9

The table of classification accuracy of the networks for different point cloud densities.

		Number	256	512	1024	2048	4096	Average
	Mold
OverallAccuracy(OA)	PointNet		85.54%	85.46%	88.05%	88.68%	89.60%	87.47%
	PointNet++		87.10%	90.39%	91.01%	95.31%	95.11%	91.78%
	PointMLP		90.30%	93.51%	94.70%	93.75%	95.43%	93.54%
	OOB-PointNet		93.42%	93.12%	93.67%	92.73%	94.37%	93.46%
MeanAccuracy(mAcc)	PointNet		63.15%	62.38%	70.65%	73.57%	72.89%	68.53%
	PointNet++		66.71%	76.31%	78.24%	85.35%	83.55%	78.03%
	PointMLP		72.70%	75.05%	80.60%	79.64%	82.84%	78.17%
	OOB-PointNet		75.70%	73.97%	82.22%	79.90%	81.38%	78.63%

Table 10

The table of classification accuracy of the networks for each type of obstacle for different point cloud densities.

		Number	256	512	1024	2048	4096	Average
	Mold
Vehicles	PointNet		98.95%	95.98%	96.87%	98.60%	94.87%	97.05%
	PointNet++		98.07%	98.02%	97.43%	98.24%	98.66%	98.08%
	PointMLP		97.93%	99.57%	99.50%	98.90%	99.62%	99.10%
	OOB-PointNet		98.72%	98.74%	98.80%	99.30%	99.40%	98.99%
Bicycles	PointNet		13.63%	13.63%	37.89%	38.15%	43.12%	29.28%
	PointNet++		15.83%	41.94%	42.04%	45.23%	52.73%	41.55%
	PointMLP		27.85%	48.72%	53.17%	52.89%	56.78%	47.88%
	OOB-PointNet		46.77%	50.47%	55.12%	56.75%	56.00%	53.22%
Trees	PointNet		78.47%	72.82%	75.16%	78.25%	76.81%	76.30%
	PointNet++		62.54%	73.49%	80.13%	90.57%	87.98%	78.94%
	PointMLP		70.36%	80.67%	83.39%	92.31%	92.46%	83.84%
	OOB-PointNet		86.60%	88.03%	92.64%	96.03%	92.65%	91.19%
Pedestrians	PointNet		61.54%	65.11%	72.24%	78.00%	76.76%	70.73%
	PointNet++		89.08%	86.08%	84.12%	82.09%	84.20%	85.11%
	PointMLP		78.54%	82.84%	85.13%	86.13%	86.99%	83.93%
	OOB-PointNet		80.39%	81.78%	83.14%	83.36%	83.84%	82.50%

References

1. Wang, L.; Deng, X.; Gui, J.; Jiang, P.; Zeng, F.; Wan, S. A review of Urban Air Mobility-enabled Intelligent Transportation Systems: Mechanisms, applications and challenges. J. Syst. Archit.; 2023; 141, 102902. [DOI: https://dx.doi.org/10.1016/j.sysarc.2023.102902]

2. Zhang, H.; Lu, X. Vehicle communication network in intelligent transportation system based on Internet of Things. Comput. Commun.; 2020; 160, pp. 799-806. [DOI: https://dx.doi.org/10.1016/j.comcom.2020.03.041]

3. Ullah, Z.; Al-Turjman, F.; Mostarda, L.; Gagliardi, R. Applications of Artificial Intelligence and Machine learning in smart cities. Comput. Commun.; 2020; 154, pp. 313-323. [DOI: https://dx.doi.org/10.1016/j.comcom.2020.02.069]

4. Li, Y.; Ma, L.; Zhong, Z.; Liu, F.; Chapman, M.A.; Cao, D.; Li, J. Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Review. IEEE Trans. Neural Netw. Learn. Syst.; 2021; 32, pp. 3412-3432. [DOI: https://dx.doi.org/10.1109/TNNLS.2020.3015992]

5. Wu, B.; Chi, X.; Zhao, C.; Zhang, W.; Lu, Y.; Jiang, D. Dynamic path planning for forklift AGV based on smoothing A* and improved DWA hybrid algorithm. Sensors; 2022; 22, 7079. [DOI: https://dx.doi.org/10.3390/s22187079] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36146427]

6. Fernandes, D.; Silva, A.; Névoa, R.; Simões, C.; Gonzalez, D.; Guevara, M.; Novais, P.; Monteiro, J.; Melo-Pinto, P. Point-cloud based 3D object detection and classification methods for self-driving applications: A survey and taxonomy. Inf. Fusion; 2021; 68, pp. 161-191. [DOI: https://dx.doi.org/10.1016/j.inffus.2020.11.002]

7. Jiang, S.; Cao, X.; Li, C.; Zhou, K.; Hu, M. Courier information recognition based on an improved YOLOv8 visual servoing system. Computing; 2025; 107, 44. [DOI: https://dx.doi.org/10.1007/s00607-024-01411-w]

8. Yang, H.; Shi, C.; Chen, Y.; Wang, L. Boosting 3D Object Detection via Object-Focused Image Fusion. arXiv; 2022; [DOI: https://dx.doi.org/10.48550/arXiv.2207.10589] arXiv: 2207.10589

9. Qi, C.R.; Chen, X.; Litany, O.; Guibas, L.J. ImVoteNet: Boosting 3D Object Detection in Point Clouds with Image Votes. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition; Seattle, WA, USA, 13–19 June 2020; pp. 4404-4413.

10. Sun, L.; Yang, K.; Hu, X.; Hu, W.; Wang, K. Real-Time Fusion Network for RGB-D Semantic Segmentation Incorporating Unexpected Obstacle Detection for Road-Driving Images. IEEE Robot. Autom. Lett.; 2020; 5, pp. 5558-5565. [DOI: https://dx.doi.org/10.1109/LRA.2020.3007457]

11. Ji, Y.; Li, S.; Peng, C.; Xu, H.; Cao, R.; Zhang, M. Obstacle detection and recognition in farmland based on fusion point cloud data. Comput. Electron. Agric.; 2021; 189, 106409. [DOI: https://dx.doi.org/10.1016/j.compag.2021.106409]

12. Qi, C.R.; Su, H.; Mo, K.; Guibas, L.J. Pointnet: Deep learning on point sets for 3D classification and segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; Honolulu, HI, USA, 21–26 July 2017; pp. 652-660. Available online: http://openaccess.thecvf.com/content_cvpr_2017/html/Qi_PointNet_Deep_Learning_CVPR_2017_paper.html (accessed on 6 August 2024).

13. Geometry Sharing Network for 3D Point Cloud Classification and Segmentation | Proceedings of the AAAI Conference on Artificial Intelligence, (n.d.). Available online: https://ojs.aaai.org/index.php/AAAI/article/view/6938 (accessed on 17 September 2024).

14. Zhao, Y.; Luo, S.; Huang, X.; Wei, D. A Multi-Sensor 3D Detection Method for Small Objects. World Electr. Veh. J.; 2024; 15, 210. [DOI: https://dx.doi.org/10.3390/wevj15050210]

15. Li, N.; Kähler, O.; Pfeifer, N. A Comparison of Deep Learning Methods for Airborne Lidar Point Clouds Classification. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.; 2021; 14, pp. 6467-6486. [DOI: https://dx.doi.org/10.1109/JSTARS.2021.3091389]

16. Yang, F.; Cao, Y.; Xue, Q.; Jin, S.; Li, X.; Zhang, W. CEDR: Contrastive Embedding Distribution Refinement for 3D point cloud representation. Signal Process. Image Commun.; 2024; 125, 117129. [DOI: https://dx.doi.org/10.1016/j.image.2024.117129]

17. Guo, Y.; Wang, H.; Hu, Q.; Liu, H.; Liu, L.; Bennamoun, M. Deep Learning for 3D Point Clouds: A Survey. IEEE Trans. Pattern Anal. Mach. Intell.; 2021; 43, pp. 4338-4364. [DOI: https://dx.doi.org/10.1109/TPAMI.2020.3005434]

18. Chen, X.; Jiang, K.; Zhu, Y.; Wang, X.; Yun, T. Individual Tree Crown Segmentation Directly from UAV-Borne LiDAR Data Using the PointNet of Deep Learning. Forests; 2021; 12, 131. [DOI: https://dx.doi.org/10.3390/f12020131]

19. Ma, X.; Qin, C.; You, H.; Ran, H.; Fu, Y. Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework. arXiv; 2022; [DOI: https://dx.doi.org/10.48550/arXiv.2202.07123] arXiv: 2202.07123

20. Qi, C.R.; Yi, L.; Su, H.; Guibas, L.J. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. Adv. Neural Inf. Process. Syst.; 2017; 30, Available online: https://proceedings.neurips.cc/paper/2017/hash/d8bf84be3800d12f74d8b05e9b89836f-Abstract.html (accessed on 6 August 2024).

21. Ma, Z.; Dong, Y.; Zi, J.; Xu, F.; Chen, F. Forest-PointNet: A Deep Learning Model for Vertical Structure Segmentation in Complex Forest Scenes. Remote Sens.; 2023; 15, 4793. [DOI: https://dx.doi.org/10.3390/rs15194793]

22. Qian, G.; Li, Y.; Peng, H.; Mai, J.; Hammoud, H.; Elhoseiny, M.; Ghanem, B. PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategies. Adv. Neural Inf. Process. Syst.; 2022; 35, pp. 23192-23204.

23. Fan, X.; Zhang, H.; Zhou, L.; Bian, L.; Jin, X.; Tang, L.; Ge, Y. Evaluating drought stress response of poplar seedlings using a proximal sensing platform via multi-parameter phenotyping and two-stage machine learning. Comput. Electron. Agric.; 2024; 225, 109261. [DOI: https://dx.doi.org/10.1016/j.compag.2024.109261]

24. Ma, L.; Li, Y.; Li, J.; Tan, W.; Yu, Y.; Chapman, M.A. Multi-scale point-wise convolutional neural networks for 3D object segmentation from LiDAR point clouds in large-scale environments. IEEE Trans. Intell. Transp. Syst.; 2019; 22, pp. 821-836. [DOI: https://dx.doi.org/10.1109/TITS.2019.2961060]

25. Zhu, Q.; Fan, L.; Weng, N. Advancements in point cloud data augmentation for deep learning: A survey. Pattern Recognit.; 2024; 153, 110532. [DOI: https://dx.doi.org/10.1016/j.patcog.2024.110532]

26. Breiman, L. Random Forests. Mach. Learn.; 2001; 45, pp. 5-32. [DOI: https://dx.doi.org/10.1023/A:1010933404324]

27. Paul, A.; Mukherjee, D.P.; Das, P.; Gangopadhyay, A.; Chintha, A.R.; Kundu, S. Improved Random Forest for Classification. IEEE Trans. Image Process.; 2018; 27, pp. 4012-4024. [DOI: https://dx.doi.org/10.1109/TIP.2018.2834830] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29993742]

28. Shaik, A.B.; Srinivasan, S. A Brief Survey on Random Forest Ensembles in Classification Model. International Conference on Innovative Computing and Communications; Bhattacharyya, S.; Hassanien, A.E.; Gupta, D.; Khanna, A.; Pan, I. Springer: Singapore, 2019; pp. 253-260. [DOI: https://dx.doi.org/10.1007/978-981-13-2354-6_27]

29. Sheykhmousa, M.; Mahdianpari, M.; Ghanbari, H.; Mohammadimanesh, F.; Ghamisi, P.; Homayouni, S. Support Vector Machine Versus Random Forest for Remote Sensing Image Classification: A Meta-Analysis and Systematic Review. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.; 2020; 13, pp. 6308-6325. [DOI: https://dx.doi.org/10.1109/JSTARS.2020.3026724]

30. Geiger, A.; Lenz, P.; Stiller, C.; Urtasun, R. Vision meets robotics: The KITTI dataset. Int. J. Robot. Res.; 2013; 32, pp. 1231-1237. [DOI: https://dx.doi.org/10.1177/0278364913491297]