1. Introduction
In karst areas, vertical cracks widen due to water erosion, or collapses occur at the tops of underground rivers, often forming surface shapes resembling funnel-shaped sinkholes that act as significant pathways for surface runoff to enter underground [1,2]. Apart from karst regions, various climate zones and soil types worldwide exhibit similar landforms [3,4,5,6]. Due to differences in their formation environments and causes, they are generally referred to as pseudokarst landforms [7,8,9]. Especially in arid and semi-arid environments, the largest and most typical sinkholes develop in loess or derived soils [3,10,11]. In the Loess Plateau of China, a large number of loess sinkholes are also widely distributed and developing [11]. This is mainly attributed to factors such as high sand content, low organic matter content, a loose structure, high porosity, well-developed vertical joints, strong self-weight settlement, low erosion resistance, easy disintegration, the development of preferential flow paths, and serious dry–wet cycles induced by long-term drought and waterlogging alternation [4,12]. Soil pipes are not only a phenomenon of underground soil erosion but also a geological hazard. Many reports indicate that these sinkholes often lead to road collapses, house collapses, farmland destruction, dam pipeline blockages, and railway damage. Under certain conditions, sinkholes not only have a significant impact on the stability of slopes but can also trigger other geological disasters such as mudslides and landslides [4,5,13,14]. Therefore, conducting surveys and the mapping of sinkholes is of great significance for the prevention and control of geological disasters and soil erosion.
In recent decades, some countries have begun to realize the importance of this work. Traditional sinkhole investigations and mapping primarily rely on topographic maps, digital elevation models (DEMs), historical aerial photography, or satellite images [2,15,16,17,18]. In recent years, many researchers have started using remote sensing [19], synthetic aperture radar interferometry (InSAR) [20], photogrammetry [21], and light detection and ranging (LiDAR) [22,23,24,25,26] to detect and identify sinkholes worldwide. For instance, high-resolution remote sensing satellite images and unmanned aerial system (UAS) images can be used for interpreting, cataloging, and mapping sinkholes [19]; InSAR technology can monitor the rate of sinkhole formation or ground subsidence [20]; and airborne LiDAR measurements can be employed for the deep detection of large sinkholes and terrain mapping [23,24]. An increasing amount of high-resolution topographic data is being utilized to identify and detect sinkholes [27,28,29,30]. Due to rugged and densely vegetated terrain, traditional survey methods struggle to achieve a detailed surveying and mapping [28,31]. Notably, LiDAR technology has uncovered more sinkholes in various karst regions [32,33,34]. For instance, using LiDAR data, Zhu [34] discovered three times as many sinkholes compared to those previously identified on topographic maps in the Floyds Fork watershed of Central Kentucky. Additionally, researchers have autonomously extracted karst collapse features based on high-resolution LiDAR DEMs, local profile tree methods, and weighted random forest approaches [35,36]. Currently, most sinkhole detection is carried out in karst environments, where sinkhole diameters and depths usually range from tens to hundreds of meters, making them easy to detect. In contrast, the sinkholes formed in loess regions tend to be smaller, usually measuring several to tens of meters in both diameter and depth [3,4]. The efficacy of advanced technologies in surveying such areas of the Loess Plateau has yet to be validated through field investigations. It is well-established that the Loess Plateau in China boasts the widest and deepest accumulation of loess in the world, resulting in numerous loess sinkholes [11,37]. Research in this domain remains in its infancy [38], particularly regarding the application of advanced laser scanning technologies for foundational surveys and assessments of loess caves, which have been reported infrequently. Therefore, we selected a typical study area in the Northwest Loess Plateau of China to investigate loess sinkholes using an airborne radar UAS.
Machine learning is a branch of artificial intelligence that constructs computer-based systems that automatically improve through training data [39]. Its main advantage is the ability to analyze the complex input–output relationships of physical behaviors using measured data and handle data space patterns on various scales [40]. In recent years, various machine learning models have increasingly been applied to the automatic recognition, assessment, and mapping of sinkholes [27,29,36,41,42]. For instance, Verachtert [43] utilized an LR model to predict the spatial patterns of sinkholes in loess soils within a moist temperate climate. They observed that sinkholes emerge when topographic thresholds dependent on slope and contributing area are met, and that in well-drained regions, an excessive slope and contributing area led to them. Hosseinalizadeh [44] utilized three machine learning algorithms—MDA, flexible discriminant analysis (FDA), SVM, and UAS imagery—to map the susceptibility of pipeline erosion in the loess-covered hilly regions of Northeastern Iran, demonstrating that all three algorithms effectively predict sinkhole susceptibility. Taheri and his colleagues used various Bayesian-based machine learning models to predict the spatial distribution of sinkholes in Iran, selecting the information gain ratio technique based on the regulating factors of sinkholes [41]. On the other hand, Peyman employed SVM, artificial neural network (ANN), and decision tree (DT) models using soil characteristics obtained from Central Iran and ground-penetrating radar (GPR) frequency data to model and forecast the hazards of sinkholes [45]. The current studies mostly use two to three machine learning models to assess sinkhole susceptibility, with little comparison to recent models that have shown better evaluation results within the same spatial framework. This research employed six classic machine learning methods—SVM, LR, CNN, KNN, RF, and GBDT—for the susceptibility modeling and spatial mapping of sinkholes in the study area.
Because there are many factors affecting the development of sinkholes, many scholars have utilized various types of data, such as meteorological, hydrological, topographic, geological, soil type, land use, and soil properties, for their susceptibility assessment [46,47,48,49,50]. However, for small catchments, the data on soil, geology, lithology, and rainfall may not have much spatial heterogeneity on the one hand, and there may be a lack of high-resolution data or a difficulty in obtaining such data on the other hand, all of which restrict the susceptibility mapping of sinkholes. In fact, hydrogeomorphologic processes at a small watershed scale play a crucial role in the development of sinkholes. Therefore, this paper tries to use an easily obtained DEM and its derived hydrogeomorphic factors to quickly predict the sinkhole susceptibility of small watersheds.
2. Study Area
The study area is located in Huining County, on the Loess Plateau in Northwest China. During the Early Jurassic–Mesozoic era, this area was once a vast lake basin. Due to the rapid uplift of the Qinghai–Tibet Plateau and mountain-building movements, the ancient lake basin was uplifted into loess hills and exposed on the surface. Since the Late Pleistocene, strong northwest winds have continuously blown sand and dust particles from the deserts and desert regions of Northwest China and Central Asia to this area, gradually depositing them here and forming the embryonic form of loess hills [11]. The regional geological map (1:200,000) shows that most of the slopes in the study area are covered by Q3 wind-blown loess (Malan loess), while flood or alluvial layers formed by Q4 are mainly distributed in the valley areas. The region is characterized by numerous gullies due to long-term rainfall and water erosion, low vegetation cover, severe soil erosion, a relatively fragile geological environment, intense human engineering activities, and prominent environmental geological issues of geological hazards and soil erosion in the research area.
The selected key investigation area in this study is the Sunjiacha basin in Huining County (Figure 1a,b), with an area of about 2.40 km2, a length of approximately 2960 m, a width of around 1280 m, and an elevation ranging from 1724.10 to 2070.88 m. It has a typical combination of loess ridges and gullies, which are part of the semi-arid temperate monsoon climate. The surface vegetation mainly consists of sparse grasslands, with precipitation concentrated from May to September, totaling about 370 mm annually. Rainfall from a single heavy rain event can account for 96% of the monthly precipitation. The river channel normally experiences no runoff except during heavy or torrential rain periods, when runoff is generated [11]. The small watershed is characterized by numerous loess geological hazards (1194 collapsing sinkholes, 288 landslides, and collapses) with deep channels and severe soil erosion (mainly by gully erosion). Preliminary field surveys and studies revealed a high sinkhole density of up to 497.5 per km2, with a maximum depth of 29.6 m, a maximum diameter of 34 m, and a maximum volume of approximately 19,600 m3. The study area distributes these sinkholes in various locations, including gullies, valley heads, valley terraces, and terraced fields. The imagery and topography obtained from the airborne radar drone in the research area are shown in Figure 1c,d.
3. Materials and Methods
The technical process of this study, as shown in Figure 2, mainly consists of five steps: (i) using the UAS to collect digital orthophoto images and original LiDAR point cloud data of the research area; (ii) preprocessing the data by figuring out how to interpret UAS images to find sinkholes, cleaning up and filtering point clouds to make DEMs, and making terrain factors; (iii) building a model to test the susceptibility of sinkholes and training it with six machine learning models; (iv) mapping the susceptibility of sinkholes using different machine learning models; and (v) confirming and choosing the best model.
3.1. UAS Survey
After identifying the research area, we conducted on-site field investigations to study and further interpret the sinkholes’ morphology. To perform the two flight missions, this study utilized the Feima D2000 unmanned aerial vehicle system equipped with the D-Lidar 2000 laser radar and D-CAM2000 optical camera. For detailed technical parameters of the drone, please visit
3.2. Data Preparation
3.2.1. Sinkholes Inventory
By modeling the photos captured by the UAS optical camera, a drone DOM with a spatial resolution of 6.87 cm was obtained. Subsequently, 1194 sinkholes were interpreted in ArcGIS 10.5, extracting the key morphological parameters of the sinkholes from the acquired point cloud, DEM, and DOM data, including major axis, minor axis, elongation ratio, roundness, perimeter, area, depth, and volume. Among the 1194 sinkholes, parameters such as major axis, minor axis, elongation ratio, roundness, perimeter, and area were successfully extracted, while 809 sinkholes had depth and volume parameters successfully extracted. Sinkholes with a length of less than 1 m were disregarded due to their extremely small size, resulting in insufficient point cloud data acquisition during aerial photography, making depth and volume calculations unfeasible. The interpreted sinkholes ranged from a maximum area of 662.18 m2 to a minimum of 0.03 m2, with an average size of 17.75 m2. The deepest detected sinkhole had a depth of 29.60 m, while the shallowest was 0.42 m, with an average depth of 6.55 m. The statistical analysis of these morphological parameters is presented in Table 1.
3.2.2. LiDAR Point Clouds Generate DEM
We processed the 40 GB of raw point cloud data obtained from the preprocessing in CloudCompare V2, resulting in a more complete terrain point cloud that filtered out noise such as buildings, people, cars, vegetation, towers, and power lines, and generated a 1 m resolution DEM from this terrain point cloud data (Figure 1d).
3.2.3. Dataset Preparation
We loaded the obtained DEM raster data into ArcMap 10.5 and resampled the sinkhole and non-sinkhole areas separately. Specifically, we conducted random sampling in the sinkhole areas by creating random points, and uniform sampling in the non-sinkhole areas using a fishnet grid with 20 m intervals. In the end, we obtained 10,880 sampling points for sinkhole areas and 8762 sampling points for non-sinkhole areas, labeled “1” and “0”, respectively. After preparing the dataset, we divided the 19,642 samples into two subsets for training and testing using a random selection method. We used around 70% of the sampling points (13,749) for training and 30% (5893) for testing.
3.3. Selection of Geomorphic Factors
After obtaining the sinkhole dataset, another crucial step involved identifying the susceptibility factors that influence sinkhole development. However, there is currently no unified guideline for selecting parameters for a sinkhole susceptibility assessment [51,52]. Baker and Shatz [53,54] indicate that the selection process for geological environmental factors is driven by the principle of parsimony. This principle emphasizes the importance of using the minimum necessary elements to effectively explain phenomena. Therefore, through field investigations and a relevant literature analysis [47,49,55,56], we ultimately selected 17 topographic factors as key indicators for assessing the susceptibility of loess sinkholes. These factors included slope, aspect, slope length factor (LS-factor), slope length, relative slope position, vertical distance to the channel network, flow path length, topographic wetness index, total catchment area, valley depth, closed depressions, convergence index, terrain ruggedness index, plan curvature, profile curvature, geomorphons, and morphometric features. Among these, slope, aspect, slope length factor, slope length, topographic wetness index, convergence index, terrain ruggedness index, plan curvature, and profile curvature are commonly used topographic and hydrological factors for assessing sinkhole susceptibility [44,57]. Additionally, we incorporated the geological environmental conditions of the study area and introduced factors that suit regions with minimal spatial variability, such as basin closure, landform type, morphological measurement characteristics, and valley depth. Table 2 presents the descriptions and sources of the geomorphic factors. We calculated the 17 topographic factors based on a LiDAR-derived DEM using the open-source System for Automated Geoscientific Analyses (SAGA,
3.4. Machine Learning Models
When it comes to susceptibility assessment in sinkhole risk analysis, the SVM model is the best statistical and machine learning method for making predictions [71,72]. SVM is a machine learning method based on statistical learning theory, applicable to classification and regression analysis [73]. Its key strength lies in transforming the initial input space into a high-dimensional feature space, estimating an optimal separating hyperplane, and accurately classifying new unknown samples into known categories [74,75]. This method works well for complicated issues involving both linear and nonlinear separations. It uses different kernel functions, such as linear, polynomial, sigmoid activation, and radial basis functions, to separate samples efficiently [76]. In this study, we selected the radial basis kernel function, whose performance can be optimized by adjusting the kernel width [77]. By finely tuning the kernel width parameter within the range of 1 to 10, we can better control the model’s complexity and generalization ability. Additionally, the regularization parameter influences the accuracy of the SVM model, with a reasonable selection within the range of 0.01 to 1.0 aiding in balancing the model’s fitting capability and preventing overfitting. We used the grid search method to look for the best combination of parameters within the given parameter range. We chose the combination that yielded the highest accuracy score in the validation data to construct the SVM model.
In this study, LR, as a multivariate statistical method, aims to construct predictive models of probabilities for specific categories or events, suitable for binary response scenarios such as yes or no, success or failure, and other classification issues. This method plays a significant role in creating susceptibility maps for geological hazards, for instance in assessing the risk of floods [78] and sinkholes [79]. In our research, we consider the presence of sinkholes as the dependent variable, and we predict sinkholes using logistic regression, which offers significant advantages over traditional linear regression models that require the dependent variable to follow a normal distribution [79,80]. Given the complex formation mechanism of sinkholes, which frequently does not conform to normal distribution characteristics, LR models better accommodate the nature of such non-normally distributed data, resulting in more accurate and reliable results in assessing geological hazard risks. Therefore, this study selects logistic regression as the primary analytical tool to effectively evaluate the risk of sinkhole disasters.
A CNN is a type of feedforward neural network architecture based on deep learning widely used in the field of visual image analysis [81]. It is characterized by the use of convolution operations in at least one network layer, replacing traditional matrix multiplication. A typical CNN consists of an input layer, an output layer, and multiple hidden layers, with the hidden layers primarily composed of a series of convolutional layers. These convolutional layers extract features from input data using convolutional operations that are distinct from conventional multiplication or dot product operations. Following the convolutional layers, there are often other types of layers, such as pooling layers, fully connected layers, and normalization layers, collectively forming the hidden layer section. Influenced by activation functions and the final convolution operation, the inputs and outputs of these layers are termed hidden layers [82]. In the training process of the CNN in this study, as the evaluation stage progresses, in order to reduce computational workload and enhance feature invariance, it is necessary to decrease the spatial resolution of the data to reduce the storage capacity of spatial information. Additionally, pooling functions are introduced to reduce the data dimensions while retaining essential feature information. Furthermore, the final convolution layers widely apply the backpropagation algorithm to optimize the network parameters and improve feature extraction accuracy by more precisely adjusting the network weights. This unique network structure and learning mechanism enable the CNN to achieve a high-precision feature extraction and classification performance in visual image analysis.
The KNN algorithm in the classification process does not determine the category of a new instance through an explicit learning process; rather, it relies on the category patterns of its k nearest neighbors to make a decision. This characteristic makes KNN an instance-based learning algorithm that classifies by directly comparing training data [83]. In the KNN model, the selection of the k value, distance measurement, and classification rules are three core elements. During classification, the k value determines how many neighboring pieces of information are considered, and its magnitude significantly affects the classification results. We use distance measurement to calculate the similarity between a new instance and the various instances in the training dataset, using common distance measurement methods such as Euclidean distance and Manhattan distance. In this study, we chose Euclidean distance as the distance measurement method because it yields the minimal distance [84]:
(1)
In this study, and represent the and feature vectors. For the classification rule, the majority voting rule was adopted, which is based on determining the category of a new instance according to the distribution of k nearest neighbors. This classification rule is characterized by objectivity and universality, making it suitable for classification problems in most scenarios.
Random forest, as an ensemble learning method, aims to construct multiple weak classification trees using the bootstrap technique and aggregate the results of these trees to classify unknown samples [85]. In this learning process, each iteration involves selecting predictive variables and reordering data with replacements [86,87]. Through this process, the RF model demonstrates a better ability to avoid overfitting issues and generally exhibits an improved generalization performance. The performance of a random forest is influenced by two key structural parameters. The first parameter is the number of features used in each random tree (mtry); by adjusting this parameter, the exploration of the feature space by each tree can be controlled, thereby affecting the model’s complexity and classification performance. The second parameter is the number of trees in the random forest (ntree); increasing the number of trees usually helps to enhance the model’s stability and accuracy, but it may also increase the computational complexity and training time [88,89]. In our case, the optimal values for mtry and ntree were estimated using the grid search technique [74].
GBDT is an iterative decision tree algorithm that focuses on utilizing the boosting ensemble method. This method was introduced by Friedman [90] to address overfitting issues in decision trees. GBDT uses regression trees as the basic classifiers and employs the gradient descent method in each iteration to build new basic classifiers. Each basic classifier learns the gradient descent direction of the loss function from the previous iteration. When the GBDT model converges, its final prediction result is the weighted sum of all the base classifiers’ outputs. It is noteworthy that, unlike in RF, where each classifier has the same weight, in GBDT, the classifiers have different weights, reflecting their varied contributions to the final prediction result. GBDT demonstrates superior performance compared to RF in solving certain classification problems, although it has limitations such as the inability to parallel train and a high susceptibility to data noise. We built the GBDT model in this study using the Python software package version 2.7.9.
3.5. Validation and Accuracy Assessment
In the evaluation of geological hazard susceptibility, previous researchers often used the ROC to validate the zoning results, evaluating the accuracy of zoning results by calculating the AUC value [91]. The confusion matrix has two axes that show the actual values and the predicted values. The TP (true positive) axis shows the correct prediction of sinkholes, the TN (true negative) axis shows the correct prediction of non-sinkholes, the FP (false positive) axis shows the misclassification of non-sinkholes as sinkholes, and the FN (false negative) axis shows the misclassification of sinkholes as non-sinkholes. We can calculate the overall accuracy, which is the ratio of all the correctly predicted points to the total number of points, and obtain the true positive rate (TPRate) and false positive rate (FPRate) from the confusion matrix:
(2)
(3)
The TPRate represents the ratio of correctly predicted sinkhole points to the total number of sinkhole points, while the FPRate represents the ratio of points incorrectly predicted as non-sinkholes to the total number of non-sinkhole points. When making a comparison using the area under the ROC curve, when AUC ≤ 0.5, it indicates that the predictive ability of the evaluation model is equivalent to or even lower than random guessing, thus the model lacks predictive value; when 0.5 < AUC ≤ 0.7, it indicates a certain level of accuracy in the predictive ability of the evaluation model; when 0.7 < AUC ≤ 0.9, it indicates a good accuracy in the predictive ability of the evaluation model; and when AUC ≥ 0.9, it indicates the high predictive ability of the evaluation model.
4. Results
4.1. Susceptibility Mapping of Different Machine Learning Models
Based on the machine learning models (SVM, LR, CNN, KNN, RF, and GBDT) and 17 terrain factors, we created a sinkhole susceptibility map (Figure 5). We divided each susceptibility map into five susceptibility levels: very low, low, medium, high, and very high. Based on these six susceptibility maps, we generated corresponding frequency distribution histograms. Among these, GBDT closely approximates the actual scenario with an exponential decay frequency distribution histogram. SVM, which has the lowest prediction accuracy, shows a probability distribution graph that approximates a normal distribution and differs significantly from the actual scenario. LR and CNN have very similar probability distribution graphs (resembling the Poisson distribution), and their susceptibility mapping is highly comparable. The probability distribution graphs of KNN and RF closely resemble the actual scenario, although with some differences: the KNN model has higher values in the 0.6–1 range, while RF has lower values in the 0–0.1 range.
Figure 6 shows the actual areas for each susceptibility level of the six models, with the SVM model indicating that low susceptibility has the largest area (0.93 km2), followed by medium susceptibility (0.87 km2), high susceptibility (0.49 km2), lower susceptibility (0.09 km2), and higher susceptibility (0.01 km2). Based on the RF model, the area of lower susceptibility is the largest (0.82 km2), followed by low susceptibility (0.68 km2), medium susceptibility (0.44 km2), high susceptibility (0.30 km2), and higher susceptibility (0.15 km2). In the CNN model, the largest area is in the lower susceptibility category at 0.72 km2, followed by 0.70 km2 (low susceptibility), 0.46 km2 (medium susceptibility), 0.33 km2 (high susceptibility), and 0.18 km2 (higher susceptibility). In the GBDT model, lower susceptibility has the largest area (1.05 km2), followed by low susceptibility (0.52 km2), medium susceptibility (0.35 km2), high susceptibility (0.28 km2), and higher susceptibility (0.19 km2). According to the LR model, the largest area of 0.73 km2 is for the low susceptibility category, followed by 0.70 km2 (lower susceptibility), 0.46 km2 (medium susceptibility), 0.33 km2 (high susceptibility), and 0.19 km2 (higher susceptibility). In the KNN model, the largest area (for lower susceptibility) is 0.76 km2, followed by low susceptibility (0.52 km2), medium susceptibility (0.38 km2), higher susceptibility (0.37 km2), and high susceptibility (0.37 km2). The general trend of area variation from the six models is the same as the frequency distribution histograms. This means that the areas with low and medium susceptibility are generally bigger in each model, while the areas with medium, high, and high susceptibility get smaller over time.
4.2. Verification Comparison of Models in Susceptibility Prediction
As shown in Figure 7, the validation results indicate that the GBDT model has the highest AUC value (0.94), followed by RF with 0.93, KNN with 0.90, CNN with 0.89, LR with 0.88, and SVM with 0.86. Accordingly, this study concludes that the predictive abilities of the GBDT, RF, and KNN models are excellent, with AUC values ≥ 0.9, while the CNN, LR, and SVM models also demonstrate a good predictive accuracy, with AUC values ≥ 0.8. The AUC values of GBDT and RF differ only by 0.01. However, susceptibility mapping shows that GBDT is more precise in identifying small and densely distributed sinkholes. Despite having a high AUC value, susceptibility mapping reveals a poor predictive performance for KNN. This is because the model uses a majority voting rule to predict whether a pixel is a sinkhole, resulting in high capability to distinguish sinkholes from non-sinkholes but a low accuracy in classifying susceptibility levels into five categories. Compared to other machine learning models, the CNN model requires a large amount of watershed data for effective training. However, our collection of watershed data is not as extensive as the terrain factor data, resulting in the CNN model’s relatively low predictive capability. Despite significant differences in feature extraction and training methods between the LR and CNN models, the AUC value of the LR model differs by only 0.01 from the CNN model. The susceptibility mapping and frequency distribution histograms of both models are very similar. The SVM model essentially distinguishes different categories based on decision boundaries. In this study, the complex data structure and issues like noise significantly affect the predictive performance, resulting in a lower prediction accuracy compared to the other five models.
The principle of GBDT involves iteratively training decision trees to minimize the loss function. Each tree attempts to fit the residuals of the predictions made by all previous trees, thereby gradually enhancing the model performance. The predictions made by the GBDT model rely on the aggregation of predictions from all trees, making it particularly sensitive to outliers. Notably, the GBDT and RF models exhibit the highest prediction accuracy in this study, as both are widely used algorithms in ensemble learning that employ multiple decision trees for decision-making. Our research focuses on a smaller dataset that encompasses numerous terrain-influencing factors. There exist complex nonlinear relationships among these factors. Tree-based ensemble learning algorithms, such as GBDT and RF, excel at capturing and managing these intricate relationships. Furthermore, they are less prone to overfitting on small datasets and can achieve a better generalization performance compared to deep learning methods.
4.3. Analysis and Verification of Optimal Susceptibility Mapping
By comparing various assessments, we observed that the GBDT model exhibits the highest predictive accuracy. This model not only performs exceptionally well overall but also demonstrates strong predictive capabilities in identifying sinkholes within different geomorphic units of the study area (Figure 8a). For instance, the lower part of a watershed is where surface water collects, and during heavy rainfall, it often generates surface runoff. In this area, the soil moisture is high, and with water flow erosion, it easily develops bead-like sinkholes (Figure 8b,c). Due to loosening soil and developed fractures, old landslide bodies are susceptible to developing sinkholes under the influence of rainfall and overland flow (Figure 8d). Transitional zones between positive and negative terrain, where localized depression areas are prone to sinkhole formation, are home to headward erosion gullies. Sinkhole incision often causes headward erosion in these regions, which are potential areas for sinkhole development (Figure 8e). The edges of terraced fields develop small sinkholes due to surface runoff erosion (Figure 8f). It is worth mentioning that the GBDT model performs very well in predicting the distribution of sinkholes in areas with dense slopes and channel sinkholes, as well as small, hard-to-identify sinkholes present in old landslides, eroded gully heads, and terraced fields. Through extensive data training, the GBDT model can accurately capture the influence of terrain factors on the distribution of sinkholes and accurately identify areas of dense sinkhole distribution.
Figure 9 shows the proportion of sinkhole areas calculated by the GBDT model at different susceptibility levels. In areas with very high susceptibility, the proportion of sinkhole areas is 69.37%, while the other proportions are 21.59% (high susceptibility), 6.38% (medium susceptibility), 2.48% (low susceptibility), and 0.18% (lower susceptibility), respectively. The total of medium, low, and lower susceptibility does not exceed 10%, while the proportion of sinkhole areas with very high and high susceptibility reaches 90.97%. The GBDT model’s susceptibility assessment in the study area meets the expected standards, predicting loess sinkholes locations as primarily falling into the categories of higher and high susceptibility. Overall, the advantages of the GBDT model are more pronounced in predicting these areas compared to the other methods.
4.4. Evaluation of Model Migration
To further evaluate the transferability of the six machine learning models employed in this study, we conducted a verification at a loess tableland in Beiguo Village, Heyang County, Shaanxi Province (110.2520°E, 35.0248°N). We mapped 779 loess sinkholes in this area through drone imagery. During the model validation phase, we utilized the same hydrological and geomorphological factors as outlined in Table 2. The validation results indicated (Figure 10) that all six machine learning models achieved AUC values exceeding 0.85, with the gradient boosting decision trees (GBDT) model displaying the highest expected accuracy (AUC = 0.93). This suggests that the GBDT model maintains an excellent transfer capability and high prediction accuracy for sinkholes even when applied to new regions. Additionally, the RF, CNN, LR, SVM, and KNN models also demonstrated commendable transferability. We illustrate the GBDT model’s predictions of the geomorphological susceptibility of loess sinkholes in Figure 11. The areas identified as high and very high susceptibility primarily coincide with the tops of old landslides, the landslide body, and erosional gullies, closely matching the actual distribution of sinkholes.
5. Discussion
5.1. Model Comparison
Over the past two decades, advancements in measurement technology and equipment have enabled many countries to establish a comprehensive detection and monitoring system for sinkholes, spanning from space to low altitudes, near-surface, and underground. However, each technology possesses unique advantages and limitations. For instance, some researchers [21,92,93,94] have successfully detected sinkholes using unmanned aerial systems (UASs) or Helikite balloons equipped with various sensors such as optical lenses, LiDAR, and thermal cameras. This technology allows for extensive aerial surveys but has a limited capability for detecting structures deep underground. Additionally, methods combining geomorphological mapping, InSAR, ERT, GPR, drilling, and trenching for sinkhole identification [20,95,96] significantly enhance data accuracy but demand high environmental and operational standards, resulting in relatively high costs. Currently, most sinkhole detection occurs in karst environments, where sinkholes can be quite large. In contrast, sinkholes formed in China’s Loess Plateau typically have smaller diameters and depths. The effectiveness of the aforementioned non-destructive testing technologies and methods still requires validation through field investigations. The Loess Plateau in China features complex soil conditions, numerous ravines, and a fragmented topography, leading to the frequent occurrence of landslides and debris flows. This makes the Loess Plateau a unique geological environment [97,98,99]. Loess sinkholes represent a specific geological hazard in this region, yet research on these areas remains in its infancy [38]. Therefore, we selected a typical small watershed in the Northwest Loess Plateau of China as our study area, aiming to utilize the most commonly employed airborne radar UAS in karst environments to survey and investigate loess sinkholes.
Sensitivity modeling and mapping sinkholes have become essential tools for assessing karst features. We can categorize the sensitivity models of sinkholes into qualitative and quantitative types [100]. Qualitative methods often rely on the expertise and experience of local specialists, such as the Analytic Hierarchy Process (AHP), which may introduce subjective bias [101,102]. In contrast, quantitative methods depend on numerical data and a statistical analysis to interpret phenomena, employing techniques such as statistics, multi-criteria decision-making, and machine learning [101,103,104]. Recently, advancements in machine learning have significantly improved its predictive capabilities, offering more accurate and efficient solutions [105,106]. Consequently, many researchers have begun to utilize data from various sensors and develop algorithms to swiftly extract and analyze sinkhole information. For instance, Hosseinalizadeh [44] predicted sinkhole sensitivity in the loess-covered hilly areas of Northeastern Iran using a drone system and three machine learning algorithms: mixed discriminant analysis (MDA), flexible discriminant analysis (FDA), and support vector machine (SVM). The validation results indicated that the AUC for these three algorithms ranged from 92.45% to 90.32%. Zhu [42] successfully identified 97% of sinkholes in the Bluegrass region of Kentucky by utilizing morphological feature data from LiDAR and applying six machine learning methods—logistic regression, naive Bayes, neural networks, random forests, RUSBoost, and support vector machines—with neural networks performing the best. However, their study relied solely on morphological feature data from sinkholes without incorporating influencing factors. In contrast, this research combines the morphological feature data of sinkholes with 17 terrain factors to identify sinkholes in the study area and classify their disaster sensitivity. While neural networks excel at handling spatial data, their advantages may be surpassed by the ensemble learning strategy of random forests when dealing with more diverse and complex feature sets, such as terrain factors. Additionally, support vector machines and logistic regression exhibit weaker capabilities in managing complex and high-dimensional feature sets. Specifically, SVM requires more computational resources to find the optimal hyperplane as features increase and become more complex, potentially leading to performance degradation. Chen [107] investigated the predictive performance of four machine learning models (a boosted linear model, boosted regression trees, boosted generalized linear model, and deep boosting model) based on 18 factors for mapping pipeline erosion sensitivity in the Zarandieh watershed of Markazi Province, Iran. The deep boosting model achieved an area under the ROC curve of 0.93, demonstrating its capability in mapping pipeline erosion sensitivity. This study employs another classic algorithm of the boosting ensemble method, gradient boosting decision trees (GBDT), achieving an AUC value of 0.94. Notably, the deep boosting model often requires substantial data for training and may encounter issues such as overfitting and complex parameter tuning. In contrast, GBDT tends to train relatively quickly, especially with medium-sized datasets, and experiments indicate that GBDT outperforms the deep boosting model in predicting sinkhole sensitivity. Encouragingly, compared to sinkhole sensitivity mapping results from other regions [42,44,104,107], this study achieves more refined results by selecting key terrain factors for sinkhole sensitivity mapping.
5.2. Assumptions and Limitations
In karst regions, numerous scholars have conducted extensive and productive research. They have made significant advancements in areas such as the formation processes and causes of sinkholes [56], morphological feature analysis [108,109], automatic extraction and identification [36], and sensitivity mapping [44,107]. However, research on sinkholes in loess areas remains scarce. Although some researchers have progressed in classifying them, understanding their distribution patterns, and identifying disaster causes related to loess sinkholes [4,5,11,50,110], overall, these studies are still in their infancy [38]. Therefore, future in-depth research on the automatic identification, sensitivity assessment, morphological characteristics, and formation mechanisms of sinkholes in loess regions holds significant practical value for preventing and managing soil erosion and geological disasters in these areas.
We employed six machine learning algorithms, utilizing data from 1194 sinkholes and 17 terrain factors, to investigate the disaster sensitivity of the Sunjiacha basin sinkholes in Huining County on the Loess Plateau in Northwest China. The results were satisfactory. All the models demonstrated strong predictive capabilities. We also conducted transfer experiments in new watersheds; the models met expectations, indicating excellent transferability. Notably, the gradient boosting decision tree model accurately identified sinkholes with high and very high sensitivity levels, enabling a detailed analysis of sinkhole sensitivity.
However, it is undeniable that our current work has certain limitations. First, the influencing factors used for training are derived from high-resolution DEM data. For larger watersheds, relying solely on topographic data may not fully capture the complex conditions that lead to the formation of sinkholes. This limitation can hinder the generalization ability of the model and restrict the learning capacity and predictive accuracy of the machine learning model. In the future, we will continue to explore and incorporate additional factors, such as soil property, precipitation, and land use, and test and improve the model over a broader area. Second, in terms of identification results, a small number of smaller sinkholes were not detected due to limited data, while some surface depressions closely resembled sinkholes, leading to misclassification by the algorithm. In future work, we will further optimize or enhance the machine learning model to improve the accuracy of sinkhole identification. Nevertheless, this research demonstrates the potential and value of using LiDAR and machine learning models for predicting sinkhole disaster sensitivity in loess environments. It also provides new insights for the sensitivity assessment and mapping of sinkholes in other loess regions, thereby offering a fresh perspective for the early identification, monitoring, and risk assessment of geological hazards.
6. Conclusions
This study utilized UAS LiDAR measurement technology to conduct a detailed investigation of sinkholes in the Sunjiacha basin in Huining County, Northwest Loess Plateau. We obtained and interpreted high-precision, high-resolution images, terrain data, and LiDAR point cloud data to identify 1194 sinkholes. The sinkhole data were resampled, with 70% used as a training dataset and 30% as a test dataset. Six machine learning operational environments were established on the server. The training dataset and 17 terrain factors were inputted into the machine learning models for training, resulting in susceptibility maps for the six models. Sinkhole susceptibility maps were evaluated using the AUC. Our research indicates the following:
(1) High-resolution LiDAR measurement technology effectively penetrates low, sparse vegetation to directly acquire a precise surface elevation, making it suitable for sinkhole investigations. This technology is great at fixing the big errors that come with regular UAS image-based terrain reconstructions. It shows a lot of promise for use in investigating sinkholes;
(2) In the evaluation phase, the AUC values of the six machine learning models were 0.94 (GBDT), 0.93 (RF), 0.9 (KNN), 0.89 (CNN), 0.88 (LR), and 0.86 (SVM). Among these, the GBDT, RF, and KNN models demonstrated excellent predictive capabilities, with AUC values ≥ 0.9, while the CNN, LR, and SVM models also showed a good accuracy in their predictions, with AUC values ≥ 0.8. This indicates that all the models can effectively predict the susceptibility map of sinkholes;
(3) The GBDT model has the best performance in predicting the susceptibility of loess sinkholes in Huining County (AUC = 0.94), especially in predicting potential sinkholes at the head of erosion gullies, the interior of erosion gullies, old landslides, and terraces;
(4) The results of the transferability evaluation of the machine learning models indicate that the six models employed in this study exhibit a good adaptability in new regions (AUC > 0.85). Among all models, the GBDT model shows a superior transfer capability and predictive accuracy (AUC = 0.93).
Conceptualization, S.H. and F.Z.; methodology, S.H., S.L. and Z.J.; software, S.H. and S.L.; validation, L.W., F.Z., N.W., S.W. and X.W.; formal analysis, S.H. and S.L.; investigation, S.H., S.L., Z.J. and S.W.; resources, S.H., F.Z. and X.W.; data curation, S.H.; writing—original draft preparation, S.L., S.H. and Z.J.; writing—review and editing, S.H., S.L. and Z.J.; visualization, S.H. and S.L.; supervision, L.W. and N.W.; project administration, S.H.; funding acquisition, X.W., S.H. and Z.J. All authors have read and agreed to the published version of the manuscript.
The raw data supporting the conclusions of this article will be made available by the authors on request.
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Footnotes
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Enlarge this image.Figure 1. Study area overview: (a) location; (b) regional geological map; (c) digital orthophoto map (DOM) by UAS optical camera; (d) LiDAR-derived DEM.
Enlarge this image.Figure 3. UAS survey and results: (a) route planning; (b,c) Feima D2000 UAS; (d–g) ground control point surveying using handheld RTK; (h) local point clouds acquired by D-LiDAR2000; (i) local DOM acquired by D-CAM2000; (j–l) typical sinkhole photos taken in field. The red circle is the artificial interpretation of the sinkhole polygon in (i–l).
Enlarge this image.Figure 5. Sinkhole susceptibility maps and frequency distribution histograms of grid by six machine learning methods, (a) SVM, (b) LR, (c) CNN, (d) KNN, (e) RF, and (f) GBDT, where mean stands for average and SD stands for standard deviation.
Enlarge this image.Figure 7. Comparison of the area under the ROC curves (AUC) for six models in the validation step.
Enlarge this image.Figure 8. Geomorphological susceptibility mapping of loess sinkholes based on the GBDT model: (a) whole watershed; (b) shallow gully; (c) sub-catchment; (d) old landslide body; (e) the heads of several erosion gullies; (f) terrace.
Enlarge this image.Figure 9. The sinkhole area proportion of different susceptibility grades by the GBDT model.
Enlarge this image.Figure 10. Comparison of area under ROC curve (AUC) of six models in validation area.
Enlarge this image.Figure 11. Geomorphological susceptibility mapping by GBDT model in validation area: (a) location of validation area; (b) susceptibility zoning map by GBDT model.
Main morphometric parameters calculated from the sinkhole inventory.
| Parameter | Maximum | Minimum | Average | Standard Deviation | Sample Quantity |
|---|---|---|---|---|---|
| * Major axis (m) | 34.68 | 0.19 | 3.78 | 4.09 | 1194 |
| * Minor axis (m) | 25.12 | 0.15 | 2.83 | 3.10 | 1194 |
| * Elongation ratio | 7.35 | 1.00 | 1.41 | 0.42 | 1194 |
| * Circularity ratio | 0.98 | 0.33 | 0.89 | 0.08 | 1194 |
| * Perimeter (m) | 104.14 | 0.60 | 10.75 | 11.74 | 1194 |
| * Area (m2) | 662.18 | 0.03 | 17.75 | 46.25 | 1194 |
| ** Depth (m) | 29.60 | 0.42 | 6.55 | 4.51 | 809 |
| ** Volume (m3) | 19601.27 | 0.21 | 334.75 | 1119.00 | 809 |
* represents the parameters extracted from the UAS-DOM; ** represents the parameters extracted and calculated from the LiDAR data. Volume = Area × Depth.
Hydrological and geomorphic factors used in the study.
| Variable | Data Type | Scale | Description | References |
|---|---|---|---|---|
| Slope | Raster | 1 m | Calculate topographic parameters slope | Florinsky [ |
| Aspect | Raster | 1 m | The orientation of the topography or land surface | Florinsky [ |
| LS-Factor | Raster | 1 m | The slope length factor in the soil erosion equation (USLE) is calculated based on the slope and (specific) watershed area | Kinnell [ |
| Slope Length | Raster | 1 m | Calculate the slope (or flow path) length | Florinsky [ |
| Relative Slope Position | Raster | 1 m | The topographic index related to topographic position is calculated from a digital elevation model using an iterative method | Boehner and Selige [ |
| Channel Network Distance | Raster | 1 m | Used to calculate the vertical distance to a channel network base level | Passalacqua et al. [ |
| Flow Path Length | Raster | 1 m | The average flow path length starting from the selected grid cells | Quinn et al. [ |
| Topographic Wetness Index | Raster | 1 m | Utilized to assess the propensity of an area to accumulate water based on its terrain characteristics | Boehner and Selige [ |
| Total Catchment Area | Raster | 1 m | Refine the calculation using the flow accumulation tool | Gruber and Peckham [ |
| Valley Depth | Raster | 1 m | Difference between the elevation and an interpolated ridge level | Skentos [ |
| Closed Depressions | Raster | 1 m | A depression created by the erosion or dissolution of the earth’s surface | Kołodyńska-Gawrysiak et al. [ |
| Convergence Index | Raster | 1 m | Calculates an index of convergence/divergence regarding overland flow | Krawchuk et al. [ |
| Terrain Ruggedness Index | Raster | 1 m | Used to quantify terrain ruggedness and topographic complexity | Riley et al. [ |
| Plan Curvature | Raster | 1 m | The curvature of the Earth’s surface as observed in a horizontal plane | Heerdegen and Beran [ |
| Profile Curvature | Raster | 1 m | The curvature of the Earth’s surface as observed along a line or profile, typically in the direction of maximum slope | Heerdegen and Beran [ |
| Geomorphons | Raster | 1 m | Derived from a digital elevation model using machine vision approaches | Jasiewicz and Stepinski [ |
| Morphometric Features | Raster | 1 m | A multi-scale approach is used to derive data for morphological feature classification by fitting quadratic parameters | Wood [ |
References
1. Kaufmann, G.; Romanov, D. Structure and evolution of collapse sinkholes: Combined interpretation from physico-chemical modelling and geophysical field work. J. Hydrol.; 2016; 540, pp. 688-698. [DOI: https://dx.doi.org/10.1016/j.jhydrol.2016.06.050]
2. Gökkaya, E.; Gutiérrez, F.; Ferk, M.; Görüm, T. Sinkhole development in the Sivas gypsum karst, Turkey. Geomorphology; 2021; 386, 107746. [DOI: https://dx.doi.org/10.1016/j.geomorph.2021.107746]
3. Bernatek-Jakiel, A.; Poesen, J. Subsurface erosion by soil piping: Significance and research needs. Earth-Sci. Rev.; 2018; 185, pp. 1107-1128. [DOI: https://dx.doi.org/10.1016/j.earscirev.2018.08.006]
4. Peng, J.; Sun, P.; Igwe, O. Loess caves, a special kind of geo-hazard on loess plateau, northwestern China. Eng. Geol.; 2018; 236, pp. 79-88. [DOI: https://dx.doi.org/10.1016/j.enggeo.2017.08.012]
5. Hu, S.; Qiu, H.; Wang, N.; Wang, X.; Ma, S.; Yang, D.; Wei, N.; Liu, Z.; Shen, Y.; Cao, M. et al. Movement process, geomorphological changes, and influencing factors of a reactivated loess landslide on the right bank of the middle of the Yellow River, China. Landslides; 2022; 19, pp. 1265-1295. [DOI: https://dx.doi.org/10.1007/s10346-022-01856-0]
6. Kariminejad, N.; Sepehr, A.; Poesen, J.; Hassanli, A. Combining UAV remote sensing and pedological analyses to better understand soil piping erosion. Geoderma; 2023; 429, 116267. [DOI: https://dx.doi.org/10.1016/j.geoderma.2022.116267]
7. Chen, C.K. The loess landform types and their development rules in the southeast of Longdong area. Acta Geogr. Sin.; 1956; 22, pp. 223-231. (In Chinese)
8. Halliday, W.R. Pseudokarst in the 21st century. J. Cave Karst Stud.; 2007; 69, pp. 103-113.
9. Asanidze, L.; Lezhava, Z.; Tsikarishvili, K.; Gaprindashvili, G.; Chikhradze, N. Karst and pseudokarst landscape of Georgia (Caucasus). A short review. Int. Multidiscip. Sci. GeoConf. SGEM; 2021; 21, pp. 59-64.
10. Hosseinalizadeh, M.; Kariminejad, N.; Campetella, G.; Jalalifard, A.; Alinejad, M. Spatial point pattern analysis of piping erosion in loess-derived soils in Golestan Province, Iran. Geoderma; 2018; 328, pp. 20-29. [DOI: https://dx.doi.org/10.1016/j.geoderma.2018.04.029]
11. Hu, S.; Qiu, H.; Wang, N.; Cui, Y.; Wang, J.; Wang, X.; Ma, S.Y.; Yang, D.D.; Cao, M. The influence of loess cave development upon landslides and geomorphologic evolution: A case study from the northwest Loess Plateau, China. Geomorphology; 2020; 359, 107167. [DOI: https://dx.doi.org/10.1016/j.geomorph.2020.107167]
12. Wang, L.; Li, X.A.; Li, L.C.; Hong, B.; Liu, J. Experimental study on the physical modeling of loess tunnel-erosion rate. Bull. Eng. Geol. Environ.; 2019; 78, pp. 5827-5840. [DOI: https://dx.doi.org/10.1007/s10064-019-01495-1]
13. Verachtert, E.; Van Den Eeckhaut, M.; Poesen, J.; Deckers, J. Spatial interaction between collapsed pipes and landslides in hilly regions with loess-derived soils. Earth Surf. Process. Landf.; 2013; 38, pp. 826-835. [DOI: https://dx.doi.org/10.1002/esp.3325]
14. Bernatek-Jakiel, A.; Nadal-Romero, E. Can soil piping impact environment and society? Identifying new research gaps. Earth Surf. Process. Landf.; 2023; 48, pp. 72-86. [DOI: https://dx.doi.org/10.1002/esp.5431]
15. Panno, S.V.; Weibel, C.P.; Li, W. Karst Regions of Illinois. Illinois State Geological Survey. Open File Series No. 1997-2. 1997; Available online: https://www.ideals.illinois.edu/items/44958/bitstreams/133464/data.pdf (accessed on 25 August 2024).
16. Luman, D.E.; Panno, S.V. Mapping palimpsest karst features on the Illinois sinkhole plain using historical aerial photography. Carbonate Evaporite; 2013; 28, pp. 201-214.
17. De Carvalho Júnior, O.A.; Guimarães, R.F.; Montgomery, D.R.; Gillespie, A.R.; Gomes RA, T.; de Souza Martins, É.; Silva, N.C. Karst depression detection using ASTER, ALOS/PRISM and SRTM-derived digital elevation models in the Bambuí Group, Brazil. Remote Sens.; 2014; 6, pp. 330-351. [DOI: https://dx.doi.org/10.3390/rs6010330]
18. Vajedian, S.; Motagh, M. Extracting sinkhole features from time-series of TerraSAR-X/TanDEM-X data. ISPRS J. Photogramm. Remote Sens.; 2019; 150, pp. 274-284. [DOI: https://dx.doi.org/10.1016/j.isprsjprs.2019.02.016]
19. Dou, J.; Li, X.; Yunus, A.P.; Paudel, U.; Chang, K.T.; Zhu, Z.; Pourghasemi, H.R. Automatic detection of sinkhole collapses at finer resolutions using a multi-component remote sensing approach. Nat. Hazards; 2015; 78, pp. 1021-1044. [DOI: https://dx.doi.org/10.1007/s11069-015-1756-0]
20. Talib, O.C.; Shimon, W.; Sarah, K.; Tonian, R. Detection of sinkhole activity in West-Central Florida using InSAR time series observations. Remote Sens. Environ.; 2022; 269, 112793. [DOI: https://dx.doi.org/10.1016/j.rse.2021.112793]
21. Al-Halbouni, D.; Holohan, E.P.; Saberi, L.; Alrshdan, H.; Sawarieh, A.; Closson, D.; Walter, T.R.; Dahm, T. Sinkholes, subsidence and subrosion on the eastern shore of the Dead Sea as revealed by a close-range photogrammetric survey. Geomorphology; 2017; 285, pp. 305-324. [DOI: https://dx.doi.org/10.1016/j.geomorph.2017.02.006]
22. Filin, S.; Baruch, A. Detection of sinkhole hazards using airborne laser scanning data. Photogramm. Eng. Remote Sens.; 2010; 76, pp. 577-587. [DOI: https://dx.doi.org/10.14358/PERS.76.5.577]
23. Launspach, J. Automated Sinkhole Extraction and Morphological Analysis in Northeast Iowa Using High-Resolution LiDAR Data. Master’s Thesis; University of Northern Iowa: Cedar Falls, IA, USA, 2013.
24. Kobal, M.; Bertoncelj, I.; Pirotti, F.; Dakskobler, I.; Kutnar, L. Using lidar data to analyse sinkhole characteristics relevant for understory vegetation under forest cover—Case study of a high karst area in the Dinaric Mountains. PLoS ONE; 2015; 10, e0122070. [DOI: https://dx.doi.org/10.1371/journal.pone.0122070] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/25793871]
25. Zhang, S.; Bogus, S.; Baros, S.; Neville, P.; Dow, R. Karst sinkhole detecting and mapping using airborne LiDAR-A conceptual framework. MATEC Web of Conferences; EDP Sciences: Les Ulis, France, 2019; Volume 271, 02005.
26. Zhang, S.; Bogus, S.M.; Baros, S.V.; Neville, P.R.; Barrett, H.A. Sinkhole Detection and Mapping Using Airborne LiDAR—A Practical Workflow. Tran-SET 2020; American Society of Civil Engineers: Reston, VA, USA, 2020; pp. 100-111.
27. Miao, X.; Qiu, X.; Wu, S.S.; Luo, J.; Gouzie, D.R.; Xie, H. Developing efficient procedures for automated sinkhole extraction from Lidar DEMs. Photogramm. Eng. Remote Sens.; 2013; 79, pp. 545-554. [DOI: https://dx.doi.org/10.14358/PERS.79.6.545]
28. Hofierka, J.; Gallay, M.; Bandura, P.; Šašak, J. Identification of karst sinkholes in a forested karst landscape using airborne laser scanning data and water flow analysis. Geomorphology; 2018; 308, pp. 265-277. [DOI: https://dx.doi.org/10.1016/j.geomorph.2018.02.004]
29. Kim, Y.J.; Nam, B.H.; Youn, H. Sinkhole detection and characterization using LiDAR-derived DEM with logistic regression. Remote Sens.; 2019; 11, 1592. [DOI: https://dx.doi.org/10.3390/rs11131592]
30. Petschko, H.; Zehner, M.; Fischer, P.; Goetz, J. Terrestrial and Airborne Structure from Motion Photogrammetry Applied for Change Detection within a Sinkhole in Thuringia, Germany. Remote Sens.; 2022; 14, 3058. [DOI: https://dx.doi.org/10.3390/rs14133058]
31. Panno, S.V.; Luman, D.E. Characterization of cover-collapse sinkhole morphology on a groundwater basin-wide scale using lidar elevation data: A new conceptual model for sinkhole evolution. Geomorphology; 2018; 318, pp. 1-17. [DOI: https://dx.doi.org/10.1016/j.geomorph.2018.05.013]
32. Rahimi, M.; Alexander, C. Locating sinkholes in LiDAR coverage of a glacio-fluvial karst, Winona County, MN. Full Proceedings of the Thirteenth Multidisciplinary Conference on Sinkholes and the Engineering and Environmental Impacts of Karst; National Cave and Karst Research Institute: Carlsbad, NM, USA, 2013.
33. Wu, Q.; Deng, C.; Chen, Z. Automated delineation of karst sinkholes from LiDAR-derived digital elevation models. Geomorphology; 2016; 266, pp. 1-10. [DOI: https://dx.doi.org/10.1016/j.geomorph.2016.05.006]
34. Zhu, J.; Taylor, T.; Currens, J.; Crawford, M. Improved karst sinkhole mapping in Kentucky using LiDAR techniques: A pilot study in Floyds Fork watershed. J. Cave Karst Stud.; 2014; 76, pp. 207-216. [DOI: https://dx.doi.org/10.4311/2013ES0135]
35. Wu, Q.; Liu, H.; Wang, S.; Yu, B.; Beck, R.; Hinkel, K. A localized contour tree method for deriving geometric and topological properties of complex surface depressions based on high-resolution topographical data. Int. J. Geogr. Inf. Sci.; 2015; 29, pp. 2041-2060. [DOI: https://dx.doi.org/10.1080/13658816.2015.1038719]
36. Zhu, J.; Pierskalla, W.P., Jr. Applying a weighted random forests method to extract karst sinkholes from LiDAR data. J. Hydrol.; 2016; 533, pp. 343-352. [DOI: https://dx.doi.org/10.1016/j.jhydrol.2015.12.012]
37. Jiang, Z.D.; Hu, S.; Deng, H.; Wang, N.L.; Zhang, F.Y.; Wang, L.; Wu, S.B.; Wang, X.G.; Cao, Z.W.; Chen, Y.X. et al. Detection and automatic identification of loess sinkholes from the perspective of LiDAR point clouds and deep learning algorithm. Geomorphology; 2024; 465, pp. 109404-109417. [DOI: https://dx.doi.org/10.1016/j.geomorph.2024.109404]
38. Li, X.A.; Wang, L.; Hong, B.; Li, L.C.; Liu, J.; Lei, H.N. Erosion characteristics of loess tunnels on the Loess Plateau: A field investigation and experimental study. Earth Surf. Process. Landf.; 2020; 45, pp. 1945-1958. [DOI: https://dx.doi.org/10.1002/esp.4857]
39. Jordan, M.I.; Mitchell, T.M. Machine learning: Trends, perspectives, and prospects. Science; 2015; 349, pp. 255-260. [DOI: https://dx.doi.org/10.1126/science.aaa8415] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/26185243]
40. Kanevski, M.; Parkin, R.; Pozdnukhov, A.; Timonin, V.; Maignan, M.; Demyanov, V.; Canu, S. Environmental data mining and modeling based on Machine Learning algorithms and geostatistics. Environ. Model. Softw.; 2004; 19, pp. 845-855. [DOI: https://dx.doi.org/10.1016/j.envsoft.2003.03.004]
41. Taheri, K.; Shahabi, H.; Chapi, K.; Shirzadi, A.; Gutiérrez, F.; Khosravi, K. Sinkhole susceptibility mapping: A comparison between bayes-based machine learning algorithms. Land Degrad. Dev.; 2019; 30, pp. 730-745. [DOI: https://dx.doi.org/10.1002/ldr.3255]
42. Zhu, J.; Nolte, A.M.; Jacobs, N.; Ye, M. Using machine learning to identify karst sinkholes from LiDAR-derived topographic depressions in the Bluegrass Region of Kentucky. J. Hydrol.; 2020; 588, 125049. [DOI: https://dx.doi.org/10.1016/j.jhydrol.2020.125049]
43. Verachtert, E.; Van Den Eeckhaut, M.; Poesen, J.; Govers, G.; Deckers, J. Prediction of spatial patterns ofcollapsed pipes in loess-derived soils in a temperate humid climate using logistic regression. Geomorphology; 2011; 130, pp. 185-196. [DOI: https://dx.doi.org/10.1016/j.geomorph.2011.03.015]
44. Hosseinalizadeh, M.; Kariminejad, N.; Rahmati, O.; Keesstra, S.; Alinejad, M. How can statistical and artificial intelligence approaches predict piping erosion susceptibility?. Sci. Total Environ.; 2019; 646, pp. 1554-1566. [DOI: https://dx.doi.org/10.1016/j.scitotenv.2018.07.396]
45. Peyman, A.; Mohammad, A.G.; Ahmet, R.M.; Ali, D.; Mohammad, A.L. Prediction of sinkhole hazard using artificial intelligence model with soil characteristics and GPR data in arid alluvial land in Central Iran. Environ. Earth Sci.; 2023; 85, 372.
46. Poesen, J. Conditions for gully formation in the Belgian loam belt and some ways to control them. Soil Technol.; 1989; 1, pp. 39-52.
47. Faulkner, H. Badlands inmarl lithologies: A field guide to soil dispersion, subsurface erosion and piping-origin gullies. Catena; 2013; 106, pp. 42-53. [DOI: https://dx.doi.org/10.1016/j.catena.2012.04.005]
48. Wilson, G.V.; Rigby, J.R.; Dabney, S.M. Soil pipe collapses in a loess pasture of Goodwin Creek watershed, Mississippi: Role of soil properties and past land use. Earth Surf. Process. Landf.; 2015; 40, pp. 1448-1463. [DOI: https://dx.doi.org/10.1002/esp.3727]
49. Nasiri Aghdam, I.; Varzandeh, M.H.M.; Pradhan, B. Landslide susceptibility mapping using an ensemble statistical index (Wi) and adaptive neuro-fuzzy inference system (ANFIS) model at Alborz Mountains (Iran). Environ. Earth Sci.; 2016; 75, 553. [DOI: https://dx.doi.org/10.1007/s12665-015-5233-6]
50. Geng, H.P.; Xu, W.Y.; Zheng, W.S.; Gao, X.L.; Pan, B.T. A hybrid mechanism for the initiation and expansion of loess caves across the Chinese Loess Plateau. Land Degrad. Dev.; 2023; 34, pp. 3329-3339. [DOI: https://dx.doi.org/10.1002/ldr.4686]
51. Intrieri, E.; Fontanelli, K.; Bardi, F.; Marini, F.; Fanti, R. Definition of sinkhole triggers and susceptibility based on hydrogeomorphological analyses. Environ. Earth Sci.; 2018; 77, 4. [DOI: https://dx.doi.org/10.1007/s12665-017-7179-3]
52. Wood, N.J.; Doctor, D.H.; Alder, J.; Jones, J. Current and future sinkhole susceptibility in karst and pseudokarst areas of the conterminous United States. Front. Earth Sci.; 2023; 11, 1207689. [DOI: https://dx.doi.org/10.3389/feart.2023.1207689]
53. Baker, D. Quantitative parsimony and explanatory power. Br. J. Philos. Sci.; 2003; 54, pp. 245-259. [DOI: https://dx.doi.org/10.1093/bjps/54.2.245]
54. Shatz, I. Why You Should Prefer Simpler Explanations, Parsimony. 25 June 2019. Available online: https://effectiviology.com/parsimony (accessed on 12 April 2024).
55. Subedi, P.; Subedi, K.; Thapa, B.; Subedi, P. Sinkhole susceptibility mapping in Marion County, Florida: Evaluation and comparison between analytical hierarchy process and logistic regression based approaches. Sci. Rep.; 2019; 9, 7140. [DOI: https://dx.doi.org/10.1038/s41598-019-43705-6]
56. Nam, B.H.; Kim, Y.J.; Youn, H. Identification and quantitative analysis of sinkhole contributing factors in Florida’s Karst. Eng. Geol.; 2020; 271, 105610. [DOI: https://dx.doi.org/10.1016/j.enggeo.2020.105610]
57. Hitouri, S.; Meriame, M.; Ajim, A.S.; Pacheco, Q.R.; Nguyen-Huy, T.; Bao, P.Q.; ElKhrachy, I.; Varasano, A. Gully erosion mapping susceptibility in a Mediterranean environment: A hybrid decision-making model. Int. Soil Water Conserv. Res.; 2024; 12, pp. 279-297. [DOI: https://dx.doi.org/10.1016/j.iswcr.2023.09.008]
58. Florinsky, I.V. Computation of the third-order partial derivatives from a digital elevation model. Int. J. Geogr. Inf. Sci.; 2009; 23, pp. 213-231. [DOI: https://dx.doi.org/10.1080/13658810802527499]
59. Kinnell, P.I.A. Alternative Approaches for Determining the USLE-M Slope Length Factor for Grid Cells. Soil Sci. Soc. Am. J.; 2005; 69, pp. 674-680. [DOI: https://dx.doi.org/10.2136/sssaj2004.0047]
60. Boehner, J.; Selige, T. Spatial Prediction of Soil Attributes Using Terrain Analysis and Climate Regionalisation. SAGA—Analysis and Modelling Applications; Boehner, J.; McCloy, K.R.; Strobl, J. Goettinger Geographische Abhandlungen: Göttingen, Germany, 2006; Volume 115, pp. 13-27.
61. Passalacqua, P.; Do Trung, T.; Foufoula-Georgiou, E.; Sapiro, G.; Dietrich, W.E. A geometric framework for channel network extraction from lidar: Nonlinear diffusion and geodesic paths. J. Geophys. Res. Earth Surf.; 2010; 115, pp. 1-18. [DOI: https://dx.doi.org/10.1029/2009JF001254]
62. Quinn, P.F.; Beven, K.J.; Chevallier, P.; Planchon, O. The prediction of hillslope flow paths for distributed hydrological modelling using digital terrain models. Hydrol. Process.; 1991; 5, pp. 59-79. [DOI: https://dx.doi.org/10.1002/hyp.3360050106]
63. Gruber, S.; Peckham, S. Land-Surface Parameters and Objects in Hydrology. Dev. Soil Sci.; 2008; 33, pp. 293-308.
64. Skentos, A. Topographic position index based landform analysis of Messaria (Ikaria Island, Greece). Acta Geobalcanica; 2018; 4, pp. 7-15. [DOI: https://dx.doi.org/10.18509/AGB.2018.01]
65. Kołodyńska-Gawrysiak, R.; Gawrysiak, L.; Poesen, J.; Plak, A. The impact of natural closed depressions on soil organic carbon storage in eroded loess landscapes of East Poland. Land Degrad. Dev.; 2023; 34, pp. 5273-5292. [DOI: https://dx.doi.org/10.1002/ldr.4843]
66. Krawchuk, M.A.; Haire, S.L.; Coop, J.; Parisien, M.A.; Whitman, E.; Chong, G.; Miller, C. Topographic and fire weather controls of fire refugia in forested ecosystems of northwestern North America. Ecosphere; 2016; 7, e01632. [DOI: https://dx.doi.org/10.1002/ecs2.1632]
67. Riley, S.J.; De Gloria, S.D.; Elliot, R. A Terrain Ruggedness that Quantifies Topographic Heterogeneity. Intermt. J. Sci.; 1999; 5, pp. 23-27.
68. Heerdegen, R.G.; Beran, M.A. Quantifying source areas through land surface curvature. J. Hydrol.; 1982; 57, pp. 359-373. [DOI: https://dx.doi.org/10.1016/0022-1694(82)90155-X]
69. Jasiewicz, J.; Stepinski, T.F. Geomorphons—A pattern recognition approach to classification and mapping of landforms. Geomorphology; 2013; 182, pp. 147-156. [DOI: https://dx.doi.org/10.1016/j.geomorph.2012.11.005]
70. Wood, J. Geomorphometry in LandSerf. Dev. Soil Sci.; 2009; 33, pp. 333-349.
71. Ballabio, C.; Sterlacchini, S. Support vector machines for landslide susceptibility mapping: The staffora River Basin case study. Italy Math. Geosci.; 2012; 44, pp. 47-70. [DOI: https://dx.doi.org/10.1007/s11004-011-9379-9]
72. Tien Bui, D.; Pradhan, B.; Lofman, O.; Revhaug, I. Landslide susceptibility as-sessment in vietnam using support vector machines, decision tree, and naïve bayes models. Math. Probl. Eng.; 2012; 2012, pp. 1-26. [DOI: https://dx.doi.org/10.1155/2012/974638]
73. Vapnik, V. Statistical Learning Theory; John Wiley and Sons, Inc.: New York, NY, USA, 1998.
74. Kavzoglu, T.; Colkesen, I. A kernel functions analysis for support vector machines for land cover classification. Int. J. Appl. Earth Obs. Geoinf.; 2009; 11, pp. 352-359. [DOI: https://dx.doi.org/10.1016/j.jag.2009.06.002]
75. Abe, S. Support Vector Machines for Pattern Classification; 2nd ed. Springer: London, UK, 2010.
76. Smola, A.J.; Schölkopf, B. A tutorial on support vector regression. Stat. Comput.; 2004; 14, pp. 199-222. [DOI: https://dx.doi.org/10.1023/B:STCO.0000035301.49549.88]
77. Tien Bui, D.; Anh Tuan, T.; Klempe, H.; Pradhan, B.; Revhaug, I. Spatial prediction models for shallow landslide hazards: A comparative assessment of the efficacy of support vector machines, artificial neural networks, kernel logistic regression, and logistic model tree. Landslides; 2016; 13, pp. 361-378. [DOI: https://dx.doi.org/10.1007/s10346-015-0557-6]
78. Al-Juaidi, A.E.M.; Nassar, A.M.; Al-Juaidi, O.E.M. Evaluation of flood susceptibility mapping using logistic regression and GIS conditioning factors. Arab. J. Geosci.; 2018; 11, 765. [DOI: https://dx.doi.org/10.1007/s12517-018-4095-0]
79. Ozdemir, A. Sinkhole susceptibility mapping using logistic regression in Karapınar (Konya, Turkey). Bull. Eng. Geol. Environ.; 2016; 75, pp. 681-707. [DOI: https://dx.doi.org/10.1007/s10064-015-0778-x]
80. Ciotoli, G.; Di Loreto, E.; Finoia, M.G.; Liperi, L.; Meloni, F.; Nisio, S.; Sericola, A. Sinkhole susceptibility, Lazio Region, Central Italy. J. Maps; 2016; 12, pp. 287-294. [DOI: https://dx.doi.org/10.1080/17445647.2015.1014939]
81. Krizhevsky ASutskever, I.; Hinton, G.E. ImagNet classification with deep convolutional neural networks. Proceedings of the Advances in Neural Information Processing Systems (NIPS); Lake Tahoe, NV, USA, 3–6 December 2012; pp. 1106-1114.
82. Lecun, Y.; Bottou, L.; Gengio, Y.; Haffner, P. Gradient-based learning applied to document recognition. Proc. IEEE; 1998; 86, pp. 2278-2324. [DOI: https://dx.doi.org/10.1109/5.726791]
83. Tien Bui, D.; Nguyen, Q.P.; Hoang, N.D.; Klempe, H. A novel fuzzy K-nearest neighbor inference model with differential evolution for spatial prediction of rainfall-induced shallow landslides in a tropical hilly area using GIS. Landslides; 2017; 14, pp. 1-17. [DOI: https://dx.doi.org/10.1007/s10346-016-0708-4]
84. Zhang, Y.; Cao, G.; Wang, B.; Li, X. A novel ensemble method for k-nearest neighbor. Pattern Recogn.; 2019; 85, pp. 13-25. [DOI: https://dx.doi.org/10.1016/j.patcog.2018.08.003]
85. Breiman, L. Random forests. Mach. Learn.; 2001; 45, pp. 5-32. [DOI: https://dx.doi.org/10.1023/A:1010933404324]
86. Youssef, A.M.; Pourghasemi, H.R.; Pourtaghi, Z.S.; Al-Katheeri, M.M. Landslide susceptibility mapping using random forest, boosted regression tree, classification and regression tree, and general linear models and comparison of their performance at Wadi Tayyah Basin, Asir Region Saudi Arabia. Landslides; 2016; 13, pp. 839-856. [DOI: https://dx.doi.org/10.1007/s10346-015-0614-1]
87. Chen, W.; Shirzadi, A.; Shahabi, H.; Ahmad, B.B.; Zhang, S.; Hong, H.; Zhang, N. A novel hybrid artificial intelligence approach based on the rotation forest ensemble and naïve Bayes tree classifiers for a landslide susceptibility assessment in Langao County, China. Geomat. Nat. Hazards Risk; 2017; 8, pp. 1955-1977. [DOI: https://dx.doi.org/10.1080/19475705.2017.1401560]
88. Lagomarsino, D.; Tofani, V.; Segoni, S.; Catani, F.; Casagli, N. A tool for classifi-cation and regression using random forest methodology: Applications to landslide susceptibility mapping and soil thickness modeling. Environ. Model. Assess.; 2017; 22, pp. 201-214. [DOI: https://dx.doi.org/10.1007/s10666-016-9538-y]
89. Hong, H.; Liu, J.; Tien Bui, D.; Pradhan, B.; Acharya, T.D.; Pham, B.T.; Zhu, A.-X.; Chen, W.; Ahmad, B.B. Landslide susceptibility mapping using J48 Decision Tree with AdaBoost, Bagging and Rotation Forest ensembles in the Guangchang area (China). CATENA; 2018; 163, pp. 399-413. [DOI: https://dx.doi.org/10.1016/j.catena.2018.01.005]
90. Friedman, J.H. Greedy function approximation: A gradient boosting machine. Ann. Stat.; 2001; 29, pp. 1189-1232. [DOI: https://dx.doi.org/10.1214/aos/1013203451]
91. Chung CJ, F.; Fabbri, A.G. Validation of spatial prediction modelsfor landslidehazard mapping. Nat. Hazards; 2003; 30, pp. 451-472. [DOI: https://dx.doi.org/10.1023/B:NHAZ.0000007172.62651.2b]
92. Lee, E.J.; Shin, S.Y.; Ko, B.C.; Chang, C. Early sinkhole detection using a drone-based thermal camera and image processing. Infrared Phys. Technol.; 2016; 78, pp. 223-232. [DOI: https://dx.doi.org/10.1016/j.infrared.2016.08.009]
93. Silva, O.L.; Bezerra, F.H.; Maia, R.P.; Cazarin, C.L. Karst landforms revealed at various scales using LiDAR and UAV in semi-arid Brazil: Consideration on karstification processes and methodological constraints. Geomorphology; 2017; 295, pp. 611-630. [DOI: https://dx.doi.org/10.1016/j.geomorph.2017.07.025]
94. Suh, J.; Choi, Y. Mapping hazardous mining-induced sinkhole subsidence using unmanned aerial vehicle (drone) photogrammetry. Environ. Earth Sci.; 2017; 76, 144. [DOI: https://dx.doi.org/10.1007/s12665-017-6458-3]
95. Gutiérrez, F.; Galve, J.P.; Lucha, P.; Castañeda, C.; Bonachea, J.; Guerrero, J. Integrating geomorphological mapping, trenching, InSAR and GPR for the identification and characterization of sinkholes: A review and application in the mantled evaporite karst of the Ebro Valley (NE Spain). Geomorphology; 2011; 134, pp. 144-156. [DOI: https://dx.doi.org/10.1016/j.geomorph.2011.01.018]
96. Liu, R.; Sun, H.; Qin, J.; Zheng, Z. A multi-geophysical approach to assess potential sinkholes in an urban area. Eng. Geol.; 2023; 318, 107100. [DOI: https://dx.doi.org/10.1016/j.enggeo.2023.107100]
97. Guo, Z.T.; Ding, Z.L.; Liu, T.S. The deposition in loess-soil forming and the quaternary weather. Chin. Sci. Bull.; 1996; 41, pp. 56-59. (In Chinese)
98. Xu, L.; Dai, F.C. Statistical analysis of the characteristic parameters of loess landslides at the South Jingyang Plateau. Hydrogeol. Eng. Geol.; 2008; 5, pp. 28-32.
99. Wang, G.H.; Zhang, D.X.; Furuya, G.; Yang, J. Pore-pressure generation and fluidization in a loess landslide triggeredby the 1920 Haiyuan earthquake, China: A case study. Eng. Geol.; 2014; 174, pp. 36-45. [DOI: https://dx.doi.org/10.1016/j.enggeo.2014.03.006]
100. Gutiérrez, F.; Guerrero, J.; Lucha, P. Quantitative sinkhole hazard assessment. A case study from the Ebro Valley evaporite alluvial karst (NE Spain). Nat. Hazards; 2008; 45, pp. 211-233. [DOI: https://dx.doi.org/10.1007/s11069-007-9161-y]
101. Taheri, K.; Gutierrez, F.; Mohseni, H.; Raeisi, E.; Taheri, M. Sinkhole susceptibility mapping using the analytical hierarchy process (AHP) and magnitude–frequency relationships: A case study in Hamadan 1170 province, Iran. Geomorphology; 2015; 234, pp. 64-79. [DOI: https://dx.doi.org/10.1016/j.geomorph.2015.01.005]
102. Qiu, X.; Wu, S.-S.; Chen, Y. Sinkhole susceptibility assessment based on morphological, imagery, and contextual attributes derived from GIS and imagery data. J. Cave Karst Stud.; 2020; 82, 1125. [DOI: https://dx.doi.org/10.4311/2018ES0118]
103. Ozdemir, A. Sinkhole susceptibility mapping using a frequency ratio method and GIS technology near Karapınar, Konya-Turkey. Procedia Earth Planet. Sci.; 2015; 15, pp. 502-506. [DOI: https://dx.doi.org/10.1016/j.proeps.2015.08.059]
104. Kim, Y.J.; Nam, B.H.; Jung, Y.H.; Liu, X.; Choi, S.; Kim, D.; Kim, S. Probabilistic spatial susceptibility modeling of carbonate karst sinkhole. Eng. Geol.; 2022; 306, 106728. [DOI: https://dx.doi.org/10.1016/j.enggeo.2022.106728]
105. Nachappa, T.G.; Piralilou, S.T.; Gholamnia, K.; Ghorbanzadeh, O.; Rahmati, O.; Blaschke, T. Flood susceptibility mapping with machine learning, multi-criteria decision analysis and ensemble using Dempster Shafer Theory. J. Hydrol.; 2020; 590, 125275. [DOI: https://dx.doi.org/10.1016/j.jhydrol.2020.125275]
106. Tehrani, F.S.; Calvello, M.; Liu, Z.; Zhang, L.; Lacasse, S. Machine learning and landslide studies: Recent advances and applications. Nat. Hazards; 2022; 114, pp. 1197-1245. [DOI: https://dx.doi.org/10.1007/s11069-022-05423-7]
107. Chen, Y.Z.; Chen, W.; Janizadeh, S.; Bhunia, G.S.; Bera, A.; Pham, Q.B.; Linh NT, T.; Balogun, A.L.; Wang, X.J. Deep learning and boosting framework for piping erosion susceptibility modeling: Spatial evaluation of agricultural areas in the semi-arid region. Geocarto Int.; 2022; 37, pp. 4628-4654. [DOI: https://dx.doi.org/10.1080/10106049.2021.1892212]
108. Basso, A.; Bruno, E.; Parise, M.; Pepe, M. Morphometric analysis of sinkholes in a karst coastal area of southern Apulia (Italy). Environ. Earth Sci.; 2013; 70, pp. 2545-2559. [DOI: https://dx.doi.org/10.1007/s12665-013-2297-z]
109. Jouves, J.; Viseur, S.; Arfib, B.; Baudement, C.; Camus, H.; Collon, P.; Guglielmi, Y. Speleogenesis, geometry, and topology of caves: A quantitative study of 3D karst conduits. Geomorphology; 2017; 298, pp. 86-106. [DOI: https://dx.doi.org/10.1016/j.geomorph.2017.09.019]
110. Li, X.A.; Song, Y.X.; Ye, W.J. Engineering Geological Research on Tunnel-Erosion in Loess; Tongji University Press: Shanghai, China, 2010; (In Chinese)
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.
Details
; Wang, Lin 1
; Zhang, Fanyu 3 ; Wang, Ninglian 2
; Wu, Songbai 2 ; Wang, Xingang 4
; Jiang, Zongda 1 1 Shaanxi Key Laboratory of Earth Surface System and Environmental Carrying Capacity, Northwest University, Xi’an 710127, China;
2 Shaanxi Key Laboratory of Earth Surface System and Environmental Carrying Capacity, Northwest University, Xi’an 710127, China;
3 MOE Key Laboratory of Mechanics on Disaster and Environment in Western China, Department of Geological Engineering, Lanzhou University, Lanzhou 730000, China;
4 State Key Laboratory of Continental Dynamics, Department of Geology, Northwest University, Xi’an 710127, China;