An Application Study on Road Surface Monitoring Using DTW Based Image Processing and Ultrasonic Sensors

Abstract

Road surface monitoring is an essential problem in providing smooth road infrastructure to commuters. This paper proposed an efficient road surface monitoring using an ultrasonic sensor and image processing technique. A novel cost-effective system, which includes ultrasonic sensors sensing with GPS for the detection of the road surface conditions, was designed and proposed. Dynamic time warping (DTW) technique was incorporated with ultrasonic sensors to improve the classification and accuracy of road surface detecting conditions. A new algorithm, HANUMAN, was proposed for automatic recognition and calculation of pothole and speed bumps. Manual inspection was performed and comparison was undertaken to validate the results. The proposed system showed better efficiency than the previous systems with a 95.50% detection rate for various road surface irregularities. The novel framework will not only identify the road irregularities, but also help in decreasing the number of accidents by alerting drivers.

1. Introduction

Treacherous road surface conditions are a significant problem for safe and comfortable transportation. The importance of road infrastructure for society can be compared with the importance of blood vessels for humans. For better road surface quality, it should be monitored continuously and repaired as necessary. The optimal distribution of resources for road repairs is possible by providing the availability of comprehensive and objective real-time data about the state of the roads. Developing countries with an escalating economy have an enormously extensive network of roads. Events within the past have shown that countries like the USA have invested about $68 billion a year on the maintenance of roads, bridges, and highways. These potholes also contribute to the degradation of suspension and fuel efficiency. In India, roads are the most common means of mobility; roads carry 90% of passengers and 65% of freight. Even with such a massive network, traveling is tiring in India as the roads are not sufficiently wide and the maintenance is inadequate. Regardless of where you are, driving is a stressful and potentially life-threatening affair. Over the last two decades, with an exponential rise in population, the number of vehicles has increased at a tremendous rate. The road infrastructure is not at par with the number of vehicles to support it, resulting in traffic congestion and accidents. As the ECONOMIC TIME’s report of January 2018 suggests, at the very least, 400 people lose their lives every day in the country to road accidents due to potholes [1].

According to the report, “Road Accidents in India, 2016 [1] ”, by the Ministry of Road Transport and Highways, there were 480,652 road accidents in 2016 as opposed to 501,423 in 2015. However, fatalities resulting from these accidents have risen by about 3.2%, which means that nearly 150,785 people were killed in 2016 compared to 46,133 in 2015 [1]. In India, most of the vehicles (unusually small hatchbacks) sold are either a 0-star rating or a 1-star rating in correspondence to the European standards, which implies that there could be fatal injuries even at moderate speeds. A considerable amount of research has been conducted on reducing traffic congestion and the number of ever-growing accidents. Vast sums of money are spent on repairing these potholes and bumps, however, small developing countries cannot afford to invest these amounts, which are a significant cause of road degradation. Irregularities on the road such as potholes caused by activities like erosion, weather, traffic, and some other factors may lead to accidents. Moreover, human-induced bumps such as speed breakers, which are made in the form of speed bumps, are used to control the speed of the vehicles, but they are not distributed at even distances, and the heights of the speed bumps have not been scientifically justified, which could also cause accidents. While said obstacles ought to be signaled according to specific road regulations, they are not always correctly labeled. One of the primary goals of this research was to detect road irregularities and decrease the number of accidents and fatalities. This approach is cost-efficient and can be applied worldwide. Generically, in modern-day field practices, the evaluation of the road irregularity data is done entirely via the raw data assessment [2]. Several road irregularity stats are used for the same. The existing technique can be categorically classified into three main types: imaging-based, sensor, and manual techniques.

There has been a substantial amount of research done in the past on the detection of potholes [3,4,5] using various techniques like reconstruction using laser [6], stereo vision [7,8], Light Detection and Raanging (LIDAR) [9], ultrasonic [10,11,12,13], etc. Forrest et al. [10] proposed a method of detecting road surface disruptions based on ultrasonic sensors in which the pothole detection rate of 62% was achieved on a paved road. Singh et al. [11] used a global positioning system receiver and ultrasonic for the identification of geographical location coordinates of the detected potholes and speed breaker with 59.23% efficiency. Madli et al. [12] worked on the automatic detection and notification of potholes and humps on roads to aid drivers using ultrasonic and a Global Positioning System (GPS). It was found that the system was 74% efficient and the data could be stored in the cloud. Shivaleelavathi et al. [13] also proposed an intelligent system using an ultrasonic sensor based on Raspberry Pi, GPS, and ultrasonic to take corrective measures. Devapriya et al. [14] proposed a real-time system that detected speed bumps by the analysis of photos of the road that were further enhanced using a Gaussian Filter (computer vision technique). This system had a correct rate ranging from 30% to 92%; however, failure was witnessed for faded speed bumps, and for a higher price, they had to be painted and maintained; this was a significant drawback of the system. Eriksson et al. [8] established the patrol system that comprised of a GPS and a 3-axis accelerometer for the detection of potholes. Likewise, Chen et al. [15] used the very same system and evaluated the power spectral density (PSD) for the detection of pavement irregularities using Fourier transformations. Inaccurate location of the irregularities was a shortcoming of the system. Nericell was a system developed by Mohan et al. [7] that comprised of amici, global system for mobile communication (GSM), accelerometer, and GPS sensors for road surface monitoring. The method yielded a False Negative Rate (FNR) of 37% and 41% for low speed and high-speed bumps, respectively. The use of smartphones, an adaptive fuzzy classifier, GPS, and inertial sensors for the detection of sudden driving events was presented by Arroyo et al. [16] and a precision performance of 0.87 and 0.91 was recorded. A fuzzy inference system was proposed by Aljaafreh et al. [17] for the detection of speed bumps using accelerometers embedded in smartphones. Nevertheless, the actual results of the tests were missing and accuracy only ranged from 70–80%. A multi-mobile phone system was developed by Astarita et al. [18], who also used accelerometers for the detection of bumps and potholes. A whopping 90% detection rate was obtained for bumps whereas the False Positive Rate (FPR) was about 35%.

Furthermore, the use of a smartphone accelerometer with six different algorithms to categorize road surface conditions was done by González et al. [19], who reported an Area Under Curve (AUC) ranging from 0.823 to 0.9445. Nature driven approaches such as genetic algorithm have also been explored: Yu and Yu [20] explored the use of genetic algorithms using images as a source of information to detect road irregularities. Moreover, abnormalities or distress data collection are increasingly being automated by using various imaging systems. However, analysis of the collected raw video clips for distress assessment is still predominantly being done manually, which is expensive, time-consuming, and slows down road maintenance management [21]. Pre-existing research suggests that three-dimensional image-based techniques are quite expensive; on the other hand, vibration sensor-based techniques are not as accurate and reliable [22] for road surface monitoring. The majority of the existing two-dimensional imaging methods have solely focused on either crack distress detection or pothole detection.

In this paper, an experimental analysis was conducted to detect potholes and speed bumps in quasi-real-time conditions using three different techniques, namely, an ultrasonic sensor with DWT, imaging-based systems, and manual inspection for validation from various roads of Noida city. In the first technique, which consists of an ultrasonic sensor, a GPS and Arduino connected to the Head Up Display (HUD) device are introduced. This system is a cost-effective method for a better, faster, and more reliable way to avoid accidents caused by potholes and raised humps. During the conduct of this experimental analysis, a collection of an exclusive data pattern of speed bumps and potholes was undertaken and directly fed to dynamic time warping (DTW) to find the likeness and similarity. A robust method for automated segmentation of frames with and without distress from the image processing technique is presented. These photo processing methods are responsible for the detection of potholes and speed bumps, which are supported by user-defined decision logic. The derivation of the decision logic is dependent on three characteristic visual properties of these irregularities (i.e., the standard deviation for calculating pixels intensities, circularity (CIRC) for calculating the shape, and average width for the dimension). An agile video separation algorithm called irregularity frame selection (IFS) was used to separate the visual frames with irregularities or not categorically. A new algorithm named the Hanuman algorithm took care of the second phase for automatically detecting and assessing potholes and speed bumps in one go. Finally, in the third technique, a manual visual inspection was undertaken to validate the above two methods. 2. Materials and Methods

The proposed threshold-based efficient road monitoring system detects the pothole and a speed break. Figure 1 shows the methodology overview of the information taken from the ultrasonic sensor. Collection and analysis of data were done using the primary server where the application of the DTW algorithm and noise filtering techniques was made. After the observation of the limitations of the existing systems, the present framework introduces the use of an ultrasonic sensor, DTW, IFS (algorithms), and filters for the detection of road irregularities. The ultrasonic sensor is fit in the moving direction of the car, as explained in Section 4. Information gathered with the help of sensors is sent to the central server for further analysis. Furthermore, noises, vibrations, and gravity components are eliminated using the filters. DTW helps to obtain the closeness score by analyzing the patterns for potholes and speed bumps. Ergo, road irregularity detection is undertaken with its GPS signature. Eventually, the final output is displayed on the mobile HUD for ease of the driver, making it even more ergonomic.

2.1. Hardware Setup

The main aim of the research was to create inexpensive hardware that runs on open software. Ergo, we used an Arduino Uno with an ultrasonic sensor (in Figure 2 and Figure 3) and a GPS. Table 1 shows the hardware specification and Figure 4 shows the components used in the setup. The architecture of the proposed system consisted of four parts: an ultrasonic sensor, Arduino, a server module, and a heads-up display module. The ultrasonic sensor and Arduino are used to gather information about potholes and speed bumps and their geographical locations, and this information is sent to the server. The server module receives information from the microcontroller module, processes, and stores in the database.

Ultrasonic sensors are utilized to determine the separation between the vehicle frame and the street surface numerically, and this value is passed to the microcontroller. Threshold distance is defined as the distance between the body of the vehicle and the smooth road surface, which depends upon the ground clearance of vehicles. If the value obtained by the ultrasonic sensor is more than the threshold distance, it recognizes it as a pothole, and if it is less, it is recognized as an unimportant irregularity, otherwise the street is smooth. The microcontroller module (Arduino) is the main controller of the entire system and is responsible for incorporating the ultrasonic sensors and sending and receiving signals between the GPS receiver and the sensors. The microcontroller module is merely the processing power of the whole system and is utilized to accumulate data about potholes and their topographical areas, and these data are passed onto the server. The server module gets this data from the microcontroller and uploads its respective location signatures to the cloud server. This location will appear on the GPS system whenever any vehicle passes through or near it. The Arduino contains several filters to smoothen out the output from the ultrasonic sensor. The basic variations from the ultrasonic sensor are bolstered to the filters, where the information is smoothed and standardized. The filters are to filter the vibration due to some irregularities from the road surface apart from speed bumps and potholes. These vibrations are removed from the ultrasonic sensor information to lessen the impact of other outside powers and for the system to be able to differentiate between a constant/normal variation and an actual pothole or a bump. 2.2. Automated Assessment of Potholes and Speed Bumps Using Imaging Processing The inception of the motivation for this research paper was induced directly or indirectly by our personal first-hand experiences with the Indian road conditions. The validation was for the determination of the efficiency of the proposed system. Initially, the pothole and bumps were detected with the help of using ultrasonic sensors paired with an Arduino attached with a heads-up display feeding the information to the GPS receiver. Irregularity frame selection is a recursive algorithm for searching all the vertices of a graph or tree data structure. This algorithm uses raw visual clips that are further put through several image processing methods; this is done using user-defined logics, which ensures the precise detection of irregularities. Segmentation of the irregular pixels in a visual frame is done by the algorithm using user defined decision logics that categorize those frames by the area covered by the irregular pixel. This segmentation is categorized into two parts: (1) Frames with irregularities, (2) frames without irregularities. The Hanuman algorithm is used in the presented technique for computerized location and estimation of speed bumps and potholes in a single go from a grouping of video outlines. This algorithm is generated by the arrangement of two visual properties of bumps and potholes, which are distinguished from video photographs of Indian streets. This aggregate arrangement of recognized visual properties incorporates the accompanying, including the following: Picture Texture: The picture surface of the interior of a pothole and speed bump is more different and distinguishing than the encompassing zone (i.e., irregularity less street surface region). Shape Factor: The state of a pothole and speed bump is distinguishable. Their shape factor is estimated as far as circularity, which changes depending on their surface territory and border. Measurement: The element of pothole and speed bump is expressed in terms of average width. This particular algorithm fundamentally includes five segments:

• Image augmentation (Algorithm steps 3 and 4),
• Photo categorization (Algorithm step 5),
• Extraction of visual properties of articles (Algorithm steps 6 to 11),
• Recognition and classification of irregularities by decision logic (Algorithm step 12) and
• Quantification (Algorithm step 13).

The calculation produced for the robotized identification, estimation, and arrangement of casings with potholes and speed bumps without basic irregularity from a grouping of street video outlines is recorded underneath.

• Categorized video frames are input;
• The initial frame is selected;
• A blue channel is selected for the conversion of the preset 24-bit depth format into 8-bit depth format;
• Median filtering is used for photo augmentation;
• Application of weighted mean based adaptive thresholding is made to covert processed photos into a binary image wherein black pixels signify objects of concentration;
• Morphological erosion is used to add black pixels to link the slits in the binary photo;
• Morphological dilation is used to eradicate secluded black pixels or their minor cluster;
• Morphological erosion is used again to add black pixels to the binary photo;
• Connected element classification and chain coding methods are used to quantify the number of substances or areas of notice and evaluate the area (A) and perimeter (P) of each of the object/substance;
• Filter out all the objects/substances whose A = 192cm² in the primary photo;
• Calculate standard deviation, circularity and average width of every remaining substance/object (critical objects, i.e., objects whose A ≥ 192 cm²);
• Categorize every substance/object into three kinds using heuristically derivative decision logic: Type (object) =

  (a) Pothole, if STD ≥ 12 & CIRC ≥ 0.14 & W ≥ 70 mm;

  (b) Speed Bump, if Convexity ≥ 5;

  (c) Unimportant Irregularity (UI), if otherwise;

• Save the visual frame with the extracted and enumerated data in its respective category type folder;
• Repeat steps 2 to 13 for all outstanding visual database;
• Finish.

The procedure applied to extract the pothole information using the proposed algorithm is shown in Figure 5. Figure 5a is the original image with potholes; (b) is the binary image after median filtering (i.e., after random noise reduction in column (a) image), and adaptive thresholding (i.e., after object segmentation in column (b) image); (c) is binary image after erosion (i.e., after joining disparate black pixels in column (b) image); column (d) image after the Hanuman algorithm and (e) is the report of the pothole. Similarly, Figure 6 shows the speed bumps.

3. Experimental Design and Location

To investigate the road surface, the framework used for the testing is shown in Table 2. The research was performed in the city of Noida, which comes under the National Capital Region of New Delhi. The city has an elevation of 200 m above sea level and a flat topographical region. Like most of the country, there are extremely contrasting roads in Noida, ranging from excellent roads to terrible roads. An expressway just outside of Noida (Figure 7a) and a slum road in Noida (Figure 7b) were considered for the experiment. Several test runs were performed, yielding different results. The experiment was conducted for three months covering around 150–170 km. For the evaluation of the performance of the framework, the algorithm presented in this research was applied in a Windows environment (Dell XPS 15Z with Intel i7 2600, 8GB DDR3 RAM on Windows 10 Pro) with the use of Visual Studio 2017 (VC ++ Redistributable) and Open CV3.3 Library. Hanuman, IFS, and DTW algorithms were used across the entire framework. For test runs, a Honda Amaze was used and image processing was done by a cost effective 4k resolution action camera. Table 2 shows the software and hardware specifications used in the experiment

4. Results and Discussion The evaluation of the three different techniques is presented while traveling over the R1 (Yamuna expressway) and R2 (Khora Village) roads of Noida city and was also compared with other existing techniques. 4.1. Proposed Novel Framework

The ultrasonic sensor detects the pothole and speed bumps; these data are saved on the server and used to determine the separation amid the vehicle body and the street surface numerically, and this value is passed by the Arduino. Threshold distance is defined as the distance between the body of the vehicle and the smooth road surface and depends upon on the ground clearance of vehicles. If the value obtained by the ultrasonic sensor is more than the threshold distance, it will recognize it as a pothole, and if it is less, it is a mound, otherwise the street is smooth. The proto test was conducted using the design described in the Methodology Section above, and the results for R1 and R2 are shown in Table 3 and Table 4.

As can be clearly seen, the uneven roads of the rural areas showed many potholes and bumps with some specific dimensions. On the other hand, the motorway road only showed irregularities up to 6.2 cm, which were filtered by filters such as Speed, Virtual Re-orientation, Filtering Z-pivot, SMA, and Band-Pass on the basis of the threshold limit being 8 cm. Figure 8 shows the experimental comparison in the z-axis displacement for R1 and R2.

4.2. Image Processing Result

Identification of the unique visuals of irregularities like illumination, size, and shape are made using image processing techniques. These accurately identify and help classify those irregularities. The irregularity frame selection algorithm is used as a quick video separation of the real-life visual clips captured of the Indian National Highways at several locations, which are divided into two frame categories (frames with irregularities and frames without irregularities). The Hanuman algorithm is responsible for the evaluation of the database of frames with irregularities, which is an automatic technique for the recognition and analysis of potholes and speedbumps. The proto test was conducted using the design described in the image processing methodology and the results for R1 and R2 are shown in Table 5 and Table 6.

4.3. Manual Inspection

Manual inspection was performed using a band meter, which is used to measure the depth of an object with precision and accuracy. Figure 9 shows the manual inspection at R1 and R2, respectively. The proto test was conducted using manual inspection, and the results for R1 and R2 are shown in Table 7 and Table 8. Generally, manual inspection is performed using some technical equipment, but in our case, we performed a very traditional manual inspection. Different scales were used to measure the different potholes and speed bumps for the validation.

4.4. Validation of Proposed System with Other Techniques

The same experiment was conducted using three different methods several times on the very same roads; the rural road of Khora village and the urban road of the Yamuna Expressway. Each method, ultrasonic sensor, image processing, and manual inspection had slight deviations, as shown below. The comparison between three different techniques for depth/height, latitude, and longitude with their error are given in Table 9, Table 10, Table 11, Table 12, Table 13 and Table 14 for both roads.

The deviation of the location of the pothole is relevant in different methods. The ultrasonic sensor and image processing detection deviated slightly from the actual position. Ergo, to calculate the accuracy of the system, the mean of the output using different method was calculated. The maximum error was 4.60% and the efficiency of the proposed system was about 95.48%.

Table 15 shows a full comparison between the results obtained with the proposed methodology and those yielded by similar approaches. As can be seen, our approach clearly outperformed the others, where TPR is the True Positive Rate, FPR is the False Positive Rate, and FNR is the False Negative Rate. One of the strongest points presented in this research is the use of a multi-variate model for pothole and speed bump detection. The presented methodology builds a model by combining different methods for the detection of road surface irregularities which provides greater accuracy to the entire framework. The collaboration of different methods with our novel framework targets a much bigger audience, and since it is very cost effective, it makes this system ideal for road surface monitoring in all regions.

5. Conclusions It has been highlighted that the current road anomaly detection frameworks used by the government or the existing authorities are very tedious and overwhelmingly time consuming, which is why this device eradicates all the problems and complications with the existing systems with much better efficiency. The analysis results demonstrated the use of ultrasonic sensors mated with an Arduino, GPS receivers, and HUD module, which runs on several filters. Dynamic time warping was also one of the algorithms used, which makes the validation process more manageable and is also one of the reasons for the increased efficiency. The presented Hanuman algorithm could detect the potholes and speed bumps accurately and efficiently. The calculated accuracy of the system was 95.48%, and the use of inexpensive components makes it the most cost effective framework among the existing systems. The model proposed in this paper serves two critical purposes (i.e., automatic detection of potholes, bumps, and alerting vehicle drivers to evade potential accidents). The HUD application provides real-time alerts about potholes and bumps right in front onto the windshield, making it more ergonomic. The framework works perfectly in all seasons, especially in monsoon as the signal/warning is generated from a database. The novel framework ha the highest efficiency when compared to all preexisting systems. Since it is a cost effective long time solution for pothole management, it can be easily provided to the maximum number of people; which can furthermore save many lives. Integration of GMaps and Satnav with the system can be done for ease of use. This framework can be applied to different types of road conditions and will be the future scope of this work.

Component	Hardware Description
Arduino	Microcontroller board based on ATmega328 using an open-source, prototyping platform, having 14 digital input/output pins (6 can be used as PWM outputs), 6 analog inputs, 16 MHz crystal oscillator, USB connection, power jack, ICSP header, and a reset button.
HY-SRF05	Narrow acceptance angle, 5-pin male header with 2.54 mm (0.1") spacing, 35 mA active, Range: approximately to 4 m (12- feet), Power requirements: +5 VDC; Communication: Dimensions: 22 × 46 × 16 mm (0.81 × 1.8 × 0.6 in), positive TTL pulse, Operating temperature range: 0 to + 70 °C (+32 to +158 °F). Figure 4a Arduino with HY-SRF05.
GPS Receiver	It is a satellite navigation system and is used to capture geographic location and time, irrespective of the weather conditions (Figure 4b).
HUD	A mobile device that used to display the information on the windshield as shown in Figure 4c.
Filters	Several filters like Speed, Separating Z-Axis, Moving Average, Band Pass, and Virtual Orientation are used.

Technique Used	DTW + Ultrasonic Sensor + Image Processing
Location of Experiment	Noida City, UP, North India.
OS of the Data Collection System	Windows 10 Pro
Equipment used for Data Collection	Dell XPS 15z, HP Pavilion DV9000
Vehicle(s) used	Honda Amaze (2016)
Distance Covered (km)/Travel time (hours)	(150–170 km/4 h)
Detection	Location of potholes, speed bumps on the roads

Scenario No.	Obstacle Type	Depth/Height (cm)	Latitude (°N)	Longitudinal (°E)
1	B	3.0	28.4215	77.5261
2	B	3.0	28.4216	77.5261
3	P	2.6	28.4215	77.5265
4	B	5.2	28.4214	77.5266
5	B	4.8	28.4214	77.5268

Scenario No.	Obstacle Type	Depth/Height (cm)	Latitude (°N)	Longitudinal (°E)
1	P	9.27	28.6186	77.5544
2	B	2.20	28.6243	77.5661
3	B	8.63	28.6543	77.5667
4	P	13.12	28.6722	77.5643
5	P	8.20	28.6932	77.5634
6	P	8.19	28.7423	77.5612
7	B	3.20	28.7865	77.5755
8	P	1.77	28.7933	77.5765
9	B	3.72	28.8654	77.5659
10	P	12.16	28.9653	77.5760

Scenario No.	Obstacle Type	Depth/Height (cm)	Latitude (°N)	Longitudinal (°E)
1	B	3.14	27.1937	76.5725
2	B	3.11	27.5405	76.4872
3	P	2.57	28.0691	76.4799
4	B	5.40	28.8080	76.6505
5	B	4.60	27.9979	77.1004

Scenario No.	Obstacle Type	Depth/Height (cm)	Latitude (°N)	Longitudinal (°E)
1	P	8.8695	27.4166	76.2903
2	B	2.1318	27.4507	75.7666
3	B	8.3590	27.6514	77.3884
4	P	12.9271	28.3281	76.5420
5	P	8.2082	27.7750	74.2999
6	P	7.8984	28.2824	75.7508
7	B	3.0682	28.0668	74.5035
8	P	1.7144	28.0620	75.3345
9	B	3.6010	27.9302	75.0760
10	P	11.6006	28.2817	75.0858

Scenario No.	Obstacle Type	Depth/Height (cm)	Latitude (°N)	Longitudinal (°E)
1	B	3.11	27.41	78.65
2	B	3.13	27.25	76.50
3	P	2.50	27.53	76.49
4	B	5.30	28.00	76.61
5	B	4.58	28.45	77.17

Scenario No.	Obstacle Type	Depth/Height (cm)	Latitude (°N)	Longitudinal (°E)
1	P	8.8	28.15	75.72
2	B	2.15	27.96	76.33
3	B	8.29	28.20	75.63
4	P	12.93	27.96	75.60
5	P	7.88	28.35	75.75
6	P	7.95	28.06	74.49
7	B	3.15	28.32	75.33
8	P	1.70	28.45	76.31
9	B	3.59	27.94	75.77
10	P	11.87	28.03	77.40

Scenario No.	Obstacle Type	Depth/Height (cm)
		Proposed System	Image Processing	Manual Inspection	% Error (PS vs. IP)	% Error (PS vs. MI)
1	B	3.0	3.14	3.11	4.67	3.67
2	B	3.0	3.11	3.13	3.67	4.33
3	P	2.6	2.57	2.50	−1.15	−3.85
4	B	5.2	5.40	5.30	3.85	1.92
5	B	4.8	4.60	4.58	−4.17	−4.58

Scenario No.	Obstacle Type	Latitude (°N)
		Proposed System	Image Processing	Manual Inspection	% Error (PS vs. IP)	% Error (PS vs. MI)
1	B	28.4215	27.1937	27.41	4.32	3.56
2	B	28.4216	27.5405	27.25	3.1	4.12
3	P	28.4215	28.0691	27.53	1.24	3.14
4	B	28.4214	28.8080	28.00	−1.36	1.47
5	B	28.4214	27.9979	28.45	1.49	−0.1

Scenario No.	Obstacle Type	Longitude (°E)
		Proposed System	Image Processing	Manual Inspection	% Error (PS vs. IP)	% Error (PS vs. MI)
1	B	77.5261	76.5725	78.65	1.23	-1.45
2	B	77.5261	76.4872	76.50	1.34	1.32
3	P	77.5265	76.4799	76.49	1.35	1.34
4	B	77.5266	76.6505	76.61	1.13	1.18
5	B	77.5268	77.1004	77.17	0.55	0.46

Scenario No.	Obstacle Type	Depth/Height (cm)
		Proposed System	Image Processing	Manual Inspection	% Error (PS vs. IP)	% Error (PS vs. MI)
1	B	9.27	8.8695	8.88	4.32	4.21
2	B	2.20	2.1318	2.15	3.10	2.34
3	P	8.63	8.3590	8.29	3.14	3.96
4	B	13.12	12.9271	12.93	1.47	1.47
5	B	8.20	8.2082	7.88	-0.10	3.96
6	P	8.19	7.8984	7.95	3.56	2.89
7	B	3.20	3.0682	3.15	4.12	1.63
8	P	1.77	1.7144	1.70	3.14	4.10
9	B	3.72	3.6010	3.59	3.20	3.50
10	P	12.16	11.6006	11.87	4.60	2.36

Scenario No.	Obstacle Type	Latitude (°N)
		Proposed System	Image Processing	Manual Inspection	% Error (PS vs. IP)	% Error (PS vs. MI)
1	B	28.6186	27.4166	28.15	4.20	1.63
2	B	28.6243	27.4507	27.96	4.10	2.32
3	P	28.6543	27.6514	28.20	3.50	1.60
4	B	28.6722	28.3281	27.96	1.20	2.50
5	B	28.6932	27.7750	28.35	3.20	1.20
6	P	28.7423	28.2824	28.06	1.60	2.36
7	B	28.7865	28.0668	28.32	2.50	1.63
8	P	28.7933	28.0620	28.45	2.54	1.20
9	B	28.8654	27.9302	27.94	3.24	3.20
10	P	28.9653	28.2817	28.03	2.36	3.24

Scenario No.	Obstacle Type	Longitude (°E)
		Proposed System	Image Processing	Manual Inspection	% Error (PS vs. IP)	% Error (PS vs. MI)
1	B	77.5544	76.2903	75.72	1.63	2.36
2	B	77.5661	75.7666	76.33	2.32	1.60
3	P	77.5668	77.3884	75.63	0.23	2.50
4	B	77.5669	76.5430	75.60	1.32	2.54
5	B	77.5654	74.2999	75.75	4.21	2.34
6	P	77.5658	75.7508	74.49	2.34	3.96
7	B	77.5755	74.5035	75.33	3.96	2.89
8	P	77.5765	75.3345	76.31	2.89	1.63
9	B	77.5659	75.0760	75.77	3.21	2.32
10	P	77.5760	75.0858	77.40	3.21	0.23

Author	Detection Rate	Approach
Devapriya et al. [14]	30–92% TPR	Computer Vision Accelerometer & GPS
Eriksson et al. [8]	0.2 FPR
Mohan et al. [7]	11.1% FPR & 22% FNR	Accelerometer, Microphone, GPS & GSM
Forrest et al. [10]	62% Accuracy	Ultrasonic
Singh et al. [11]	59.23% Accuracy	Global positioning system receiver and ultrasonic
Madli et al. [12]	74% Accuracy	Ultrasonic and GPS
Shivaleelavathi et al. [13]	79% Accuracy	Raspberry Pi, GPS, ultrasonic
Mohamed et al. [23]	75.6–87.8% (Accuracy)	Accelerometer
Astarita et al. [18]	90% Accuracy, 35% FP	Accelerometer using Smart Phone
Proposed system	95.48% (Accuracy)	Ultrasonic Sensor, Image Processing and manual Inspection

Author Contributions

S.K.S. prepared the inspection system for the experiment and performed the experiments of this research. H.P. and J.L. conceived the original idea. J.L. designed the methodology and gave guidance and helped to improve the quality of the manuscript. S.K.S. wrote the original draft preparation in consultation with H.P. J.L reviewed and edited the final manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by a National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2019R1A5A808320111, No. NRF-2019R1G1A1004577).

Acknowledgments

The authors would like to thank Amrit Seth, Ansh Mittal, and Vansh Malik, students at Amity University, who helped in conducting this research.

Conflicts of Interest

The authors declare that they have no conflict of interest. We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

List of Abbreviations

GPS	Global Positioning System	PSD	Power Spectral Density
DTW	Dynamic Time Warping	UI	Unimportant Irregularity
HUD	Head Up Display	STD	Standard deviation
IFS	Irregularity Frame Selection	TPR	True Positive Rate
CIRC	Circularity	FPR	False Positive Rate
W	Average width	FNR	False Negative Rate