Fault Diagnosis Method of Distribution Equipment Based on Hybrid Model of Robot and Deep Learning

Abstract

In view of the poor effect of most fault diagnosis methods on the intelligent recognition of equipment images, a fault diagnosis method of distribution equipment based on the hybrid model of robot and deep learning is proposed to reduce the dependence on manpower and realize efficient intelligent diagnosis. Firstly, the robot is used to collect the on-site state images of distribution equipment to build the image information database of distribution equipment. At the same time, the robot background is used as the comprehensive database data analysis platform to optimize the sample quality of the database. Then, the massive infrared images are segmented based on chroma saturation brightness space to distinguish the defective equipment images, and the defective equipment areas are extracted from the images by OTSU method. Finally, the residual network is used to improve the region-based fully convolutional networks (R-FCN) algorithm, and the improved R-FCN algorithm trained by the online hard example mining method is used for fault feature learning. The fault type, grade, and location of distribution equipment are obtained through fault criterion analysis. The experimental analysis of the proposed method based on PyTorch platform shows that the fault diagnosis time and accuracy are about 5.5 s and 92.06%, respectively, which are better than other comparison methods and provide a certain theoretical basis for the automatic diagnosis of power grid equipment.

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. Introduction

With the continuous improvement of social economy and people's living standards, the power demand is increasing day by day, and the scale of power system is growing day by day. It includes transmission and transformation networks of various voltage levels. Therefore, ensuring the safe and stable operation of complex power grid is an inevitable requirement to ensure the economic and social development. At the same time, the economic and social development has a great impact on the security and economy, and higher reliability is required [1, 2]. As the terminal of the whole power grid operation, distribution network is the part with the widest coverage and the largest scale in China’s power system, and it is the key link to ensure that power can be “allocated and used” [3]. The distribution equipment will have some faults under long-term operation, resulting in abnormal temperature. Therefore, by detecting the temperature of the distribution equipment, the thermal fault diagnosis of the distribution equipment can be carried out quickly, which plays a great role in the safe operation of the power grid.

The infrared image of equipment is used for fault diagnosis with high efficiency, accurate judgment, safety, and reliability. At the same time, it is free from electromagnetic interference, fast detection speed, and no power failure of live equipment. Therefore, infrared diagnosis is widely used in the field of equipment fault monitoring and diagnosis technology [4]. However, due to the characteristics of large quantity and complex types of distribution equipment, if we only rely on manual work in the process of data acquisition, analysis, and processing, the workload is relatively large, the efficiency is low, and the accuracy is relatively low due to the high dependence on manual experience. Therefore, automatic image acquisition and analysis of distribution equipment are of great significance to ensure the safety and stability of distribution network [5].

In recent years, the automatic inspection technology of power distribution room has been popularized, and various automatic robots and UAVs have made great progress in the original data acquisition stage. However, the accurate and efficient processing of collected image data is still in its infancy. How to extract the features of interest from infrared images for power distribution equipment recognition is a problem to be solved [6]. Among them, the deep learning algorithm has made great achievements in image processing, speech recognition, and text analysis. By establishing a deep-seated neural network, high-level features are extracted from low-level features layer by layer, so as to achieve the effect of target classification and recognition [7]. Compared with the manually designed feature extraction method, the distributed features obtained by deep learning network model can better express the essence of data [8, 9]. Therefore, in order to improve the efficiency of thermal fault detection of distribution equipment, improve the intelligence of power grid, reduce the labor cost of detection, and reduce the false detection rate, a fault diagnosis method of distribution equipment based on the hybrid model of robot and deep learning is proposed, which effectively ensures the safe and reliable operation of distribution equipment.

2. Related Research

At present, there are many researches on fault diagnosis of distribution equipment at home and abroad, which can be divided into traditional fault identification and classification methods and machine learning based identification and classification methods [10]. Among them, the traditional fault identification and classification methods mainly include fuzzy clustering, discrete wavelet transform, and chaotic algorithm. For example, [11] proposed an infrared image segmentation algorithm based on intuitionistic fuzzy clustering algorithm based on spatial distribution information, which is suitable for power equipment. It can well suppress the strong interference of nontarget objects in infrared image to image segmentation, but the method is more traditional and has poor segmentation effect for complex intelligent power grid equipment. Reference [12] proposed an anomaly detection method based on spatial clustering applied by auxiliary feature vector and density noise. The auxiliary feature vector of each conditional variable is constructed for clustering to identify normal data patterns and different types of anomalies. Reference [13] proposed a data mining driving scheme based on discrete wavelet transform to realize high impedance fault detection in active distribution network, but the universality of the method is not high. Reference [14] proposed a method to obtain the vibration characteristics of circuit breaker based on time-frequency and chaos analysis to realize circuit breaker fault identification. The image analysis process is complex, resulting in the reduction of fault identification efficiency. Reference [15] proposes a method based on feature model for single-phase grounding fault in active distribution network system, which transforms the solution of nonlinear feature model into single-objective optimization of feature entropy, which can well identify single-phase fault, but the identification effect of equipment with feature type is not ideal.

With the continuous development of computer technology and the rapid development of 5G communication technology, machine learning algorithm has been widely used this year, especially the deep learning algorithm has certain advantages in the field of fault identification and classification. Reference [16] proposes artificial neural network algorithm to identify the insulator state and uses single-layer and multilayer perceptron artificial intelligence algorithm to classify the conditions of distribution insulators. This technology can make the automatic inspection of electrical system more accurate and efficient, but it lacks high reliable database for support. Reference [17] proposed a Mask R convolution neural network method and used transfer learning and dynamic learning rate algorithm to realize efficient recognition of annotated image data sets, but it relied too much on graphics annotation and lacked practical application value. In [18], appropriate traveling wave time-frequency characteristic parameters of fault current are selected as the input of adaptive depth belief network model to obtain the fault type, but only considering the fault current characteristics as the basis, the reliability needs to be further improved.

Based on the above analysis, aiming at the problems such as the complexity and diversity of smart grid distribution equipment and the unsatisfactory effect of most existing image recognition methods, a distribution equipment fault diagnosis method based on robot and deep learning hybrid model is proposed. Its innovations are summarized as follows:

(1) In order to obtain the image information of distribution equipment more comprehensively, the proposed method introduces the robot to construct the corresponding image knowledge database, which provides the basis for fault classification and fault location.

(2) In order to locate the equipment defect area in the infrared image of distribution equipment, the proposed method performs threshold segmentation on the infrared image in hue saturation value (HSV) space and uses OTSU method to extract the equipment defect area, so as to improve the accuracy of subsequent fault diagnosis.

(3) Aiming at the problem that the deep learning algorithm is prone to gradient disappearance and gradient explosion, the proposed method uses the residual network to improve the region-based fully convolutional networks (R-FCN) algorithm and applies it to the learning of faulty equipment, so as to obtain the fault type and location with high accuracy and further improve the safety of equipment.

3. Proposed Method

3.1. Construction of Image Information Database of Distribution Equipment Based on Robot Inspection

The traditional equipment status is usually determined by manual analysis. The workload is huge and error-prone, which affects the judgment of system status, resulting in potential safety hazards. Therefore, the robot is used for patrol inspection to obtain the status image of distribution equipment and build the corresponding information base for the analysis of equipment status, so as to find the faulty equipment in time and ensure the reliable operation of power grid [19]. The construction process of distribution equipment image information base based on robot inspection is shown in Figure 1.

[figure(s) omitted; refer to PDF]

The basic data sources of the database mainly include production system, online monitoring system, and robot background inspection system. The relevant data of the state quantity of power equipment mainly comes from the power production management system (PMS), which can provide the real-time operation condition, historical operation state, historical maintenance record, historical test data, equipment account, equipment parameters, and other information of the equipment. The online monitoring system mainly relies on various sensors on each power equipment for real-time monitoring. The robot background inspection system can not only provide the observation of some state quantities, but also carry out corresponding state evaluation and analysis for different equipment states according to the automatic state evaluation system. In addition, the data composition of the system includes infrared temperature measurement, visible light reading, and telemetry reading.

The robot inspection cycle generally refers to the inspection plan formulated by the distribution network operation inspection center, and two inspection robots complete the tasks of infrared temperature measurement and data transcription of equipment in the area [20]. At the same time, the robot background uses the threshold out of limit judgment method to automatically evaluate the equipment status. In order to ensure sufficient charging time of the robot and avoid the daily patrol and infrared temperature measurement period, the special patrol at night is set in the nonbusy working period of the robot every day, with the upper limit of one time. The data reports collected by the special patrol at night and infrared temperature measurement are included in the database for screening and preprocessing.

In addition, the background of the inspection robot is equipped with a system server, which includes data analysis software terminal, data exchange server, data storage server, data operation server, and other modules. The data exchange server is responsible for collecting and classifying the production system, online monitoring system, and robot patrol data into the storage server. There are three-party databases, fault information base and knowledge information base in the data storage server.

(1) The data of the three-party platform includes the information required in the database structure table. After eliminating the redundant information, the integrated data in a unified format can be obtained, and the defect alarm data can be located and retrieved quickly.

(2) The fault information base is mainly taken from the defect system records and contains a large number of relevant equipment fault cases, including fault characteristics, solutions, expert opinions, and manufacturer records. At the same time, the maintenance record database and equipment account database are used to build a comprehensive database of fault information, so as to screen the fault inspection points.

(3) The knowledge information base is the engine for the system to evaluate the equipment status and judge the fault. The internal rules at all levels provide the logical basis for the system to judge the fault. The key is knowledge acquisition, that is, collecting and mining the knowledge at all levels to enrich the knowledge base.

3.2. Defect Feature Extraction of Distribution Equipment

When extracting the defect features of distribution equipment, it is necessary to perform threshold segmentation on the infrared image in HSV space, separate the infrared image background from irrelevant equipment and defective equipment, and then extract the equipment defect area [21].

3.2.1. OTSU Threshold Segmentation

OTSU is considered to be one of the best algorithms in image threshold segmentation. The threshold segmentation process of OTSU algorithm is as follows: firstly, the image is processed in gray level, the number of pixels in the whole image is counted, and the probability distribution of each pixel in the whole image is calculated; then, the gray level is traversed and searched in the whole image, and the interclass probability of the image foreground and background at the current gray level is calculated; finally, the threshold corresponding to the variance between classes and within classes is calculated by the given objective function.

Suppose there are $D$ gray levels in the image, in which the number of pixels with gray value of $i$ is $N_i$ and the total number of pixels in the image is $N$. Then, the average gray value of the whole image is$$\begin{array}{}\text{(1)}& {\mu }_{\sum }={\displaystyle \sum _{i=0}^{D-1}i\frac{N_i}{N}}.\end{array}$$

According to the gray characteristics of the image, the image is divided into foreground $B_0$ and background $B_1$. Then, $p_{0}T$ and $p_{1}T$ represent the probability of occurrence of foreground $B_0$ and background $B_1$ when the threshold is $T$, respectively. The calculation is as follows:$$\begin{array}{c}\text{(2)}& p_{0}T={\displaystyle \sum _{i=0}^T\frac{N_i}{N}},p_{1}T=1-p_{0}T.\end{array}$$

Then, the mean values of foreground $B_0$ and background $B_1$ are$$\begin{array}{}\text{(3)}& \begin{array}{c}{\mu }_{0}T=\frac{{\displaystyle {\sum }_{i=0}^{T}i{N}_i/N}}{p_{0}T},\\ {\mu }_{1}T=\frac{{\mu }_{\sum }-{\displaystyle {\sum }_{i=0}^{T}i{N}_i/N}}{p_{1}T}.\end{array}\end{array}$$

The interclass variance with threshold $T$ in the gray histogram is calculated as follows:$$\begin{array}{}\text{(4)}& {\sigma }_B^{2}T=p_{0}T{{\mu }_{0}T-{\mu }_{\sum }}^2+p_{1}T{{\mu }_{1}T-{\mu }_{\sum }}^2.\end{array}$$

The optimal threshold is defined as the $T$ value corresponding to the maximum variance between classes, which is calculated as$$\begin{array}{}\text{(5)}& {\sigma }_B^2\widehat{T}=\underset{0\le T\le D-1}{\max }{\sigma }_B^{2}T.\end{array}$$

3.2.2. Defect Region Extraction Based on HSV

In order to improve the accuracy of equipment fault image classification, the defect region and background in the infrared image of fault power equipment are separated by using the defect region segmentation algorithm based on HSV. Since it is impossible to determine the defect type only by analyzing the fault area, it is necessary to segment the defect area based on mathematical morphology according to the location of the defect area. Through this method, the defective power equipment and the background area in the infrared image are separated, so as to reduce the interference of the background area in the infrared image on the defect type classification [22]. The flow of HSV based defect region extraction algorithm is shown in Figure 2.

[figure(s) omitted; refer to PDF]

When processing the infrared image of defective equipment, first merge the similar pixels corresponding to the area with the same temperature, and segment the image according to the threshold of the three components of the defective area in the HSV color space to extract the defective area. Then, the discrete defect regions are connected through the closed operation in mathematical morphology, and the threshold segmentation of the original image is carried out by OTSU method to separate the power equipment and the background region. Finally, the defect area is found in the binary image separated by OTSU method; that is, the defective power equipment is separated from other areas, so as to achieve the purpose of extracting defective power equipment and facilitate the identification and diagnosis of power equipment types and fault types.

3.3. Fault Diagnosis of Distribution Equipment Based on Deep Learning Hybrid Model

3.3.1. Defect Training Based on Deep Learning Hybrid Model

R-FCN algorithm architecture mainly includes backbone network, region proposal network (RPN), and region of interest (ROI) subnet [23]. When fault diagnosis of power distribution equipment is carried out, first input the collected infrared image of power equipment into convolution neural network and extract the convolution feature map of infrared image. In this process, deeper and more abstract image features can be extracted by using a larger backbone network (ResNet 101) to improve the recognition accuracy [24]. Then, the feature map is sent to the RPN network to generate anchors, which are marked with foreground and background, and the foreground area with high score is selected as the recommended area ROIs. These ROIs are sent to the ROI subnet for further training, and 300 recommended windows are generated for each infrared image of power equipment. At the same time, the characteristic map of the full convolution layer is calculated with the multilayer convolution kernel to generate a position sensitive score map. The ROI and Score Maps are input into the later Softmax layer for vote. Through the Softmax layer for classification, the ROI with the highest score is finally obtained, that is, the location and type of the object located and recognized. The architecture of R-FCN algorithm is shown in Figure 3.

[figure(s) omitted; refer to PDF]

(1) Residual Network. When the depth of the deep learning network reaches a certain degree, the problems of gradient disappearance and gradient explosion often appear during training. In order to solve this problem, the residual network (ResNet) is used to improve the R-FCN algorithm; that is, the residual network is selected as the backbone network. The residual element is essentially the mapping residual required for fitting through these stacked layers. Suppose that the network mapping is $Hx$ and the residual mapping function of the network is $Fx$, $Fx=Hx-x$. The so-called residual is the difference between the observed value $Hx$ and the estimated value $x$. The advantage of ResNet network is that it uses the stacking layer to fit $Hx$ to get the mapping $Hx=Fx+x$. The advantage of this representation is that if the model has been fitted to the best state, it only needs to make $Fx=0$ to get $Hx=x$, so as to avoid the disappearance and explosion of gradient.

The input $x_m$ and output $x_{m+1}$ of the $m$-th residual unit are expressed as follows:$$\begin{array}{c}\text{(6)}& x_{m+1}=fh{x}_m+\delta x_m,{\omega }_m,x_M=x_m+{\displaystyle \sum _{i=1}^{M-1}\delta x_i,{\omega }_i},\end{array}$$where $\delta$ is the ReLu activation function, $\omega$ is the weight between each unit, $m,M$ respectively represent the shallow residual unit and deep residual unit, $h\cdot$ represents the identity mapping, and $x_M$ is the final output of the residual unit.

In order to learn more and more abstract image features, the proposed method selects ResNet 101 network, and its configuration is shown in Table 1.

Table 1

Structure table of ResNet 101.

Layer name	101-Layer
conv1	7${}^{\ast }$7, stride 2
conv2_x	3${}^{\ast }$3max pool, stride 2
	$\begin{array}{cc}1\times 1& 64\\ 3\times 3& 64\\ 1\times 1& 256\end{array}\times 3$
conv3_x	$\begin{array}{cc}1\times 1& 128\\ 3\times 3& 128\\ 1\times 1& 512\end{array}\times 4$
conv4_x	$\begin{array}{cc}1\times 1& 256\\ 3\times 3& 256\\ 1\times 1& 1024\end{array}\times 23$
conv5_x	$\begin{array}{c}1\times 1512\\ 3\times 3512\\ 1\times 12048\end{array}\times 3$
Average pool, 1000-D FC, softmax

(2) RPN Network. The input of RPN network is image feature graph. According to anchor mechanism, 9 rectangular boxes with different sizes are generated for each point. When training the RPN network, compare the anchor with the manually calibrated true value area in the data set, mark the anchor frame with the largest overlap ratio as the foreground, and mark the anchor frame with an overlap ratio greater than 0.7 as the foreground sample. Mark the anchor box whose overlap ratio is less than 0.3 as the background sample. Select the positive and negative samples of anchor in proportion, use the maximum suppression method (NMS) and other methods to screen the top 250 ROIs with the highest score, and send these preliminarily screened preselected frames to the ROI subnet.

In addition, RPN network adopts anchor mechanism, which not only solves the problem of translation invariance, but also enables R-FCN algorithm to identify and locate targets with different overall dimensions. In the actual process of infrared image recognition of power distribution equipment, due to different equipment with different shape and structure, different sizes and variable aspect ratio, in order to ensure that there are targets in the receptive field corresponding to each sliding window on the feature map, multiscale anchor is required to ensure that the candidate frame is as complete as possible to select the target [25]. In the implementation of RPN network anchor, multiscale anchor can be obtained by setting the area of reference window (base_size), different area multiples and anchor aspect ratio, so that RPN can give more accurate foreground recommendation area.

(3) ROI Subnet. The role of ROI subnet is to correct the ROIs location obtained from RPN network, so as to obtain a more accurate target location of power equipment, so as to identify ROI. A position sensitive convolution layer is added after the last layer of the full convolution network, which can realize the translation variability of the algorithm and output the $k^{2}c+1$-dimensional position sensitive fractional graph. The RPN network filters out the characteristic map of ROI with size $a\times b$, which is divided into $k\times k$ parts (bin) and convoluted with $k^{2}c+1$ convolution cores; that is, each part is mapped to a score map, in which $c+1$ is the number of categories plus the background to obtain the $k^{2}c+1$-layer position sensitive score map. The number of channels of bin in different positions is $a\times b\times c+1$, and the score of class $c+1$ in this position is stored. After the pooling process is completed, vote on the ROI. Sum the $k\times k$ parts bin to get the output of $c+1$ dimension, that is, the probability of category (score), and classify it through the softmax layer. At this time, the output result is the positioning coordinates and type of the object.

(4) OHEM. In RPN network, a large number of rectangular boxes are generated, and hundreds or thousands of regions will participate in the training of predicting target categories and locations. The proportion of power equipment is small, so the ratio of equipment background area to target area is too large, resulting in sample imbalance, which makes it difficult to identify distribution equipment. Therefore, online hard example mining (OHEM) method is used to train the network model [26]. During training, when there is a preselected area with large loss, the hard example can be trained and classified again, which can solve the problem of imbalance between positive and negative sample categories and improve the accuracy of infrared image recognition model of power equipment. The training framework of OHEM method is shown in Figure 4.

[figure(s) omitted; refer to PDF]

When OHEM carries out specific training, firstly, ResNet 101 is used to extract the image features of training samples, train the image classification and positioning branches, and calculate the classification loss $L_{cls}$ and regression loss $L_{reg}$ of each target. The loss function is calculated as follows:$$\begin{array}{}\text{(7)}& Ls,d_{x,y,a,b}=L_{cls}{s}_{c^{\ast }}+\zeta c^{\ast }>0L_{\text{reg}}d,d^{\ast },\end{array}$$where $d_{x,y}$ is the upper left coordinate of the target area, $d_a$ is the width of the target area, $d_b$ is the height of the target area, $\zeta$ is the balance coefficient of classification loss and regression loss, and $s$ is the image type.

Then, sort according to the loss value from high to low, and use the NMS method to select the first $\mathrm{\Omega }$ samples with the largest loss value to screen out the difficult cases of this round of samples. Finally, these difficult samples are backpropagated to the network and trained again to update the weight of the whole network, and the penalty for low loss samples is ignored. Among them, the weight update is realized based on the random gradient descent method.

3.3.2. Fault Diagnosis of Distribution Equipment

There are many kinds of equipment in the distribution network, such as circuit breaker, potential transformer (PT), current transformer (CT), lightning arrester, and transformer. These equipment can be classified into current heating type, voltage heating type, and comprehensive heating type according to the heating factors. Different diagnostic methods are selected for different types of equipment, and different diagnostic methods have different diagnostic criteria. The defects of the equipment, such as general defects, serious defects, and critical defects, are determined through various diagnostic methods.

(1) The heating of current heating equipment is mainly due to the current thermal effect. Generally, the surface temperature judgment method and relative temperature difference judgment method are used for fault diagnosis. The relative temperature difference ${\nabla }_T$ and the maximum surface temperature $T_{\max }$ of different defect degrees of each distribution equipment are shown in Table 2.

(2) Voltage heating equipment is mainly due to voltage effect. The main equipment categories include zinc oxide arrester, high-voltage bushing, and coupling capacitor. This kind of equipment generally adopts image feature judgment method, similar comparison judgment method, and comprehensive analysis judgment method. Using the thermal image characteristics of the equipment and the image feature judgment method, the fault can be found quickly. If the similar comparison discrimination method and comprehensive analysis judgment method are used, the temperature difference $\mathrm{\Delta }T$ shall be taken as the fault diagnosis index, in which the $\mathrm{\Delta }T$ of zinc oxide arrester is 0.5∼1 K, and the $\mathrm{\Delta }T$ of high-voltage bushing, coupling capacitor, oil immersed PT and CT are 2∼3 K.

(3) Comprehensive heating equipment needs to be diagnosed in combination with the diagnosis methods of voltage heating equipment and current heating equipment, mainly including insulators, generators, and transformers. In the actual thermal fault diagnosis, the fault diagnosis indexes of various methods should be combined to improve the efficiency and accuracy of fault diagnosis [27].

Table 2

Fault diagnosis criterion of current heating equipment.

	General defect	Serious defect	Emergency defect
Circuit breaker	$35\%\le {\nabla }_T<80\%$	$80\%\le {\nabla }_T<95\%$	${\nabla }_T\ge 95\%$
	—	$55\%\le T_{\max }\le 80\%$	$T_{\max }>80\%$
Disconnecting switch	$35\%\le {\nabla }_T<80\%$	$80\%\le {\nabla }_T<95\%$	${\nabla }_T\ge 95\%$
	—	$90\%\le T_{\max }\le 130\%$	$T_{\max }>130\%$
CT	$35\%\le {\nabla }_T<80\%$	$80\%\le {\nabla }_T<95\%$	${\nabla }_T\ge 95\%$
	—	$55\%\le T_{\max }\le 80\%$	$T_{\max }>80\%$
Capacitor	$35\%\le {\nabla }_T<80\%$	$80\%\le {\nabla }_T<95\%$	${\nabla }_T\ge 95\%$
	—	$55\%\le T_{\max }\le 80\%$	$T_{\max }>80\%$
High voltage bushing	$35\%\le {\nabla }_T<80\%$	$80\%\le {\nabla }_T<95\%$	${\nabla }_T\ge 95\%$
	—	$55\%\le T_{\max }\le 80\%$	$T_{\max }>80\%$

According to different types of distribution equipment, the corresponding thermal fault diagnosis and judgment methods can be selected, the corresponding parameters can be calculated, and the thermal fault diagnosis of distribution equipment can be carried out according to the diagnosis criteria. The fault diagnosis process of power distribution equipment is shown in Figure 5.

[figure(s) omitted; refer to PDF]

Firstly, the infrared image of distribution equipment is input into the detection model for image preprocessing, and the defect area is extracted based on HSV to divide the structure of distribution equipment. Then, the trained depth learning hybrid model is used to obtain the type and location of the target equipment, and the temperature information of each structural area on the infrared image of the distribution equipment is read at the same time. Finally, according to the selected fault diagnosis method, the thermal fault state, thermal fault level, and thermal fault location of distribution equipment are determined by using the diagnosis criterion. Different from the existing fault diagnosis methods of artificial equipment, the proposed method can use the deep learning hybrid model to realize the intelligent classification and fault type diagnosis of infrared images of distribution equipment, greatly reduce the workload of inspectors, and improve the automation level of fault diagnosis of distribution equipment.

4. Experiment and Analysis

The experiment is carried out in the 64 bit operating system environment of Ubuntu 6.04.4 LTS, in which the deep learning hybrid model uses the deep learning framework PyTorch v1.3 version, the programming language is Python 3.6.0, and third-party dependent libraries such as Open CV 4.0 and NumPy 1.3 are used for batch processing of data. At the same time, the network model is trained on the GPU of dual card Tesla P100, with a total video memory of 8 GB. In this way, large batch data can be set during training to improve the convergence speed of the model. Other hardware environments are as follows: 512 GB of memory resources and 1 TB of hard disk. When processing data and I/O operations on a large scale, this can realize parallel processing and high speed and ensure the training requirements of network model.

4.1. Experimental Data Set

The research scenario is the power equipment in the distribution network, so it is necessary to collect the picture materials of equipment faults in the distribution system and then construct a fixed format data set for test training. Through safety training and professional leadership, use mobile phones, cameras, and other equipment to shoot at the site of power distribution equipment. The scene of abnormal power grid equipment is selected, and a large number of positive samples are collected in multiple directions according to the shooting angle of video monitoring. In order to ensure the rationalization of data distribution, all types and forms of equipment anomaly types are covered in the acquisition process. After that, the abnormal categories and areas of power grid equipment are marked through the open-source and free wizard marking software.

The data set contains 2580 on-site abnormal images of RGB power grid equipment during the day, 6300 mark boxes in total, and 2769 on-site abnormal images of infrared power grid equipment at night, 7000 mark boxes in total, all from the real power distribution room, power equipment plant, etc. After the data annotation is completed, the annotation file in XML format is generated, which corresponds to the real image.

4.2. Comparative Analysis of Training Speed

The proposed method combines the hybrid model of robot and deep learning for equipment fault diagnosis. ResNet network is used to optimize the R-FCN algorithm, and the defect area is extracted based on HSV to improve the diagnosis effect. In order to demonstrate the improvement effect of the proposed method, it is compared with the diagnosis methods of ResNet network, Otsu threshold segmentation, and OHEM training. The diagnosis accuracy and training time are shown in Table 3.

Table 3

Fault diagnosis criterion of current heating equipment.

Method	Accuracy/%	Training time/s
Original model	82.49	4.6
Original model + OTSU threshold segmentation	85.82	5.9
Original model + OTSU threshold segmentation + ResNet network	92.37	10.5
Original model + OTSU threshold segmentation + ResNet network + OHEM	96.78	11.2

It can be seen from Table 3 that extracting deeper equipment fault features using ResNet network can greatly improve the diagnosis accuracy, which is 9.88% higher than that of the model. However, due to the deepening of network layers, the training time is also increased, more than 10 s. At the same time, by integrating OHEM training depth learning model, the diagnostic accuracy continued to improve by 4.41%. Due to the simple and easy implementation of the training process, the training time is only increased by 0.7 s. It can be seen that the diagnostic accuracy of the proposed method is higher than that of the basic method, but the training time increases, and the training speed decreases.

In order to demonstrate the performance of the proposed method in training speed, it is compared with reference [11], reference [13], and reference [16]. The results are shown in Figure 6.

[figure(s) omitted; refer to PDF]

As can be seen from Figure 6, the training time of reference [13] is the shortest, only about 5S. Because its intuitionistic fuzzy clustering algorithm based on spatial distribution information for image recognition is simple and easy to implement, the training speed is fast. Reference [13] combines discrete wavelet transform and support vector machine algorithm to complete fault diagnosis, and [16] uses artificial neural network algorithm to classify faults. Both methods are complex and take a long time to calculate, so the training time is about 10 s. By using the improved R-FCN algorithm for fault diagnosis, the proposed method uses OHEM method to train it, which can simplify the data processing process, the training speed is fast, and the training time is about 5.5 s. At the same time, the robot background is used for data analysis, which can reduce the transmission time of image information.

4.3. Comparative Analysis of Fault Diagnosis Accuracy

The accuracy of fault diagnosis is a key judgment index. The accuracy of the proposed method and the methods in reference [11], reference [13], and reference [16] for fault diagnosis of distribution equipment is shown in Figure 7.

[figure(s) omitted; refer to PDF]

As can be seen from Figure 7, compared with other methods, with the iteration of epoch, the fault diagnosis accuracy of the proposed method tends to be stable, about 92.06%. Due to its combination of robot and deep learning hybrid model, it deeply extracts the characteristics of various types of fault equipment for diagnosis, which further ensures the reliability of diagnosis results. Similarly, [16] uses artificial neural network algorithm for state recognition, but there is no efficient way to obtain the equipment state, and there is no complete database to support it. Therefore, the diagnosis accuracy is reduced by about 6% compared with the proposed method. Reference [13] adopts the improved support vector machine algorithm of genetic algorithm for fault detection, which has a good effect on high impedance fault diagnosis, but its universality is not high, so the diagnosis accuracy is about 80%. Reference [11] uses the traditional intuitionistic fuzzy clustering algorithm for graphic classification. The traditional method is difficult to apply to a large number of distribution equipment, so the diagnosis accuracy is low.

For the three fault types, the diagnostic accuracy of different methods is shown in Table 4.

Table 4

Comparison results of diagnostic accuracy of each fault type.

Method	Current heating type (%)	Voltage heating type (%)	Comprehensive heating type (%)
Reference [11]	73.23	69.18	65.75
Reference [13]	82.84	80.36	79.04
Reference [16]	86.69	87.05	84.27
Proposed method	93.52	91.88	89.31

It can be seen from Table 4 that the diagnostic accuracy of comprehensive heating equipment is generally lower than that of current heating equipment and voltage heating equipment. Taking the proposed method as an example, the diagnostic accuracy of comprehensive heating equipment is 89.31%, and the other two types are higher than 90%. Because the diagnostic criteria of comprehensive heating equipment are complex and easy to be confused, they affect the fault diagnosis. The recognition accuracy of current heating type defects is slightly higher, which may be due to the obvious characteristics and large amount of data of infrared images of current heating type defects in the data set. However, the diagnosis accuracy of the proposed method is higher than that of other comparison methods. Taking the current heating equipment as an example, its diagnosis accuracy is as high as 93.52%, because it can well distinguish all kinds of faulty equipment by using the improved R-FCN algorithm to learn the equipment image features and evaluate the fault level according to the fault judgment. Other comparison methods only diagnose whether the equipment is faulty or not, but the diagnosis effect is poor for various specific fault types.

5. Conclusion

Nowadays, the construction of smart grid in China has entered a new stage of comprehensive and rapid development. The traditional manual detection methods have been difficult to deal with a large amount of infrared image data of power equipment. Therefore, a fault diagnosis method of distribution equipment based on the hybrid model of robot and deep learning is proposed. The image information database of distribution equipment based on robot inspection is constructed, and the OTSU method is used to extract the defect features of distribution equipment from the binary image of equipment based on HSV space. Then, the equipment defect characteristics are sent to the improved R-FCN algorithm for learning and analysis to obtain the fault type and location, and the fault level is obtained through the calculation of fault criterion. The experimental results based on PyTorch platform show that:

(1) Using the robot platform to build the image information database of distribution equipment can improve the accuracy of fault diagnosis of various equipment. The diagnostic accuracy of the proposed method for current heating equipment, voltage heating equipment, and comprehensive heating equipment is 93.52%, 91.88%, and 89.31%, respectively.

(2) Using OHEM method to train the improved R-FCN algorithm can shorten the model training time, improve the fault diagnosis efficiency, and further improve the diagnosis effect. The fault diagnosis time and accuracy are 5.5 s and 92.06%, respectively.

Since the extraction degree of the defective region will affect the recognition accuracy of the subsequent model, considering the subsequent semantic segmentation of the overall infrared image by using the convolution neural network optimized by conditional random field, the defective power equipment can be segmented in a more complex background to further improve the recognition accuracy of the subsequent model.

References

[1] S. Gangolu, P. Raja, M. P. Selvan, V. K. Murali, "Effective algorithm for fault discrimination and estimation of fault location in transmission lines," IET Generation, Transmission & Distribution, vol. 13 no. 13, pp. 2789-2798, DOI: 10.1049/iet-gtd.2018.5815, 2019.

[2] N. Hu, H. Du, S. Liu, Q. Lin, "Power equipment status information parallel fault diagnosis of based on MapReduce," Journal of Computational Methods in Science and Engineering, vol. 19 no. 88,DOI: 10.3233/jcm-191023, 2019.

[3] E. Bashar, Q. Han, R. Wu, L. Ran, O. Alatise, S. Jupe, "Analysis of DC offset in fault current caused by machines in a medium voltage distribution network," Journal of Engineering, vol. 2019 no. 17, pp. 3494-3499, DOI: 10.1049/joe.2018.8221, 2019.

[4] H. Tian, P. Liu, S. Zhou, Q. Wang, Z. Wu, J. Zhang, Z. Peng, "Research on the deterioration process of electrical contact structure inside the ± 500 kV converter transformer RIP bushings and its prediction strategy," IET Generation, Transmission & Distribution, vol. 13 no. 12, pp. 2391-2400, DOI: 10.1049/iet-gtd.2019.0110, 2019.

[5] W. Luo, H. Wang, L. Wang, Z. Zhu, H. Gao, "Faulted line location method for distribution systems based on the equipment’s information exchange," Dianli Xitong Baohu yu Kongzhi/Power System Protection and Control, vol. 47 no. 4, pp. 73-82, 2019.

[6] N. Narasimhulu, D. Kumar, M. V. Kumar, "Detection and classification of high impedance fault in power distribution system using hybrid technique," Journal of Circuits, Systems, and Computers, vol. 29 no. 08, pp. 67-976, DOI: 10.1142/s0218126620501182, 2020.

[7] K. Chen, J. Hu, Y. Zhang, Z. Yu, J. He, "Fault location in power distribution systems via deep graph convolutional networks," IEEE Journal on Selected Areas in Communications, vol. 38 no. 1, pp. 119-131, DOI: 10.1109/jsac.2019.2951964, 2020.

[8] F. Wan, P. Madhika, J. Chwa, M. Mozumdar, A. Ameri, "Automatic optimal synthesis of aircraft electric power distribution system," International Journal of Computing and Digital Systems, vol. 9 no. 3, pp. 363-375, DOI: 10.12785/ijcds/090303, 2020.

[9] K. Jia, Q. Zhao, T. Feng, T. Bi, "Distance protection scheme for DC distribution systems based on the high-frequency characteristics of faults," IEEE Transactions on Power Delivery, vol. 35 no. 1, pp. 234-243, DOI: 10.1109/tpwrd.2019.2909130, 2020.

[10] A. S. Alayande, I. K. Okakwu, O. E. Olabode, O. K. Nwankwoh, "Analysis of unsymmetrical faults based on artificial neural network using 11 kV distribution network of University of lagos as case study," Journal of Advances in Science and Engineering, vol. 4 no. 1, pp. 53-64, DOI: 10.37121/jase.v4i1.91, 2021.

[11] F. Hu, H. Chen, X. Wang, "An intuitionistic kernel-based fuzzy C-means clustering algorithm with local information for power equipment image segmentation," IEEE Access, vol. 8 no. 6, pp. 4500-4514, DOI: 10.1109/access.2019.2963444, 2020.

[12] H. Liu, Y. Wang, W. Chen, "Anomaly detection for condition monitoring data using auxiliary feature vector and density-based clustering," IET Generation, Transmission & Distribution, vol. 14 no. 1, pp. 108-118, DOI: 10.1049/iet-gtd.2019.0682, 2020.

[13] Youness, Mohammadnian, A. Turaj, A. Soroudi, "Fault detection in distribution networks in presence of distributed generations using a data mining–driven wavelet transform," IET Smart Grid, vol. 2 no. 2, pp. 163-171, 2019.

[14] Q. Yang, J. Ruan, Z. Zhuang, "Fault diagnosis of circuit breakers based on time-frequency and chaotic vibration analysis," IET Generation, Transmission & Distribution, vol. 14 no. 7, pp. 1214-1221, DOI: 10.1049/iet-gtd.2019.0985, 2020.

[15] T. Zhang, H. Yu, P. Zeng, L. Sun, C. Song, J. Liu, "Single phase fault diagnosis and location in active distribution network using synchronized voltage measurement," International Journal of Electrical Power and Energy Systems, vol. 117 no. 5,DOI: 10.1016/j.ijepes.2019.105572, 2020.

[16] S. Frizzo-Stefenon, M. C. Silva, D. W. Bertol, L. H. Meyer, A. Nied, "Fault diagnosis of insulators from ultrasound detection using neural networks," Journal of Intelligent and Fuzzy Systems, vol. 37 no. 5, pp. 6655-6664, DOI: 10.3233/jifs-190013, 2019.

[17] B. Wang, M. Dong, M. Ren, Z. Wu, C. Guo, T. Zhuang, O. Pischler, J. Xie, "Automatic fault diagnosis of infrared insulator images based on image instance segmentation and temperature analysis," IEEE Transactions on Instrumentation and Measurement, vol. 69 no. 8, pp. 5345-5355, DOI: 10.1109/tim.2020.2965635, 2020.

[18] H. Liang, Y. Liu, G. Sheng, X. Jiang, "Fault-cause identification method based on adaptive deep belief network and time-frequency characteristics of travelling wave," IET Generation, Transmission & Distribution, vol. 13 no. 5, pp. 724-732, DOI: 10.1049/iet-gtd.2018.6334, 2019.

[19] M. Gholami, A. Abbaspour, M. Moeini-Aghtaie, M. Fotuhi-Firuzabad, M. Lehtonen, "Detecting the location of short-circuit faults in active distribution network using PMU-based state estimation," IEEE Transactions on Smart Grid, vol. 11 no. 2, pp. 1396-1406, DOI: 10.1109/tsg.2019.2937944, 2020.

[20] W. Hu, C. Ruan, H. Nian, D. Sun, "Simplified modulation scheme for open-end winding PMSM system with common DC bus under open-phase fault based on circulating current suppression," IEEE Transactions on Power Electronics, vol. 35 no. 1, pp. 10-14, DOI: 10.1109/tpel.2019.2924821, 2020.

[21] K. Zhu, P. W. T. Pong, "Fault classification of power distribution cables by detecting decaying DC components with magnetic sensing," IEEE Transactions on Instrumentation and Measurement, vol. 69 no. 5, pp. 2016-2027, DOI: 10.1109/tim.2019.2922514, 2020.

[22] L. Romero, J. Blesa, V. Puig, G. Cembrano, C. Trapiello, "First results in leak localization in water distribution networks using graph-based clustering and deep learning," IFAC-PapersOnLine, vol. 53 no. 2, pp. 16691-16696, DOI: 10.1016/j.ifacol.2020.12.1104, 2020.

[23] W. Wang, N. Yu, Y. Gao, J. Shi, "Safe off-policy deep reinforcement learning algorithm for volt-VAR control in power distribution systems," IEEE Transactions on Smart Grid, vol. 11 no. 4, pp. 3008-3018, DOI: 10.1109/tsg.2019.2962625, 2020.

[24] D. A. León-Vargas, V. A. Bucheli-Guerrero, H. A. Ordoez, "Solar radiation prediction on photovoltaic systems using machine learning techniques," Revista Facultad de Ingeniería, vol. 29 no. 10, 2020.

[25] S. Yamane, K. Matsuo, "Adaptive control by convolutional neural network in plasma arc welding system," ISIJ International, vol. 60 no. 5, pp. 998-1005, DOI: 10.2355/isijinternational.isijint-2019-306, 2020.

[26] M. Nabati, H. Navidan, R. Shahbazian, S. A. Ghorashi, D. Windridge, "Using synthetic data to enhance the accuracy of fingerprint-based localization: a deep learning approach," IEEE Sensors Letters, vol. 4 no. 4,DOI: 10.1109/lsens.2020.2971555, 2020.

[27] M. S. Erbnescu, N. C. Manea, L. Streba, S. Belciug, I. E. Pleşea, I. Pirici, R. M. Bungărdean, R. M. Pleşea, "Automated gleason grading of prostate cancer using transfer learning from general-purpose deep-learning networks," Romanian Journal of Morphology and Embryology, vol. 61 no. 1, pp. 149-155, 2020.