Algorithm of Smart Building Supervision for Detecting and Counting Column Ties Using Deep Learning

Abstract

The Building Supervision of South Korea has been developed under the government′s desire to prevent poor construction and improve the quality of buildings to promote a safer life for the people. Although the building environment of today has been achieved through several improvements, the introduction of such technology is insufficient. Especially since building supervision is primarily carried out almost by manpower, in an era where numerous convergence technologies from the 4th industry are being utilized. Therefore, this study aimed to solve this inefficiency in the building supervision system by using the object detection technology of deep learning in the BIM environment. As a basic study to develop a smart supervision checking system that checks whether the information on the construction site matches the design information, the column ties were selected as a supervision item, and research was conducted. For this, we constructed the tie detection network and suggested an algorithm for information checking between the construction site and BIM environment. Through this, it was possible to confirm the possibility of practical supervision work and improve the efficiency of the work, and furthermore, to see the possibility of using convergence technology.

1. Introduction

The Republic of Korea suffered building collapses from the Pohang earthquake in 2017. The walls of apartments collapsed, the exterior brick finish of a university was removed, the pillars of piloti buildings were damaged and the buildings themselves collapsed (Figure 1). For this reason, the Building Act of Korea strengthened supervision work by adding supervision submission documents upon completion, as well as the building supervision of the 3rd floor or more of piloti buildings in cooperation with structural engineers [1].

In fact, in the Pohang earthquake accident, the piloti buildings collapsed due to the damage to the pillars on the first floor of the buildings, and the structural weakness of the piloti structures along with the omission of rebars during construction are considered the cause of the collapse [2]. Therefore, this legislation was established under the judgment that the structure of the piloti is vulnerable as a structure in response to safety accidents, and at the same time, it has the purpose of preventing insolvent construction by strengthening the contents of the supervision system.

As part of an effort to improve the quality of buildings, the supervision system has been strengthened. However, completing all the items in the supervision system and getting them all down on paper is not an easy task. In the case of the omission of the rebars related to the Pohang Earthquake, even though counting the rebars is mentioned in the checklist of the supervision guide, it is practically difficult in the field. It is not easy for the supervisor to secure the time to count individual rebars at the building site, which becomes more difficult at the site of large buildings. Even if the number is counted, the supervision work is not limited to counting the rebars; so, even if the rebar is an important factor in supporting the stress of the structure, it is difficult to actually perform this task in supervision.

Recently, new technologies of the 4th industry are being researched and utilized in many fields. The development of new technologies is accelerating, and efforts to increase efficiency by utilizing these new technologies are being attempted in many fields. In addition, unmanned-based automation technologies, such as drones and robots with artificial intelligence, are replacing those jobs previously believed to only be possible for humans, and it is not uncommon to see better results than humans. Moreover, in the architectural field, ‘Constructech’, which combines Construction with IT technology, is leading the construction industry upward, which was considered to be a downward trending industry [3], and the premise of this trend is IT-based technology, BIM (Building Information Modeling).

Therefore, in this study, we used these new technologies, focusing on rebar counting, especially the horizontal rebar, known as tie counting, among the various tasks of supervision, to improve the possibility of practical supervision work and improve the efficiency of work, and furthermore, to see the possibility of using convergence technology.

2. Research Framework

2.1. Research Scope

In the construction of reinforced concrete structures, rebars are important elements that support the stress of the building and have a significant effect on the overall structure of the building, which is directly related to the safety and quality of the building. Furthermore, among the supervision items related to rebars, while counting the rebars of columns has an important influence on the structure of the building, the supervisor should do this process. But it is also cumbersome work for the supervisor to check the size and number of each rebar. As the size of the building grows, the number of rebars becomes too large for human resources to handle. Especially with the regard to the tie, or horizontal rebar, the number of the ties is usually larger than the vertical rebars, and the counting of ties is also neglected and not usually done in the field. Therefore, as a basic study of the development of a supervisory checking system, we intended to develop an automatic checking algorithm of column ties. To achieve this purpose, in this study, we tried to develop a technology that could automatically classify the presence or absence of band reinforcing bars in a column and count them. For this, we intended to utilize object detection technology.

2.2. Research Flow

This study attempted to automatically identify the presence or absence of a tie in a column, count the number, then determine whether it coincides with BIM information. Supervision work involves overseeing whether a building is built as planned (Figure 2). Therefore, the main flow of research is to determine whether the two sites are consistent by obtaining information from the construction site using deep learning-based object detection technology and drawing information from the BIM model. Deep learning-based object detection technology classifies the types of objects included in an image among deep learning image processing technologies and detects their locations.

The flow of the study is to build a detection network for the tie, then automatically detect the ties through photographs from the construction site using the network, and compare and check the result with the quality automatically calculated by the BIM model (Figure 2).

A network for automatically detecting the ties in the photos of reinforcement columns is constructed. Make the rebar model and export the quantity of the rebars to an Excel file. The number data determined by the network and the Excel data of the number of the tie output by the BIM are automatically compared by MATLAB. In the research flow, the tie detection network is constructed follows.

• (1). Try 1: Network Training experiments (First detection)

• (2). Try 2: Network Training experiments (Fixed/Variation)

• (3). Try 3: Network Additional Training (Variation)

The process of building a deep learning-based tie detection network starts with learning a lot of data; and, since the performance of the detection network is determined by the combination of the number of training data, the composition layer, and the option values, the number of cases is numerous and the performance is difficult to predict, so the performance of each network must be evaluated by learning the number of each case. Therefore, the minimum criterion for constructing a tie detection network was found first. That is, a combination of a minimum number of learning data, an existing pre-trained network to be used for transfer learning, and various option values for designating a learning method was found, and each item was changed in the combination to investigate the detection performance of the network. This is to distinguish items that need to be fixed and items that can improve network performance by making changes. Next, based on the items and basic values classified in the network construction, the network trainings were repeated while maintaining the values for the fixed items and changing the values for the variable items, the scores were assigned to the networks constructed in ‘Try 3′ to compare accuracy, and a network with a relatively high accuracy was selected and used for comparison and discrimination.

3. Literature Review

3.1. BIM Technology

NBS [4] defines BIM as ‘a process for creating and managing information on a construction project across the project lifecycle’. In short, BIM can be described as the process of managing, exchanging, and sharing construction project information among participants by creating a digital model for the entire life cycle of a building. The process starts with architectural design and includes demolition, the final stage of the building life cycle.

By utilizing BIM in the design phase, masses are designed to fit the concept and surrounding environment, design is performed with 3D modeling, and optimized designs can be derived based on an analysis of energy, structure, etc., through various simulations. This enables real-time stakeholder collaboration. A 3D drawing made in this way not only facilitates the output of planes, elevations, and sections to 2D drawings, but also directly transmits information to a factory for the production of building members, improving the efficiency in manpower and time, while reducing human error.

Before construction, it is possible to shorten the construction time through automation and construction simulation, and this can facilitate the linkage of facilities, structures, and civil engineering, as well as reduce construction damage by reviewing interference in advance, and be used for construction safety. In the maintenance of buildings after construction, BIM can be used to utilize materials and members’ information for building utilization, remodeling, and reconstruction.

It is the trend of the building and construction industry that the use of BIM alone has resulted in the saving of manpower, money, and time in design and construction as well as the saving of risk, but its combination with other technologies is creating greater synergy. Clyde Zhengdao Li et al. [5] tried to reduce the risk of prefabricated house construction (PHC) and improve schedule performance by utilizing the integrating technology of RFID and BIM.

In the integration between BIM and GIS, Fabrizio D′Amico et al. [6] tried to minimize the constraints of the infrastructure and environment by designing the transport infrastructure through BIM and GIS data integration. Moreover, Mohamed Marzouk and Ahmed Othman [7] proposed an integrative way between BIM and GIS to plan and predict the infrastructure demand that is created in line with the creation and expansion of cities in the planning phase of smart cities. Furthermore, Haythem Bahri et al. [8] showed its effectiveness by suggesting the use of the integrated technology of MR, utilizing the advantage of VR, AR, and BIM. As such, the combination of advanced technology and BIM expands more possibilities and leads to efficiency in each field. The combination of Deep-learning technology and BIM is also being tried, but it has been difficult to find any research in the supervision field.

3.2. Deep Learning-Based Object Detection

CNN (Convolutional Neural Network) [9] is a network in which filter-extracted features are combined in the Fully Connected Layer and, finally, classed to output results.

Object detection technology is a technology that utilizes CNN′s technology to detect the location of objects in an input image and to classify classes in those locations using CNN. Starting with R-CNN [10], proposed by Girschick′s research team in 2014, object detection technology emerged. R-CNN uses the Selective Search algorithm for location detection separately from class classification in the input image, and has a process of proposing 2000 candidate regions and calculating them with CNN. Since the proposed 2000 areas are cropped/warped and then calculated with CNN, it takes a lot of time, memory, and lacks accuracy; therefore, SPP-net [11] tried to improve the detection accuracy and speed by calculating the input image itself as CNN without dividing the image. Using the Selective Search algorithm to detect object candidate regions is the same, but the difference is applied to the convolution feature map, which is characterized by being output as three feature vectors in the spatial pyramid pooling layer after the max pooling process.

Fast R-CNN [12] is similar to SPP-net, but instead of spatial pyramid pooling layer of SPP-net, RoI (Region of Interest) pooling is applied. If SPP-net uses three kernels, Fast R-CNN is different in that it adopts a single-scale kernel. The detection speed and accuracy of the object are improved compared to the existing methods. Faster R-CNN [13] is RPN (Region Proposal Net) plus RoI pooling of Fast R-CNN. Until Fast R-CNN, a selective search algorithm was used to estimate the object candidate region, but from Faster R-CNN, the technique for estimating the candidate region also used deep learning, known as RPN. If the previous models estimate the bounding box, and the class is classified after the bounding box is calculated on CNN again, YOLO [14] can find the bounding box and the bounding box class at the same time. The tensor, the core of the YOLO model, finds B bounding boxes for each cell and, at the same time, indicates the class classification probability of C. The performance itself is similar to Faster R-CNN, but it is very fast. The disadvantages are that it cannot detect close or small objects, and the accuracy of the bounding box is also poor.

Next, YOLOv2 [15], used in this study, overcomes the problems of YOLO and improves performance. The speed was increased by using batch normalization, high resolution image use, and the offset of the anchor box. The default size of the anchor box was determined by referring to the bounding boxes in the actual training data. YOLOv2 can be implemented by using input images of various sizes. The lower the resolution image, the higher the speed, but the accuracy is lower. In class classification, hierarchies are used to improve performance by considering upper layers at the same time. The most recent YOLO-related object detection technology is YOLOv3, which was developed to be slightly larger, but faster and more accurate than the existing YOLO network.

YOLO is an algorithm created by a new method outside the existing framework, such as CNN or RNN. CNNs were developed with the idea of visually navigating through images when a cat checks an object. It is composed of several layers of CNN, and it continues to expand by ‘maxpooling’, and extracting specific image parts, and searches for images in this way.

YOLOv2, used in this study, was developed in the process of continuous development by many developers to improve the performance of YOLO, and YOLOv5 is being developed in 2021. Among YOLOv5 models, the ‘s’ attached to YOLOv5s is the algorithm that shows the highest performance among various models of existing YOLOv5.

4. Experiment

4.1. Network for Column Tie Detection

4.1.1. Network Environment

• (1). MATLAB and Yolov2

The program used for training network was MATLAB from Mathworks, and YOLOv2 was used as a detection technique. There are various types of object detection technologies, but the technologies provided by the MATLAB R2019b version are RCNN, Fast R-CNN, Faster R-CNN, and YOLOv2. Among them, excellent performance and high speed YOLOv2 are used to construct the tie detection network.

MATLAB provides an interface to easily build deep learning YOLOv2 networks. Learning through Equation (1), which specifies the training dataset, the layer to be trained, and the options for how to train using MATLAB, returns the learned YOLOv2 network and information about this network.

[Yolo, info] = trainYOLOv2ObjectDetector (TrainingData, Layers, Options)(1)

• (2). Transfer Learning

When training with YOLOv2, the tie detection network was constructed with transfer learning by re-learning a specific layer in an existing trained model rather than creating a new layer from the beginning. Transfer learning can be implemented quickly and simply by using a pre-trained network as a starting point and, by training the network on a new data set, it has the advantage of fine-tuning the deeper layers of the network while also creating a network specific to the user’s own new data set [16].

4.1.2. Network Construction

• (1). Training Data Set

The learning data of the object detection network consists of three types of information. It is an image, a classification label, and a bounding box that displays the location of an object in the image. The bounding box consists of a combination of numbers indicating the positions of four points, and the number of combinations of the numbers increases as much as the number of bounding boxes in the image. In MATLAB, this learning data is called Ground Truth, and it provides an interface to create it using an app called Image Labeler. It is necessary to repeat the process of assigning a label name and drawing its position on the image for each image. In this way, the Ground Truth is displayed as a table showing the path of the image file in the first column and the boundary box position in the image in the second column (Figure 3).

Four methods (Figure 4) were used to construct the training data set. First, a column rebar model was produced to obtain images; second, a rebar image was scrapped from the Internet; third, it was directly photographed at a construction site; and finally, after making the rebar model at Autodesk′s Revit, it was rendered. By acquiring images like this, we tried to collect a large amount and wide variety of data. The data set was classified into B116, C380, E400, G400, and J580, and was constructed as a set with many images as the alphabetical order increased, as shown in Table 1.

• (2). Feature Layers

MATLAB provides a pre-trained network that can be used for transfer learning. The provided networks are 13 existing networks with 3 SeriesNetworks [17], including AlexNet and 10 DAGNetWork [18], which includes GoogleNet. To select the network to use for transfer learning, we refer to the size and parameters of each network (Table 2) and the speed and accuracy.

Considering that the speed and accuracy are relatively high and the size and parameters are small, ResNet50, Inceptionv3, and InceptionResNetv2 networks were selected to use for transfer learning. ResNet50 is the speed and accuracy of the middle of the network. Incetionv3 has half the size and parameters of ResNet101, but has higher performance in terms of speed and accuracy. InceptionResNetv2 has the largest depth and size of the networks, but the accuracy and speed are the best in terms of performance.

• (3). Options and others

Adam was selected as the optimization function. MATLAB provides ‘sgdm’, ‘RMSprop’, and ‘Adam’ as function options to optimize training. Among them, Adam (adaptive moment estimation) is a method that combines the advantages of momentum and RMSprop, and is the most widely used optimization technique. Adam reflects the momentum that gives path efficiency by keeping the gradient′s past change to some extent, and the RMSProp′s feature reflects the latest information to a greater extent than in the past.

The learning rate has a fixed value and a piecewise learning rate that decreases the learning rate by a predetermined magnification as the learning progresses. When the learning rate was fixed, the value was set to be 1.0 × 10⁻⁵, and when decreasing, it started at 0.01 and decreased by 0.1 times for every five epochs. Mini Batch and Max Epoch started at values of 4 and 20, respectively, and the input image was fixed to 500*500 and the anchor box to 10.

4.2. Network Experiment and Accuracy

4.2.1. Network Experiment for Initial Detection

In the first experiment trial, we experimented to find the minimum combination that results can be obtained, that is, that the network can distinguish the ties. When we trained the network, three existing pre-trained networks used for transfer learning (ResNet50, Inceptionv3, InceptionResNetv2), Mini Batch, Max Epoch, the number of training data, and whether to sort the training data were changed and experimented, as shown in Table 3. Ties were detected when the network was trained with more than 116*5 data. At this time, the existing networks used were ResNet50, Inceptionv3, and InceptionResNetv2, the Mini Batch value was 8, Max Epoch was 20, and the training data was 580 (5 times 116 data).

4.2.2. Network Experiment to Find Fixed Items and Variables

In the second trial, in order to select the fixed items and variable items that make up the tie detection network, we tested the success or failure of the network by changing the values of each item one by one. The detection network, which was transitively trained with InceptionResNetv2, had a poor performance in accuracy and speed and, in particular, it took a lot of time to train the data. When the training data was sorted in order by data name, and when the learning rate was decreased according to the learning progress, the tie was not detected, and when the Mini Batch size was increased to 16, network learning was not performed due to insufficient GPU memory. Based on this experiment (Try-1), we tested with ResNet50 (Table 4) and Inceptionv3 (Table 5) and selected fixed and variable items (Table 6).

4.2.3. Additional Network Training Experiment and Accuracy

In order to increase the accuracy of the network, the third experiment was conducted while changing the values of the items in Table 6. The accuracy of each network was measured and compared with 10 test images (Figure 5) that did not participate in the training. Subfigures, ‘a’ to ‘d’ of Figure 5, are 4 rebar- making model photos, and subfigures, ‘e’ to ‘j’ of Figure 5, are 6 on-site photos.

There is an indicator called Precision and Recall as a method of evaluating computer vision performance. The precision is the ratio of the actual ties in the detected ties and the reproducibility is the ratio of the ties detected by the network among the actual ties. In this performance index, precision misses the problem of not detecting the tie and recall misses the problem of the incorrect detection of the tie.

Since the detection of the tie must be evaluated both in position and in number, the accuracy of the network was compared with the following method. When the network detected the tie correctly, it was given points, and if incorrect, points were deducted; the results of the four rebar-making model photos were given one point per tie, and two points per on-site photo were given if the results of the detection match the actual photo. This is to differentiate the rebar-making model photos from the on-site photos because the network was trained through similar photographs to the rebar-making-model photos, although they did not participate in network learning directly. Table 7 is about the additional network trainings and the accuracy of each.

4.3. Calculation of Tie Quantity from BIM Model

4.3.1. Column and Rebar Modeling

The rebar-making model was modeled with Revit. The height of the floor was 1500, and this model was reinforced at intervals of HD10@150 from the bottom to the height 600, and HD10@300 from 600 to 1500. At this time, rebar reinforcement was done with ‘Naviate’ provided as an Add-In in Revit. The rebar modeled by Revit was output as Excel data, as shown in Figure 6. In the Excel data, 10 M of Type represents the diameter of the tie, and 22 M represents the diameter of the vertical rebar.

4.3.2. Comparing the Quantity of Ties from BIM with the Ties Identified in the Image

The MATLAB of Mathworks was used to detect and measure the ties of the column and to compare and discriminate the BIM output. At first, we imported a photo taken in the building site into MATLAB, and MATLAB detected the ties in the photograph. Next, we imported the tie quantities Excel file into MATLAB. Finally, we compared the number of ties detected in the photo with the number of ties in the Excel file. In Figure 7, DQ stands for Detected Quantity and means the number of ties detected in the photo, and MQ stands for Modeling Quantity and means the number of ties in BIM. For example, the number of ties detected in the second picture is five, but the number of ties that should actually be seven, and this result indicates that there is a lack of two ties.

5. Development and Validations

5.1. Pre-Processing for Improving Detection Results and Accuracy

The network showed an accuracy of 92.85% for an image similar to the training data (as shown in Figure 8), and this network also distinguished wall and column rebars for the new images and distinguished vertical rebars and ties within columns.

For entirely new images not related to training, when the same column was targeted, it was possible to confirm the difference in detection performance according to the photographing distance (Figure 9). It was confirmed that the detection performance was affected by the tilt of the object, the sharpness of the image, and the surrounding brightness.

Based on this, we tried to improve the detection performance by adjusting the photographic perspective and brightness before detection. We tried to solve the perspective issue by increasing the proportion of the columns in the whole photo through the photo cropping operation. The pre-processing was performed by using the function of automatic brightness correction and other enhancement functions for photos that are difficult to distinguish due to backlight with Photoshop.

5.2. Deep Learning-Based Smart Supervision Checking System

5.2.1. Building Site and Pre-Processing of Images

To apply the implemented technology, one construction site was selected and partially applied to one floor of the building. Figure 10 is the column plan, Table 8 is the column specifications.

5.2.2. Application of Tie Detecting Technology Based on Deep-Learning (Validation)

For comparison and matching check, the on-site photos, detection network, quantity Excel file, and MATLAB code were prepared and executed in MATLAB. The result, as shown in Figure 11, was obtained in 7.75 s, and accuracy of 95% or more was confirmed (Table 9).

6. Conclusions

The conclusions obtained through the series of processes performed in this study are as follows:

• (1). The possibility of smart detection of column reinforcement was seen by automatically classifying the column rebars and counting the number automatically. By using deep learning technology, it was possible to achieve the purpose of the study to classify and count the ties by separating the wall rebars and the column rebars among the numerous rebars at the construction site, and also by distinguishing the vertical and horizontal rebars from the columns;

• (2). By using the proposed technique in the supervision work, it was possible to perform the task of counting the ties, which is difficult to actually perform. In order to increase the versatility and accuracy of the built network, image pre-processing was performed. By obtaining a high accuracy of 95.24%, it was possible to see the possibility of utilization in the actual supervision work, and the possibility of improving the efficiency of the supervision work by obtaining a time-saving effect through the application of a process that could automatically check whether or not it matches the information of the BIM;

• (3). It was confirmed that by partially using it at the construction site, the time was reduced by more than a few tenths of that of a manpower. This will save more time and manpower if it is used in large buildings that take a significant amount of construction time;

• (4). The possibility of using BIM and deep learning convergence technology could be confirmed by suggesting a process of checking whether construction site information and BIM information match.

This study was conducted for the purpose of increasing the efficiency of supervision work by utilizing the technology of deep learning, and it was possible to confirm the possibility of its use as a basic study of deep learning convergence technology in the BIM environment. Based on this possibility, we would like to suggest one step pre-processing programming and real-time drone object detection as future work to increase the usability as a real technology. Through the future work, this study is expected to be used in the real field. Therefore, it is considered to be research that has sufficient academic and practical development potential.

Author Contributions

T.K. conceived experiments, analyzed data, and wrote papers; S.H. investigated prior research and edited thesis; S.C. supervised the research. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures and Tables

Figure 1. Damage from the Pohang earthquake.

Figure 2. Research flow.

Figure 3. Labeling using Image Labeler.

Figure 4. Training data sources.

Figure 5. Test images.

Figure 6. Rebar modeling through ‘Naviate’.

Figure 7. Detection and result.

Figure 8. Detection performance of similar images involved in training.

Figure 9. Detection performance by distance.

Figure 10. Column plan and column list.

Figure 11. Detection and identification result.

Table 1

Training data Set.

Training Data Set	A38	B116	C380	E400	G400	J580
(a) Rebar-Making-model	38	8	175	184	179	176
(b) Internet	-	29	58	58	58	200
(c) On-Site	-	79	139	139	139	177
(d) Revit Model Rendering	-	-	8	19	24	27
Data Number	38	116	380	400	400	580

Table 2

Pretrained network data from ‘kr.mathworks.com/help’ [16].

Network	Depth	Size (MB)	Parameter (Unit: Million)
alexnet	8	227	61.0
vgg16	16	515	138
vgg19	19	535	144
squeezenet	18	4.6	1.24
googlenet	22	27	7.0
inceptionv3	48	89	23.9
densenet201	201	77	20.0
mobilenetv2	54	13	3.5
resnet18	18	44	11.7
resnet50	50	96	25.6
resnet101	101	167	44.6
xception	71	85	22.9
inceptionresnetv2	164	209	55.9

Table 3

Try 1: network training experiment for initial detection.

FeatureNetwork	ResNet50				Inceptionv3				InceptionResNetv2
Data Numbers	A38	B116	B116	B116*5	A38	B116	B116	B116*5	A38	B116	B116*5
Max Batch	20	20	20	20	20	20	20	20	20	20	20
Mini Batch	4	4	8	8	4	4	8	8	4	4	8
Initial Rate	1.0 × 10−5	1.0 × 10−5	1.0 × 10−5	1.0 × 10−5	1.0 × 10−5	1.0 × 10−5	1.0 × 10−5	1.0 × 10−5	1.0 × 10−5	1.0 × 10−5	1.0 × 10−5
Sort/Shuffle	shuffle	shuffle	shuffle	shuffle	shuffle	shuffle	shuffle	shuffle	shuffle	shuffle	shuffle
Duration	0:04:02	0:09:52	0:08:22	0:43:06	0:04:04	0:15:37	0:09:01	0:44:47	0:10:28	0:28:26	3:35:24
Result	fail	fail	fail	part success	fail	fail	fail	part success	fail	fail	partsuccess

Table 4

Try 2: network training experiments to find fixed/variable items for ResNet50.

FeatureNetwork	ResNet50 (Data Numbers: B116*5)
Max Epoch	20	20	20	20	20	30
Mini Epoch	8	8	8	16	8	8
Initial Rate	1.0 × 10−5	1.0 × 10−5	piecewise	1.0 × 10−5	1.0× 10−5	1.0 × 10−5
Sort/Shuffle	shuffle	sort	shuffle	shuffle	shuffle	shuffle
Duration	0:43:06	0:43:54	0:43:57	-	0:26:50	1:04:26
Result	Part success	fail	fail	fail(GPU memory)	partsuccess	partsuccess

Table 5

Try 2: network training experiments to find fixed/variable items for Inceptionv3.

FeatureNetwork	ResNet50 (Data Numbers: B116*5)
Max Epoch	20	20	20	20	20	30
Mini Epoch	8	8	8	16	8	8
Initial Rate	1.0 × 10−5	1.0 × 10−5	piecewise	1.0 × 10−5	1.0 × 10−5	1.0 × 10−5
Sort/Shuffle	shuffle	sort	shuffle	shuffle	shuffle	shuffle
Duration	0:44:47	0:44:05	0:44:48	-	0:29:16	1:05:34
Result	part success	fail	fail	fail(GPU memory)	part success	part success

Table 6

Fixed and variable Items.

Fixed Items		Variable Items
Input size	500*500	Feature Layers	ResNet50/Inception v3
Optimizer	Adam	Max Epoch	More than 20
Mini batch	8	Training Data Set	More than 116*5
Initial rate	1.0 × 10−5
Sort/Shuffle	Shuffle

Table 7

Try 3: additional networking training experiment.

Feature Network	Data Number	Max Epoch	Duration	Score
ResNet50	C380	30	1:21:42	27.5
	C380*4	30	3:06:49	28
	C380*5	30	4:00:47	29.5
	E400*5	30	4:04:40	28
	E400*6	30	5:09:22	28
	E400*10	20	5:26:33	29.5
	E400*10	30	8:12:08	36.5
	G400*5	30	4:05:23	35
	G400*6	30	4:51:54	35
	G400*10	20	5:23:40	37
	J580*5	30	5:39:18	69.5
	J580*10	30	11:12:49	46
	J580*5	50	9:29:10	52
	J580*5	100	19:01:50	61
Inceptionv3	C380	30	0:48:04	27.5
	C380*4	30	3:14:56	25.5
	C380*5	30	3:53:08	46
	E400*5	30	4:15:26	34.5
	E400*6	30	5:03:30	20
	E400*10	20	5:32:40	28
	E400*10	30	8:14:40	26.5
	G400*5	30	4:14:21	33
	G400*6	30	4:53:29	35.5
	G400*10	20	5:35:52	32.5
	J580*5	30	5:37:59	62.5
	J580*10	30	11:20:00	41.5
	J580*5	50	9:30:13	42
	J580*5	100	18:01:50	39

Table 8

Column specifications.

Location	X1Y1	X1Y3	X2Y1	X2Y3	X3Y1	X3Y2
Column Type	C2	C1	C1	C1	C2	C2
Height	3450	3450	3450	3450	3450	3450
Girder Depth	800	700	800	700	550	400
Top/bottom Tie	HD13@150	HD13@150	HD13@150	HD13@150	HD13@150	HD13@150
Middle Tie	HD13@150	HD13@250	HD13@250	HD13@250	HD13@150	HD13@150

Table 9

Error and accuracy of each column detection.

Location	X1Y1	X1Y3	X2Y1	X2Y3	X3Y1	X3Y2	Sum
DQ	18	15	16	16	19	20	104
MQ	18	16	16	16	19	20	105
Error	0	1	1	2	0	1	5
Accuracy (%)	100	93.75	93.75	87.5	100	95	95.24

References

1. The Enforcement Decree of the Building Act (Article 91-3 (1) 5 and 6) (Cooperation with Relevant Specialized Engineers). Available online: http://www.law.go.kr/LSW/eng/engLsSc.do?menuId=2&section=lawNm&query=building+act&x=34&y=24#EJ91:3 (accessed on 29 April 2022).

2. Joongang Daily. Available online: https://news.joins.com/article/22123714 (accessed on 29 April 2022).

3. Mckinsey Global Institute. Reinventing Construction: A Route to Higher Productivity; McKinsey Global Institute: South Tower, Singapore, 2017.

4. NBS: Integrated Global Platform for Everyone Involved in the Design, Supply and Construction of the Built Environment. Available online: https://www.thenbs.com/ (accessed on 29 April 2022).

5. Li, C.Z.; Zhong, R.Y.; Xue, F.; Xu, G.; Chen, K.; Huang, G.G.; Shen, G.Q. Integrating RFID and BIM technologies for mitigating risks and improving schedule performance of prefabricated house construction. J. Clean. Prod.; 2017; 165, pp. 1048-1062. [DOI: https://dx.doi.org/10.1016/j.jclepro.2017.07.156]

6. D’Amico, F.; Calvi, A.; Schiattarella, E.; Prete, M.D.; Veraldi, V. BIM And GIS Data Integration: A Novel Approach of Technical/Environmental Decision-Making Process in Transport Infrastructure Design. Transp. Res. Procedia; 2020; 45, pp. 803-810. [DOI: https://dx.doi.org/10.1016/j.trpro.2020.02.090]

7. Marzouk, M.; Othman, A. Planning utility infrastructure requirements for smart cities using the integration between BIM and GIS. Sustain. Cities Soc.; 2020; 57, pp. 102-120. [DOI: https://dx.doi.org/10.1016/j.scs.2020.102120]

8. Bahri, H.; Kracmarik, D.; Moezzi, R.; Kočí, J.K. Efficient Use of Mixed Reality for BIM system using Microsoft HoloLens. IFAC Pap.; 2019; 52, pp. 235-239.

9. Krizhevsky, I.; Sutskever, I.; Hinton, G.E. ImageNet Classification with Deep Convolutional Neural Networks. NIPS 2012, Harrahs and Harveys, Lake Tahoe, USA. Mon.–Sat. (3–8 December 2012). Available online: https://papers.nips.cc/paper/2012 (accessed on 29 April 2022).

10. Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. Rich feature hierarchies for accurate object detection and sementic segmentation. arXiv; 2014; arXiv: 1311.2524v5

11. He, K.; Zhang, X.; Ren, S.; Sun, J. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition. arXiv; 2015; arXiv: 1406.4729v4[DOI: https://dx.doi.org/10.1109/TPAMI.2015.2389824] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/26353135]

12. Girshick, R. Fast R-CNN. arXiv; 2015; arXiv: 1504,08083v2

13. Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. arXiv; 2015; arXiv: 1506.01497v3[DOI: https://dx.doi.org/10.1109/TPAMI.2016.2577031] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/27295650]

14. Redmond, J.; Divvala, S.; Girshick, R.; Farhadi, A. You Only Looks Once: Unified, Real-Time Object Detection. Proceedings of the CVPR 2016; Las Vegas, NV, USA, 27–30 June 2016.

15. Redmon, J.; Farhadi, A. YOLO9000: Batter, Faster, Stronger. arXiv; 2017; arXiv: 1612.08242v1

16. Pre-Trained Deep Neural Networks (Transfer Learning). Available online: https://kr.mathworks.com/help/deeplearning/ug/pretrained-convolutional-neural-networks.html (accessed on 29 April 2022).

17. SeriesNetworks. Available online: https://kr.mathworks.com/help/deeplearning/ref/seriesnetwork.html (accessed on 29 April 2022).

18. DAGNetWork. Available online: https://kr.mathworks.com/help/deeplearning/ref/dagnetwork.html (accessed on 29 April 2022).