Vision-Based Structural FE Model Updating Using Genetic Algorithm

Abstract

Structural members can be damaged from earthquakes or deterioration. The finite element (FE) model of a structure should be updated to reflect the damage conditions. If the stiffness reduction is ignored, the analysis results will be unreliable. Conventional FE model updating techniques measure the structure response with accelerometers to update the FE model. However, accelerometers can measure the response only where the sensor is installed. This paper introduces a new computer-vision based method for structural FE model updating using genetic algorithm. The system measures the displacement of the structure using seven different object tracking algorithms, and optimizes the structural parameters using genetic algorithm. To validate the performance, a lab-scale test with a three-story building was conducted. The displacement of each story of the building was measured before and after reducing the stiffness of one column. Genetic algorithm automatically optimized the non-damaged state of the FE model to the damaged state. The proposed method successfully updated the FE model to the damaged state. The proposed method is expected to reduce the time and cost of FE model updating.

1. Introduction

Recent earthquakes around the world have shown how earthquake disasters can wreak havoc in society. For example, an earthquake in Taiwan in 1999 resulted in over 10,000 casualties and economic losses of over USD 10 billion. The magnitude 7 earthquake in Haiti in 2010 caused tens of thousands of casualties, and the collapse of social infrastructure resulted in huge economic losses. The earthquake and tsunami that occurred in Japan in 2011 not only killed more than 10,000 people, they also caused a major radioactive leak owing to the explosion of the Daiichi nuclear power plant. The earthquake in Mexico in 2017 caused more than 1000 casualties, and a number of skyscrapers in Mexico City, the capital city of Mexico, were severely damaged [1].

Structures can be damaged through seismic loadings, and the conditions of such structures need to be determined to ensure their safety. The current practice for inspecting damaged structures requires engineers to directly evaluate the damage by viewing the structure by sight. However, this assessment can be subjective and might expose the engineers to risk. In recent years, vibration-based Structural Health Monitoring (SHM), a method for monitoring the health of such structures by measuring the response of the structures, has recently entered the spotlight. SHM aims to ensure the safety of civil infrastructure by measuring its dynamic response.

The traditional SHM system measures the structural response by installing wired sensors such as accelerometers and displacement transducers, e.g., the Alamosa Canyon Bridge, I-10 Bridge, Hakucho Bridge in Japan, Bill Emerson Memorial Bridge in Missouri, and the Tsing Ma Bridge in Hong Kong [2,3,4,5,6]. However, the wired SHM system cannot be easily deployed to existing structures, and is costly. To overcome these limitations, wireless sensor technology has been introduced. Westermo and Thompson presented a sensor that can evaluate the condition of the structure, and Pines and Lovell discussed a sensor and wireless communication technology that remotely monitors the health of large civil structures. Krantz et al. developed a micro-sensor that can retrieve data from an embedded strain gauge, and Oshima et al. proposed a monitoring system using a mobile phone [7]. Although enhancements in wireless sensor technology could reduce the cost of SHM systems, these methods still require manual installation of the sensors. Computer vision technology has recently been applied to SHM, overcoming the limitations of wired and wireless systems. Jiang et al. [8] concluded that close-range photogrammetry technology is a powerful measurement method that can provide unique solutions for a variety of bridge engineering applications. Kwiatkowski et al. [9] compared and evaluated the usefulness of various measurement techniques (terrestrial laser scanning (TLS), tachymetry, photogrammetry) applied to establish the behavior of suspension bridges under various loads. Gawronek et al. [10] measures the vertical displacement of railroad bridges using terrestrial laser scanning (TLS) technology, and confirmed that this technology can provide reasonable results for quick assessment of railway infrastructure. Kwiatkowski et al. [11] confirmed that it is possible to obtain a 3D model of a structure with satisfactory accuracy through TLS and photogrammetry. Porras-Amores et al. [12] presented the most efficient method for generating 3D models of underground structures using TLS techniques. Shih and Sung [13] conducted a study to extract the displacement of the structure and characterize the vibration mode of the cantilever beam using digital image correlation technology. Feng et al. [14] and Yoon et al. [15] confirmed the reliability of a vision-based system for system identification by comparing vision-based and wired sensor systems. Furthermore, a method used to measure the structural displacement from an unmanned aerial vehicle (UAV) was proposed by Yoon et al. [16]. In addition, Lim and Yoon [17] presented a motion sensing technique using vision-based motion sensing to acquire the location information of the real-time dynamic load for pedestrian suspension bridges. Although these vision-based methods have made significant improvements in measuring the displacement or conducting system identification, a study for applying computer vision technology to FE model updating has yet to be conducted.

This study aims to develop a framework for computer-vision based structural finite element (FE) model updating. Various computer vision algorithms, including Kanade–Lucas–Tomasi (KLT), multiple instance learning (MIL), kernelized correlation filter (KCF), and minimum output sum of squared error (MOSSE) are applied to measure the seismic loads and the corresponding response (displacement) of the structures. Genetic algorithm was applied to optimize the FE model of the structure into an as-is damaged model, and to predict the stiffness reductions for all members. To validate the proposed system, a lab-scale test using a three-story building model was conducted. In Section 4.4, the validation results showed that the proposed method can update the FE model to an as-is damage state, within an accuracy of approximately 80%. The proposed method is expected to reduce the time and cost of FE model updating [15].

2. Background

In this study, vision-based structural model updating was performed based on the following three topics.

• Tracking algorithm.

• FE model updating.

• Genetic algorithm.

In this chapter, brief concepts of various tracking algorithms, finite element model updating, and genetic algorithms are described.

2.1. Tracking Algorithm

The KLT tracker was introduced by Lucas and Kanade and extended by Tomasi and Kanade. The KLT tracker tracks an object in two simple steps. First, it finds the traceable feature points in the first frame, and then uses the calculated displacement to track the features detected in the next frames. The feature points of an object are tracked using an optical flow tracker [18]. The channel and spatial reliability (CSRT) tracker was proposed by LuNežič et al. [19], and improves the discriminative correlation filter (DCF) tracker by introducing spatial and channel reliability. The spatial reliability map was used to resize the filters to find optimal filter sizes by taking advantage of non-rectangular targets. The KCF tracker proposed by Henriques et al. [20] is a tracker that extends the MOSSE tracker. The KCF tracker improves the tracking accuracy by extracting the histogram of gradient (HoG) function, but requires more data. Therefore, the data are calculated in the Fourier domain to speed up the data processing procedure. The MedianFlow tracker proposed by Kalal et al. [21] tracks the point of each object forward and backward, and compares the results of the trajectory. If the object is tracked correctly, the trajectory forward and backward will yield the same results; otherwise, a tracking error will indicate the opposite value. The error was used to track points inside the bounding box and classify the inliers and outliers. The MIL tracker proposed by Babenko et al. [22] conducted object tracking similar to the online boosting approach. The MIL used a set of image patches (called bags) instead of a single sample for training. For classification, the MIL uses a multi-instance learning approach that considers object bags by grouping similar samples into such bags. A bag containing more than one positive example was called a positive bag, which is otherwise called a negative bag. This technique not only prevents the MIL tracker from losing important information, it also prevents mislabeling problems. The MOSSE tracker proposed by Bolme et al. [23] achieved high efficiency by calculating the correlation in the time domain. The correlation filter improved the efficiency for scaling, rotation, deformation, and occlusion problems from the traditional approach. The MOSSE is more efficient than the other correlation filter-based trackers because the target does not need to be at the center of the image initially. The TLD tracker proposed by Kalal et al. [24] was a technique for long-term tracking and consisted of three parts: tracker, learner, and detector. The tracker aimed to track the objects in successive frames, and the learner used the P-expert and N-expert to estimate false positives and false alarms to update the detector. In addition, the detector finds targets based on the appearance model, provides an output to the learner, and modifies the tracker if necessary. The goal of the TLD is to improve the robustness of the tracking by disabling the online learning when an object is out of the frame or completely obscured by another object, preventing learning through misinformation. In addition, the detection component allows the TLD to redetect the object if it later appears in the video again.

2.2. FE Model Updating

The finite element method (FEM) is the most widely used method for civil engineering and other applications. A finite element method is characterized by a variation formulation, a discretization strategy, one or more solution algorithms, and post-processing procedures [25,26,27]. In this study, model updating was applied that the concept of reverse engineering to the finite element method. FE model updating is the process of ensuring that finite element analysis results in models that better reflect the measured data than the initial models. It is part of verification and validation of numerical models. In order to facilitate model updating, a method to find the properties of the member must be presented. Genetic algorithm is one of the most popular and powerful optimization algorithms, and thus is being applied to various applications including FE model updating.

2.3. Genetic Algorithm

The genetic algorithm is based on the biogenetics of the natural world, and is a parallel and global search algorithm, based on Darwin’s theory of survival of the fittest. Genetic algorithms produce better and better solutions by expressing the possible solutions to the problem to be solved in a data structure of a fixed form and then gradually transforming them. Here, the data structure representing the solutions is genes, and the process of making better and better solutions by modifying them can be expressed as evolution.

Genetic algorithms are more of an approach to solving a problem than an algorithm to solve a specific problem, and can be applied to any problem that can be expressed in a format that can be used in genetic algorithms. In general, when a problem is too complex to be computed, it can be approached through a genetic algorithm as a way to obtain an answer close to the optimal solution, even though the actual optimal solution cannot be obtained. In this case, it does not perform as well as an algorithm optimized to solve the problem, but it can provide an acceptable level of solution in most cases.

3. System Development

Figure 1 shows an overview of the proposed FE model updating method described in this study. The FE model updating method can be composed of two main components: (i) vision-based displacement measurements and (ii) FE model updating, and each chapter explains the main components in detail.

3.1. Vision-Based Dis Placement Measurement

This process demanded four consecutive steps, shown in Figure 1: (i) region of interest (ROI) selection, (ii) feature detection, (iii) detected feature tracking, and (iv) tracking outlier removal. Here, the ROI represents the location of the object that needs to be tracked. Each ROI contains several functions that are typically used for displacement measurement purposes. The ROI is selected manually by drawing a box in the first frame of the video.

Object tracking is a computer vision technique used to track the motion of an object in an image captured by a general camera through real-time or post-processing [16]. Numerous object tracking algorithms have been introduced to track an object including KLT, CSRT, KCF, MedianFlow, MIL, MOSSE, and TLD. Among these tracking algorithms, a few methods have been applied for structural health monitoring purposes; CSRT has been applied to human motion tracking [17], and KLT has been applied to track the dynamic behavior of various structures [14].

This study followed the vision-based displacement measurement system proposed by Yoon et. al. [14], expanded the system to be applied for FE model updating, instead of system identification. In addition, this study applied six additional tracking algorithms (KLT, CSRT, KCF, MedianFlow, MIL, MOSSE, and TLD) in addition to the KLT tracker, which was used in the original system. To measure the displacement of a structure using tracking algorithms, the first step is to calibrate the camera. There are two main reasons for camera calibration: (1) the removal of the radial distortion and tangential distortion, and (2) to compensate the camera ego motion. Camera lenses have tended to use very wide-angle lenses. However, these lenses intentionally introduce radial distortion to increase the field of view. In addition, the camera can move during the measurement, which can result in significant errors for displacement measurements [16]. The extrinsic camera matrix, which contains the translation and rotation information of the camera, needs to be estimated to compensate for the ego-motion of the camera. Therefore, the camera must be calibrated to minimize these distortions and achieve accurate displacement measurements.

Once the camera calibration is conducted, the video is analyzed frame by frame to determine the dynamic response of the structure. This process can be summarized in four steps [14]: (1) selecting a region of interest (ROI), (2) detecting the features, (3) tracking the detected features, and (4) removing tracking outliers, as shown in Figure 2. Here, the region of interest (ROI) indicates the object that needs to be tracked and manually selected in the first frame of the video.

3.2. Structural FE Model Updating Using Genetic Algorithm

An FE analysis has been considered a powerful tool for decades, and is capable of simulating the behavior of the structures. However, creating accurate FE models is not an easy task. Various modeling techniques and guidelines are used to make the appropriate selection of the element type (beam, plate, or solid), degrees of freedom, and analysis methods. These modeling techniques depend on the skills and experience of the engineer, and the purpose of the model (e.g., a static FE analysis and a dynamic FE analysis require different FE models for the same accuracy). The initial FE model, created on the basis of the techniques described above, can accurately predict the initial (undamaged) state of the structure. However, as the structure ages, the FE model created in the design stage does not reflect the characteristics of the as-is structure. Therefore, the FE model updating technique is used to reflect the as-is (damaged) state of the structure. The FE model updating procedure involves three steps: (1) measuring the response as the reference data, (2) selecting the parameters to update, and (3) tuning the model. Prior experience has shown that measuring the accurate response and selecting the model parameters are the key components for successfully updating the FE model [28].

The genetic algorithm is an optimization algorithm rooted in the principles of biological evolution, applying the natural law that individuals with superior traits produce superior offspring. The genetic algorithm is quite different from classical optimization algorithms. Traditional optimization methods use derivatives and search for the objective function of all points in a limited space one at a time, or select an arbitrary point to start the search. However, the genetic algorithm conducts a directional search and a probability search without using the concept of derivatives. This leads to an advantage of solving optimization problems, including continuous–discontinuous mixing, discontinuity, and non-convex regions. The genetic algorithm optimizes the problem by taking a population of individuals consisting of a combination of binary numbers and performing three processes: selection, crossover, and mutation as shown in Figure 3 [29].

In this study, the natural frequency of the structure was used as the objective function, which can best describe the dynamic characteristics of the structure. Decision variables were considered as the cross-sectional area and the thickness of the structure members, and the objective function, decision variables, and constraint set for the discrete optimization problem are shown in Equations (1)–(3).

(1)$Minimize :W= \sqrt{{{\displaystyle \sum }}_{i=1}^n\frac{{\left({\overline{\omega }}_i-{\omega }_i\right)}^2}{n}}$

(2)$Subject to :0< A_{obj} \le A_0$

(3)$0< t_{obj} \le t_0$

In Equations (1)–(3), ${\overline{\omega }}_i$ is the ith natural frequency of the structure of the FE model, and ${\omega }_i$ is the ith natural frequency of the structure obtained from the experiment. In addition, $A_0$ is the cross-sectional area of the undamaged column, $A_{obj}$ is the cross-sectional area of the damaged column, $t_{obj}$ is the thickness of the damaged column, and $t_0$ is the thickness of the undamaged column.

Crossover and mutation are as important items as the objective function in the genetic algorithm (GA). Crossover specifies how the genetic algorithm combines two parents to form a crossover son for the next generation and mutation specifies how the genetic algorithm randomly alters individuals in a population to create a mutant son. Mutation provides genetic diversity and enables the genetic algorithm to search a broader space.

In this study, because the wall stiffness was set as an input parameter, individuals of each generation were set for six. The number of individuals constituting the first generation was 200 for each of the wall stiffness values, and after each individual from generation “0” was simulated, the objective function W for the natural frequency of the structure was evaluated. The best individuals of this generation “0” were those with the lowest value of the objective function $W$, and the next generations (Generation 1 and subsequent generation) were generated by crossover and mutation as described below.

In order to compose the parents of the new generation, 5% of individuals with the lowest objective function W value of the previous generation were selected, and 80% of individuals were generated by crossover of the selected parents for a new generation. At this time, the crossover was made by the following method. For example, if p1 and p2 are parents, then p1 and p2 can be expressed as follows:

p1 = [433,111; 353,778; 380,222; 327,333; 433,111; 10000]p2 = [504,749; 36,379; 417,040; 10,000; 142,329; 596,282]

and the binary vector is [1 1 0 1 0 0], the function returns the following son:

Son = [433,111; 353,778; 417,040; 327,333; 142,329; 596282]

Excluding the 5% with the lowest objective function W value and the 80% obtained through crossover, the remaining 5% were obtained through mutation. The values of individuals generated by mutations should not excessively exceed the range of individual values for each generation. Thus, the mutation adds a random number taken from a Gaussian distribution with mean 0 to each entry of the parent vector. The reason for generating individuals by mutation was to find new individuals in areas that have not been previously applied. This procedure was individually repeated for the wall stiffness for several generations until the objective function W reached a value within error tolerance or the number of generations reached a set value [30,31,32]. Figure 4 shows how the fitness of the objective function changes through the genetic algorithm used in this study. Initially, it can be seen that the objective function rapidly decreases, but it can be seen that the value changes little after about 150 generations. This means that after the 150th generation, the individual constituting the wall stiffness has found an appropriate range to satisfy the objective function.

4. Validation Test

In order to verify the proposed method in this paper, shaking table test of the lab-scale model and model updating of the FE model were performed. The displacement of the structure was measured using a vision-based technique to obtain the result of the shaking table test of lab-scale model, and the natural frequency of the structure was calculated based on the test result. Model updating was performed by applying the natural frequency of the structure calculated through the test as the objective function of the genetic algorithm. The validation of the proposed method was confirmed by comparing the estimated stiffness of the wall using FE model updating with the theoretical stiffness of the wall.

4.1. Test Setup

To validate whether the proposed method can apply FE model updating accurately, a lab-scale validation test was conducted. A three-story building model with equally distributed masses connected by aluminum columns was chosen as the validation structure. The total height of the structure is 920 mm and the width is 240 mm. In order to generate large displacement of the structure when excitation load is applied, the thickness of the wall is 1.2 mm. Figure 5 shows an overview of the lab-scale validation test. To validate the performance of the FE model updating, this study conducted an experiment twice: (1) using an undamaged structure, and (2) by changing one of the aluminum columns to a damaged column. The column between the first and second floors was assumed to be damaged by drilling five holes into the column. The model was attached to a unidirectional shaking table with a maximum displacement of 100 mm. Band-limited white noise (BLWN) was adopted as an input motion to provide energy within a wide range of frequencies. Consumer-grade cameras are used to record the displacement output of the model and the shaking table. These cameras are oriented perpendicular to the motion axis as shown in Figure 5. A smartphone camera (Samsung GALAXY S8) was used to record the video with a frame rate of 60 fps and a resolution of 1080p (1920 pixels × 1080 pixels). In order to improve the accuracy in detecting features necessary for displacement measurements, a fixed white background is used behind the experiment setup. The recorded video was converted into grayscale to improve the accuracy of detecting the features required for displacement measurements.

4.2. FE Model

The FE model for the validation test is as shown in Figure 6. A simple frame model was used [33,34,35,36] to validate the proposed vision-based FE model updating method. The element used in the FE model is the beam type, and the material properties are assumed to be elastic, and the basement and slab were applied steel (Unite Weight: 7.85 t/m3, E: 210 GPa, Poisson’s Ratio: 0.3), and the wall was applied stainless steel (Unite Weight: 7.90 t/m3, E: 201 GPa, Poisson’s Ratio: 0.3). The boundary condition of the FE model was assumed to be fixed at the lower part of the structure, and Cholesky factorization was used to calculate the natural frequency of the structure.

In finite element analysis, the size of the element is an important factor affecting the results. Íñiguez-Macedo et al. [37] analyzed the sensitivity of the element size applied to the finite element analysis to the analysis in a study proposing a methodology for automatically determining the most appropriate Ci constants for modeling the behavior of a group of elastomers. Therefore, this study also confirmed the effect of element size on the calculation of the natural frequency of the structure. The size of the element was calculated by applying the sizes of 1/2, 1/4, 1/8, and 1/16 based on the FE model shown in Figure 6 and the results are as shown in Table 1.

As a result of the sensitivity analysis, it was found that the FE model applied in this study did not significantly affect the natural frequency of the structure even if the structural analysis was performed while reducing the size of the element. Therefore, it was confirmed that the FE model shown in Figure 6 has a sufficient element size to perform model updating.

4.3. Object Tracking Results

The shaking table test was conducted based on the described test setup, and recorded videos were analyzed using seven different algorithms (KLT, CSRT, KCF, MedianFlow, MIL, MOSSE, and TLD). The tracking video with the region of interest (ROI) area is shown in Figure 7a. As a result, KLT (Figure 7b), KCF (Figure 7d), MIL (Figure 7f), and MOSSE (Figure 7g) successfully tracked the object during the shaking table test. However, the CSRT (Figure 7c), MedianFlow (Figure 7e), and TLD (Figure 7h) algorithms showed that the ROI deviated from the initially designated object.

Figure 8 shows the time–displacement curve of the KLT algorithm that succeeded in tracking (Figure 8a) and the TLD algorithm in which the ROI deviated from the initially designated object (Figure 8b). The time–displacement curve of the CSRT algorithm in which the ROI deviated from the initial designated object can confirm that the displacement was measured abnormally.

The frequency response function (FRF) was calculated, as shown in Figure 9, using the displacement measured by the four object tracking algorithms that successfully tracked the object. Based on the FRF calculated using the four object tracking algorithms, the dynamic characteristics of the structure were confirmed, the results of which are shown in Table 2. The dynamic characteristics of the structure shown in Table 2 are used as the fitness function in the FE model updating, and the details are described in the next section.

4.4. FE Model Updating

In this section, the FE model updating technique is applied to a three-story model to which BLWN input motion is applied. The natural frequency of the structure analyzed by the four object tracking algorithms based on a recording using a smartphone (Samsung GALAXY S8) was adopted for the fitness function in the FE model updating. The FE model applied to the model updating is shown in Figure 9a, and the stiffness of the structure at the design stage and the stiffness of the damaged structure are shown in Figure 10b,c, respectively. The FE model was created based on the structure used in the lab-scale test, and it was composed of 10 nodes and 12 frame elements under a fixed boundary condition.

Model updating was conducted by applying the results of Section 3.2 and the FE model described above. The stiffness of each floor of the structure was estimated through model updating, and the stiffness of each floor of the structure estimated for each algorithm is shown in Table 1 and Figure 9d–g. As a result of the FE model updating, the four object tracking algorithms were confirmed to be able to search for the damaged location of the structure. This can be confirmed through the stiffness of the second floor of the structure, which was estimated to be relatively small compared to the stiffness of the other floors. For the estimated stiffness of the undamaged floor, it was confirmed that the four object tracking algorithms showed an error of approximately 5–7%. In addition, for the estimated stiffness of the damaged floor, it was confirmed that the four object tracking algorithms showed an error of approximately 18–21%. It was confirmed that the damaged floor of the structure showed a relatively low accuracy compared to the undamaged floor. However, owing to the characteristics of the finite element model, the lab-scale model cannot be fully simulated; therefore, an error of approximately 20% is considered to be a fairly reliable result. It was confirmed that the estimated stiffness of the structure using the four object tracking algorithms showed a similar accuracy. This is because the value of the natural frequency (Table 3), which are the dynamic characteristics of the structure estimated by the four object tracking algorithms, have similar values. However, when only the estimated stiffness of the damaged structure was considered, it was confirmed that the most appropriate algorithm to be applied to the FE model updating is the KLT algorithm.

Although the natural frequency (Table 2) of the structure predicted by the four object tracking algorithms showed similar values, it was confirmed that the accuracy of the estimated stiffness of the structure differs by floor. Based on the above results, it was confirmed that when FE model updating will be performed using the natural frequency of the structure as the fitness function, the accuracy of the FE model updating may be lowered if the highly accurate natural frequency of the structure is not applied.

5. Discussion

This paper presented a new technique to perform the FE model updating of the structures using vision-based displacement measurements. Although the proposed method achieved reliable results for updating the FE model into a damaged state, there were some errors owing to the following reasons.

• (1). One of the main sources of the error is a modeling error. The FE model used in this study was a frame structure. Detailed information such as bolt connections was not considered in this model. Therefore, the response owing to these connections was not fully reflected in the updated model. The accuracy of the FE model updating is expected to be improved by developing a more detailed FE model.

• (2). Another source of error is inaccurate damping. The damping ratio of the structure is an important factor influencing the response of the structure. In practice, the damping ratio is assumed to be 5% for reinforced concrete structures and 2% for welded steel structures but can be varied by the material and structural conditions. Kudu et al. [38] compared the modal damping ratio obtained by considering the measurement duration, frequency range, and sampling rate, with the method used to identify the modal parameters as variable parameters. As a result, it was observed that the change in natural frequency was extremely small whereas the modal damping ratio changed significantly depending on the selected parameters. In this study, the damping ratio of the structure was assumed to be 2%, which caused an error in the FE model updating.

• (3). Imperfection in vision-based tracking algorithms can also be a source of error. In this study, displacement was measured using four tracking methods, and FE model updating was performed using the results. However, the results of the model updating showed differences according to each tracking method. Each tracking algorithm has its own tracking mechanisms that can cause errors during the tracking process. Some common phenomena in computer vision, such as radial distortion and a rolling shutter effect, have also reduced the accuracy. In this study, these issues were minimized using the method proposed by Yoon et al. [39], but these distortions could not be fully removed.

6. Conclusions and Future Studies

In this study, a vision-based finite element model updating using the genetic algorithm was proposed. The proposed system automatically measured the displacement of the structure from a video and updated the parameters of the FE model by using the genetic algorithm. To validate the performance of the proposed method, a lab-scale test with a three-story building was conducted. The displacement of each story of the building was measured before and after reducing the stiffness of one of the columns. The genetic algorithm was used to automatically optimize a non-damaged state of the FE model to a damaged state. Seven different object tracking algorithms (KLT, CSRT, KCF, MedianFlow, MIL, MOSSE, and TLD) were used and the FE model updating results were compared. As a result, proposed method using KLT, KCF, MIL, and MOSSE algorithms successfully updated the FE model to the as-is damaged condition, whereas CSRT, MedianFlow, and TLD were not able to update the FE model appropriately. Proposed method using KLT tracker showed the most reliable result with 89% accuracy for estimating the member stiffness. While some limitations are remaining as a future study, the proposed method is expected to reduce the time and effort for FE model updating by using computer vision.

Author Contributions

Conceptualization, H.Y.; methodology, H.Y.; software, G.P.; validation, G.P.; formal analysis, G.P.; investigation, G.P.; resources, H.Y.; data curation, G.P.; writing—original draft preparation, G.P.; writing—review and editing, H.Y., K.-N.H.; visualization, G.P.; supervision, H.Y., K.-N.H.; project administration, H.Y., K.-N.H.; funding acquisition, H.Y. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by the Korea Agency for Infrastructure Technology Advancement (KAIA) grant funded by the Ministry of Land, Infrastructure and Transport (Grant 20CTAP-C153021-02).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures and Tables

Figure 1. Overview of the vision-based finite element (FE) model updating.

Figure 2. Overview of the vision-based displacement technique [14].

Figure 3. Genetic selection method using genetic algorithm.

Figure 4. Fitness values at each generation.

Figure 5. Setup for shaking table test.

Figure 6. FE model used for model updating.

Figure 7. Vision-based displacement measurement: (a) region of interest (ROI) at the initial frame, (b–h) ROI at the final frame using different tracking algorithms.

Figure 8. Time-displacement curve using different object tracking algorithms.

Figure 9. Comparison of the transfer function using different object tracking algorithms.

Figure 10. Comparison of the estimated stiffness using different object tracking algorithms: (a) FE model, (b) column stiffness of design stage, (c) damaged column stiffness, and (d–g) estimated column stiffness using different tracking algorithms.

Table 1

Mesh sensitivity analysis result.

	Original Mesh	Size ½ Mesh		Size ¼ Mesh		Size 1/8 Mesh		Size 1/16 Mesh
Mode	Value(Hz)	Value(Hz)	Accuracy(%)	Value(Hz)	Accuracy(%)	Value(Hz)	Accuracy(%)	Value(Hz)	Accuracy(%)
1st	1.543	1.544	99.884	1.544	99.881	1.544	99.881	1.544	99.881
2nd	4.532	4.574	99.092	4.575	99.067	4.575	99.066	4.575	99.066
3rd	6.724	6.785	99.109	6.788	99.058	6.789	99.056	6.789	99.056

Table 2

The natural frequency for different object tracking algorithms.

	Natural Frequency (Hz)
Mode	KLT	MIL	KCF	MOSSE
1st	1.543	1.550	1.547	1.541
2nd	4.539	4.529	4.531	4.539
3rd	6.726	6.722	6.724	6.726

Table 3

Comparison of column stiffness between experiment and prediction model using different tracking algorithms.

	Design Stage(N/mm)	Damaged(N/mm)	Analysis (N/mm)				Error (%)
			KLT	MIL	KCF	MOSSE	KLT	MIL	KCF	MOSSE
Floor 1	1,286,400	1,286,400	1,194,896	1,211,896	1,211,896	1,217,792	6.833	5.792	5.792	5.333
Floor 2	1,286,400	1,270,320	1,031,800	995,888	993,208	983,024	18.776	21.603	21.814	22.616
Floor 3	1,286,400	1,286,400	1,189,384	1,194,744	1,199,568	1,188,848	7.542	7.125	6.75	7.583
Average	-	-	-	-	-	-	11.050	11.507	11.452	11.844