Real-Time Collection and Analysis of Sports Index Time Series Based on Multimodal Sensor Monitoring

Abstract

With the further development of microelectronics technology and sensors, sensors can be widely embedded in mobile phone devices and portable devices. The use of acceleration sensors for human motion monitoring has broad application prospects. Monitoring the daily exercise of the human body is of great significance for formulating scientific exercise and fitness plans and improving physical health. This paper uses the measurement data of multiple types of sensors to propose an index recognition method based on the fusion of multiple types of sensor information. We take the measurement value of a single type of sensor as input and output the index value of the moving part without a strain sensor. The pattern recognition method is used to establish a pattern library, a recognition library, and a measurement library. This article considers noise interference or malfunction of sensor measurements. Aiming at uncertain factors such as the error of the finite element model, a pattern matching method considering the uncertainty is proposed. This article takes aerobics as an example to simulate and analyze the dynamic response of aerobics under wind load. In addition, by simulating the recognition results under different levels of noise interference, the robustness and anti-interference of the pattern matching method are verified.

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

1. Introduction

With the rapid development of wireless sensor technology and wireless communication technology, the main problem of data transmission is to choose which wireless communication technology to transmit data [1]. Information fusion refers to the process of decision-making and estimation task information processing through automatic analysis and comprehensive realization of multiple sensor observation information obtained according to time sequence in the relevant criteria by computer technology, because information fusion process has multiple sensors to obtain information connection and processing. An information fusion system is a processing system that obtains information as objects through sensors. In addition to sensors, the information fusion system also includes other links, each of which has special functions and characteristics [2]. The development cycle realizes the design of an information fusion analysis system through object-oriented thought and realizes the standardization in the process of system design. Due to the particularity of terminal nodes, how to ensure less data and energy consumption in the process of data transmission is a difficult problem in the current wireless transmission process. The current wireless communication technologies include Bluetooth, UWB, WiFi, and Zigbee. In the specific environment to choose a reasonable way, or even a combination of transmission mode, another problem is how to design high-performance transport protocols.

The so-called multisensor information fusion (MSIF) is an information processing process that uses computer technology to automatically analyze and synthesize the information and data from multiple sensors or sources under certain criteria to complete the required decisions and estimates [3]. The basic principle of multisensor information fusion technology is to make the multilevel and multispace information complementary and optimal combination processing of various sensors and finally produce consistent interpretation of the observation environment. This process should make full use of multisource data for reasonable control and utilization. The ultimate goal of information fusion is to separate observation information obtained from each sensor and extract more useful information through multilevel and multidirectional information combination. It not only takes advantage of the cooperative operation of multiple sensors but also comprehensively processes the data of other information sources to improve the intelligence of the whole sensor system. Fusion methods include neural network, decision theory, information theory, statistical reasoning, and evidence theory. It can analyze data according to certain criteria and obtain reliable and accurate prediction results, which is the multisource information fusion technology. The core of big data technology is the remote large server cluster. In order to solve the problem of large amount of data, it is necessary to apply a data fusion algorithm to a cloud server. In this way, the stability and efficiency of the system can be improved, and problems such as poor scalability, high cost, poor data sharing, and difficult system maintenance can be solved in the monitoring platform [4, 5]. Cloud server processing technology plays a very important role in the current monitoring platform, which can classify, store, manage and share the explosive growth of data and provide a platform for subsequent data fusion.

2. Related Work

Sports is one of the important means to ensure people’s health, among which, aerobics is popular among people in recent years [6]. The perception of the teaching effect of aerobics is the key factor to learn aerobics, but the current aerobics exercise method is not professional, and the teaching effect cannot be effectively evaluated in the process of fitness. Deng and Jiang identified the motion features of calisthenics decomposition by the feature extraction method, solved the optical flow between adjacent difference frames and by the Laplace method, reduced the impact of clutter, set the similarity threshold, extracted the motion features of calisthenics decomposition by similarity detection, and output the actions with high similarity as the results [7]. Wichit and Choksuriwong realized the design of a multisensor-based athlete training information fusion analysis system. Based on the acquisition and analysis of movement information, it analyzed the ground reaction information, motion image analysis, human surface mechanics, and so on in the process of athletes’ movement, so as to further study the extraction of athletes’ information features [8]. Bharti et al. proposed an image-based method to monitor the movement accuracy of calisthenics. The kindest depth image acquisition method was used for preanalysis of calisthenics movements, and HOG3D was used to extract the movement characteristics of calisthenics [9]. Mohsin et al. use GPRS network transmission and embedded system to collect sports information in real time and generate monitoring process diagram of sports training parameters, providing scientific basis for targeted training [10]. Physical education teaching evaluation is an important part of physical education teaching, which plays an important role in the process of physical education teaching. It provides information that controls and adjusts physical activity as a whole and ensures that it develops towards its intended goals. With the development of the popularization of physical education, people’s demand for the quality of physical education is constantly improving, and the role and status of evaluation in teaching are becoming increasingly obvious, and evaluation has become an indispensable process in teaching activities.

With the progress of science and technology, human-computer interaction is more and more widely used in our daily life [11]. It has become a current development trend to study human-computer interaction system that meets current needs and to introduce human body as a reference coordinate system to directly map some actions and patterns of human body to a computer with a coordinate system. Traditional human-computer interaction technology research focusses on computer as the centre; now, human-computer interaction technology research focusses on human-computer interaction technology, all-round development, multimode, and deep coexistence, to achieve the interaction system between the user and the system. Line of sight is one of the most intuitive ways for human to receive information from the outside world. Body movements based on human eyes are characterized by directness, authenticity, and simplicity. However, visual judgment of some subtle actions shows great limitations. A human-computer interaction system based on microaction information acquisition module can effectively make up for the deficiency of visual judgment.

With the rapid development of electronic industry, especially computer, the development of the human-computer interaction system is not only reflected in the hardware but also has great progress in software. At present, research based on the human-computer interaction system not only tends to be practical and aesthetic but also the ease of operation and the degree of fitting with people will become a very important development direction [12, 13]. Human-computer interaction (HCI) refers to a technology in which the information to be tested is exchanged with a certain algorithm by certain input and output devices between human and computer and finally realizes the mutual communication between human and machine. The system consists of a computer providing some useful information to people through a display device or an output device, while people providing some instruction information to the server through some input devices. In human-computer interaction technology, interface design is very important, to realize the media and carrier of interaction between people and computers; users can receive information through the most intuitive way.

Data fusion is from multiple channels, multiple sources, and various data or information accurately, to determine the organic relation, comprehensive analysis, the overall evaluation, and the organic fusion of multiple parameters, multilevel, many elements of the process, and then get fused state, as well as the processing object, a comprehensive evaluation in the end gets information or data. Data fusion is regarded as the organic combination of data or information of different information sources, forms, media, time, and presentation, so as to accurately understand the object state [14]. The advantage of multisensor data and information fusion is that the characteristics of the object to be measured can be obtained in a very short time by a relatively simple method. The multisensor fusion information has strong reliability, timeliness, and robustness. In the process of data fusion, it involves the processing of many uncertain factors, the decision-making mechanism and process, and the analysis of the characteristics of the information source, and it will be different with the environment of the demander.

Multisensor data fusion technology has three important characteristics: multisensor data fusion is based on the different abstraction degree of information source, multilevel processing of information; the process of multisensor data fusion includes detection, correlation, tracking, analysis, evaluation, and merger [15]. The output of multisensor data fusion can be divided into initial assessment of identity and status at low level and advanced assessment of situation and decision at high level. The target object of the multisensor data fusion system is the different information collected by various types of sensors, which has various forms of expression, such as sound, text, image, or electrical signal. The information collected by the multisensor system is called source information [16]. The purpose of multisensor data fusion is to obtain the optimal estimation of target state and properties by analyzing and processing various original information from different sensors through specific models and algorithms. Multisensor data fusion technology arises at the historic moment; it is to use computer technology to the time sequence of number of sensor observation information, and information database and knowledge base, to a certain criterion, which automatically collected, relevant, analysis and synthesis for a representation, to complete the required for estimation and decision task of information processing. Information fusion has always played an important role in the evolution of any life.

According to the data fusion structure, there are many kinds of classification. One of these methods is divided into sensor-level (distributed), central-level (centralized), and hybrid mode according to the way data is processed before finally entering data fusion. Sensor-level fusion architecture works: different sensors are used to collect information for the same target, and they are independent signals. First, each individual sensor captures and analyzes information; then, the analysis results of all sensors are transmitted to the fusion centre. Finally, the fusion results are used for state estimation. Because the structural model deals with feature vector data in the fusion centre, it does not require much computer and reduces the computing pressure. At the same time, sensor-level fusion has good stability and low system cost, so it is widely used in most projects. Central-level data fusion means that each terminal sensor transmits the preprocessing results to the fusion centre after minimal processing [17, 18]. In the fusion centre, the information obtained by each sensor is checked, correlated, fused, and decided. In most cases, this structure is the best data fusion method in the case of smooth data association and fusion in the fusion centre. The structure also requires powerful CPU processing power and high storage capacity at the convergence centre. If the sink fails, the whole system collapses. Therefore, although the structure is very good, there are some disadvantages such as poor stability. Hybrid fusion is a combination of the first two. It is the algorithm processing of adding and subtracting sensors before data reaches the convergence point and fusion [19]. In general, if the measurement of each terminal node cannot be completely independent of each other, the hybrid fusion method is the best way to classify the data. The advantage of this structure is that the sensor fusion is added in the fusion process and it has strong adaptability. However, due to the increased sensor fusion, the hybrid structure increases the complexity of data processing, reduces the transmission efficiency, and increases the cost of data processing and wireless communication.

3. Support Vector Machine Algorithm

Support vector machine (SVM) algorithm was first developed from the generalized portrait method in pattern recognition algorithm. It is a kind of generalized linear classifier that performs a binary classification of data according to supervised learning, and its decision boundary is the maximum margin hyperplane of sample data to perform a solution. If a sample data can be correctly divided into two categories, then there must be an optimal hyperplane in between which can be expressed by $$\begin{array}{}\text{(1)}& gx=W^t\times x+b,\end{array}$$where $W$ represents a vector of weights and $b$ is a constant term. In general, training for sample data is to get an optimal hyperplane and classify it correctly. A standard SVM model satisfies formula (2) in order to classify samples correctly. $$\begin{array}{}\text{(2)}& \begin{array}{c}{W}^t\times x+b\ge 0,y=1,\hfill \\ W^t\times x+b<0,y=-1.\hfill \end{array}\end{array}$$

The farther away it is, the more obvious the classification of “normal” and “abnormal” states is, and the classification results can be easily distinguished. In order to achieve the maximum classification effect, the distance between $H1$ and $H2$ is defined as $$\begin{array}{}\text{(3)}& M=\frac{2}{\sqrt{w^2}}.\end{array}$$

The magnitude of $M$ is determined by the points closest to $H$, which are called support vectors. Let $H1$ and $H2$ from the farthest be equivalent to the value of $M$ which is the largest, also to make the $\mid W\mid$ minimal. So, finding the optimal hyperplane is equivalent to solving the optimization constraint problem. For the above case, it is transformed into a quadratic normalization problem. $$\begin{array}{}\text{(4)}& \min \varphi w=\frac{1}{2}{w}^{t}w.\end{array}$$

The constraint conditions are expressed by $$\begin{array}{}\text{(5)}& y{W}^{t}W+b\ge 1.\end{array}$$

It is usually solved using Lagrange multipliers, so formula (4) becomes $$\begin{array}{}\text{(6)}& Lw,b,a=\frac{1}{2}{W}^{t}W-\sum \partial y{W}^{t}x+b-1.\end{array}$$

The optimal solution is the inflection point of equation (6). $$\begin{array}{c}\text{(7)}& \frac{\partial L}{\partial W}=w-\sum \alpha yx=0,\frac{\partial L}{\partial W}=-\sum \alpha yx=0.\end{array}$$

Meanwhile, the solution of the quadratic optimization problem must satisfy $$\begin{array}{}\text{(8)}& \beta yw\times x+b=0.\end{array}$$

The training sample points with medium values are called support vectors, which are usually only a small part of the total sample. For SVM, support vector is the key factor in training process. Finally, the optimization function of the classification problem is obtained as $$\begin{array}{}\text{(9)}& fx=\mathrm{sgn}\sum y\alpha +b.\end{array}$$

4. Teaching Effect Monitoring System Based on Multisensor Information Fusion

4.1. Overall System Design Architecture

The monitoring system is designed according to several principles, including data acquisition part, data transmission, and monitoring centre part. The data acquisition part is mainly composed of terminal nodes based on a STM32 microprocessor, including LoRa wireless communication module and various sensor modules. The data transmission part is composed of a stM32-based microprocessor, which includes LoRa wireless communication module and WiFi module. The monitoring centre is mainly constructed based on the server, and the corresponding monitoring function can be realized by deploying the Web server. The monitoring system architecture mainly consists of two parts: hardware and software. The hardware unit mainly includes terminal node and sink node. Terminal nodes and sink nodes communicate through LoRa to realize wireless data transmission. After the sink node receives the data, the MQTT communication protocol is used to send the data of the sink node to the cloud server for monitoring through Web pages. The hardware design block diagram of the monitoring system is shown in Figure 1.

[figure omitted; refer to PDF]

The system software includes the design of terminal node data acquisition terminal, aggregation node, and cloud server. In the monitoring system, the sensor terminal node is located at the bottom of the whole system, responsible for the collection of various parameters. It determines the efficiency of subsequent data collection. Cloud server processing technology plays a very important role in the current monitoring platform, which can classify, store, manage, and share the explosive growth of data and provide a platform for subsequent data fusion. The sink node is mainly composed of a processor, wireless communication module, and storage module to realize wireless data interaction with sensor node and cloud.

4.2. Aerobic Aerobics Monitoring System Based on Multisensor Information Fusion

In the process of fitness, dynamic information can be obtained through sensors, which has an important influence on fitness analysis. Dynamic parameters generally include human body displacement, plantar pressure, joint force, angle, and acceleration, which can be obtained by force sensor, displacement sensor, speed sensor, accelerometer, inertial sensor, and goniometry. The six-dimensional force testing platform can be divided into three modules: force sensor, signal processing module, and computer module. The measuring area is large, and the data of force cabinet and three-dimensional space can be obtained at the same time.

The aerobics’ monitoring platform is mainly based on a wireless sensor network, which collects and transmits the physical signs data of guardians by wearing different kinds of sensors on people [20]. The data collected by different sensors are signed and sent to the cloud through the wireless sensor network server. After receiving the corresponding algorithm for data processing, the processed data is sent back to the server for storage. Data and statement results were simultaneously sent to the mobile device of the monitored object and the hospital database of community service, so as to monitor the aerobics teaching. The overall framework of the system is shown in Figure 2.

[figure omitted; refer to PDF]

As can be seen from the figure, the system consists of four layers: data layer, feature layer, feature fusion layer, and decision layer. The results of teaching and training are obtained by fusion analysis based on the information acquisition and processing of human movement. The multiobjective and multiparameter data in the training process of athletes can be obtained for effective fusion analysis. The system can transfer the physiological information parameters of the measured target to the server for corresponding processing and analysis, which makes the remote diagnosis more convenient, accurate, and low cost.

4.3. Terminal Node Design

In WSN, the terminal node is a full-function device, which perfectly combines the sensing technology, embedded technology, and wireless communication technology to realize the collection of human physiological parameters. Temperature, heart rate, blood pressure, respiration, and blood oxygen were collected [21]. An end node is a device that specifically performs data collection transmission and cannot transmit messages from other nodes. The terminal node adopts a modular design idea, and its main function is divided into two parts: various types of sensors for data collection; the LoRa module is responsible for sending and receiving data. Other parts also include the design of serial communication and power supply modules. The hardware structure of terminal node mainly includes a multisource sensor, STM32 processor, LoRa wireless communication module, and power module. The power module of the system uses USB power supply to provide normal working voltage for terminal node components. The terminal node as the data acquisition end, its power supply design adopts battery power. In addition to the master control chip, the peripheral circuit of the terminal node only retains the basic LoRa wireless data transceiver module, JTAG debugging module, and power reset and other basic circuits.

In the aerobic aerobics monitoring system, the terminal node of the physical sign sensor is located at the bottom of the whole system, which is responsible for collecting various parameters of the human body. It determines the efficiency of subsequent data collection. The wearable terminal nodes in the monitoring platform include motion sensors and biosensors. Motion sensors include accelerometers and gyroscopes; biosensors include blood pressure, heart rate, blood oxygen, and body temperature. The terminal node is one of the core parts of the whole system, which is now developing towards miniaturization and low power consumption.

4.4. Sink Node Design

The sink node is mainly composed of a processor, wireless communication module, and storage module. STM32F103ZET6 chip is used as the processor. The chip is a 32-bit processor product based on armCortex-M3 architecture kernel, and the maximum operating frequency can reach 72 MHz, with rich resources and low power consumption and cost. The wireless communication module consists of a LoRa module and a WiFi module. Both modules connect to the processor through a serial port for wireless data interaction with the sensor node and the cloud. The FLASH module and EEPROM module are used to store data such as WiFi hotspot name and password to prevent loss after power failure. The function of sink node mainly includes the following two aspects: LoRa module receives data of terminal node; the WiFi module packages and sends data from the terminal node to the server. The sink node is mainly used to forward the data of the terminal node. Besides the master control chip, it also contains LoRa wireless data transceiver module, WiFi module, JTAG debugging module, and power reset and other basic circuits.

As the core of the entire network, the sink node is mainly responsible for network construction, maintenance, information aggregation, and data uploading. The program design of sink node includes LoRa wireless communication program and WiFi transmission program. LoRa module mainly receives data transmitted from terminal nodes, while WiFi module sends data to the cloud server. Sink node software and networking program.

4.5. Server Design

Based on the functional requirements of a server layer, establish a data server. The data server is used to receive sensor data information to ensure that all sensor data can be stored in the cloud server in real time and accurately, providing data support for subsequent fitness effect analysis and human-computer interaction [22]. The aggregation node communicates with the cloud server using the MQTT protocol and using the JavaScript Object Notation (JSON) data format, which is a lightweight data exchange format that is easy to parse and generate by machines and can effectively improve network transmission efficiency. The TCP/IP protocol is designed for poor hardware performance and poor network conditions of the remote device; it is designed as a publish/subscribe messaging protocol. An MQTT protocol typically has two roles, publisher and subscriber. The cloud server is mainly responsible for data processing, analysis, storage, and visual display uploaded by the sink node. Based on the object-oriented language Java development, the server program uses Java Web development framework JFinal as the project framework; it uses Maven for project development and management, using MySQL as a database. The web client of the monitoring centre is designed in the form of a webpage, which is convenient for multiscreen display and remote viewing. The DataV of Aliyun is used for the front-end interface design.

The server program provides the data interface, and the monitoring centre provides real-time data display and historical data display. The server is mainly composed of a data server, human-computer interaction server, and corresponding database. A database server is built on the basis of a database system and has the characteristics of a database system. Its functions include system configuration and management and data access and update management. Similarly, a human-computer interaction server is built on the basis of a human-computer interaction system, responsible for receiving data from the data server after processing released to the client display through visual processing.

5. Monitoring System Testing and Effect Analysis

After completing the design of the overall hardware and software of the monitoring system, the function and stability of the monitoring system are tested on this basis. It is very important to test the functionality and stability of the whole system in the development process. Through the test of the system, we can find and deal with some problems that may exist in the whole system, so as to ensure that the whole system can run stably and smoothly when users use it.

5.1. Server Testing

The server test mainly tests whether the server can receive and display the uploaded hardware data. For normal operation of the overall system, testing system, adopted two terminal nodes, a gathering node, each kind of sensor mounted on sign parameter collection, will gather the node to receive data transmission via a serial port to display data from serial port assistant, by the MQTT protocol to TIO server parsed. The parsed data will be added to the message queue MQ, and then, the corresponding API of the platform stores the data in MQ into the database. Finally, APP obtains the historical data in the API through RPC for visual display. The scene of this experiment is to collect human body parameters in the room under normal conditions. You can obtain the current data information by entering the corresponding address in the browser of the PC. Basic physiological parameters of users in the current monitoring system, including temperature (degC), blood pressure (mmHg), blood oxygen (%), respiration (bpm), and heart rate (bpm), are shown in Figure 3.

[figure omitted; refer to PDF]

As can be seen from the figure, the sensor data mounted on the terminal node is collected, and these are uploaded to the server for storage and display. After a period of operation test, the system can run stably and continuously through the test of the overall operation of the system, indicating that the system is normal. Sensors provide accurate data for the monitoring system. The sensor measurement can correctly determine the effect of aerobics teaching. The system can be collected, transmitted, stored, and displayed and can achieve the purpose of human monitoring.

5.2. Data Fusion Algorithm Testing and Result Analysis

The data fusion algorithm used to mimic the public data sets was tested, in order to state the general algorithm and the single exception of experiments, the same to 20 times the simulation model and data, using the data acquisition of the system for the user’s physiological parameter acquisition and validated using data fusion algorithm, and the results are necessary analysis. Part of physiological parameters of multiple users collected by the system in a conventional environment is shown in Figure 4.

[figure omitted; refer to PDF]

The test results show that the accuracy of the model based on real user data is basically consistent with the calculated results, and the algorithm proposed in this paper is feasible in practical application. The accuracy of the data fusion model is above 90%, and the difference is not large. Basically, it fluctuates within a very small range of average accuracy, which indicates the high stability of the algorithm in this paper. Ensure that the system can run continuously and stably, the measured physiological data is accurate and reliable, and the physiological parameters of human body are obtained. In addition, the relationship between the parameter gamma and the penalty factor is shown in Figure 5.

[figure omitted; refer to PDF]

As can be seen from the figure, in the aerobics movement, there are jumping movement, V-step, cross step, parallel step, jumping jacks low-impact movements, students’ physical quality has also been improved to a certain extent, and aerobics project in enhancing people’s strength, flexibility, and coordination plays a significant role.

5.4. Analysis on the Teaching Effect of Students’ Movement Skills in Calisthenics

Before the experiment, according to the teaching objectives and requirements of the teaching task of calisthenics class, the prescribed level 1 calisthenics movement is selected as the teaching content. Before the experiment, the students’ simply four eight-beat calisthenics movements are examined once. In the assessment from the following aspects of the students’ technology monitoring: the accuracy of movement, the strength of the action, dexterity of movement, a sense of rhythm of movement, and the expressiveness of movement, the comparison results are shown in Figure 7.

[figure omitted; refer to PDF]

From the figure analysis, it can be seen that after the experiment, the control group had significant changes in movement intensity, accuracy, proficiency, rhythm, and expression, indicating that the control group also had significant differences before and after the experiment. In action on the total score, which also has a very significant change, after the experiment by comparing two groups of aerobics skill test indicators, it can be seen that experimental class students of aerobics skills to the test indexes were higher than that in comparative classes of students, the experimental group and control group in action on the strength and rhythm, expressive differences are more obvious than movement accuracy and movement skill, and the effect is more prominent. For the comprehensive aerobics movement, in order to achieve significant teaching effect, in the teaching process to pay attention to the aerobics movement structure, movement characteristics and time and space effect, monitoring system to show each student’s movement accuracy, enhance the students’ ability to understand the movement.

6. Conclusion

The progress of science and technology brings about the improvement of people’s quality of life, and people pay more attention to aerobic exercise. The perception of the teaching effect of aerobics is the key factor to learn aerobics, but the current aerobics exercise method is not professional, and the teaching effect can not be effectively evaluated in the process of fitness. This paper discusses the basic theory of human-computer interaction and multisensor information fusion, designs the multisensor information fusion system framework from software and hardware, respectively, and puts forward the monitoring system of aerobic aerobics teaching effect based on multisensor information fusion. On the basis of computer simulation, the accuracy of the data fusion model is above 90%, and the difference range is not big and basically fluctuates within a very small range of average accuracy. The results show that the designed system can achieve the acquisition and fusion of information, the use of multisensor equipment to collect the characteristics of aerobics data, movement strength, sense of rhythm, and expressiveness of the difference is more obvious than the accuracy of movement and movement proficiency, and the effect is more significant. The monitoring system designed in this paper can basically meet the monitoring needs of human physiological parameters, but the functions of some modules are not perfect, the overall hardware structure of the system is slightly complex, the overall volume is not small enough, and the system is not intelligent enough to collect and process data. The model is only simulated on a computer and tested on a simple hardware platform. How to transplant the model to a more advanced hardware platform and the miniaturization of the terminal is the direction of development. After using big data, the research direction should be structured, miniaturized, and intelligent.

References

[1] A. N. Patel, T. P. Jung, T. J. Sejnowski, "A wearable multi-modal bio-sensing system towards real-world applications," IEEE Transactions on Biomedical Engineering, vol. 66 no. 4, pp. 1137-1147, 2018.

[2] J. Zhang, Z. Yin, P. Chen, S. Nichele, "Emotion recognition using multi-modal data and machine learning techniques: a tutorial and review," Information Fusion, vol. 59, pp. 103-126, DOI: 10.1016/j.inffus.2020.01.011, 2020.

[3] F. Ali, S. el-Sappagh, S. M. R. Islam, D. Kwak, A. Ali, M. Imran, K. S. Kwak, "A smart healthcare monitoring system for heart disease prediction based on ensemble deep learning and feature fusion," Information Fusion, vol. 63, pp. 208-222, DOI: 10.1016/j.inffus.2020.06.008, 2020.

[4] E. Kanjo, E. M. G. Younis, C. S. Ang, "Deep learning analysis of mobile physiological, environmental and location sensor data for emotion detection," Information Fusion, vol. 49, pp. 46-56, DOI: 10.1016/j.inffus.2018.09.001, 2019.

[5] J. Zhou, X. Hong, P. Jin, "Information fusion for multi-source material data: progress and challenges," Applied Sciences, vol. 9 no. 17,DOI: 10.3390/app9173473, 2019.

[6] H. Wang, Q. Zhang, "Dynamic identification of coal-rock interface based on adaptive weight optimization and multi-sensor information fusion," Information Fusion, vol. 51, pp. 114-128, DOI: 10.1016/j.inffus.2018.09.007, 2019.

[7] X. Deng, W. Jiang, "Fuzzy risk evaluation in failure mode and effects analysis using a D numbers based multi-sensor information fusion method," Sensors, vol. 17 no. 9,DOI: 10.3390/s17092086, 2017.

[8] N. Wichit, A. Choksuriwong, "Multi-sensor data fusion model based Kalman filter using fuzzy logic for human activity detection," International Journal of Information and Electronics Engineering, vol. 5 no. 6, pp. 450-453, DOI: 10.7763/IJIEE.2015.V5.577, 2015.

[9] P. Bharti, D. de, S. Chellappan, S. K. Das, "HuMAn: complex activity recognition with multi-modal multi-positional body sensing," IEEE Transactions on Mobile Computing, vol. 18 no. 4, pp. 857-870, DOI: 10.1109/TMC.2018.2841905, 2019.

[10] A. H. Mohsin, A. A. Zaidan, B. B. Zaidan, A. S. Albahri, O. S. Albahri, M. A. Alsalem, K. I. Mohammed, "Real-time remote health monitoring systems using body sensor information and finger vein biometric verification: a multi-layer systematic review," Journal of Medical Systems, vol. 42 no. 12, pp. 32-36, DOI: 10.1007/s10916-018-1104-5, 2018.

[11] J. O. de Oliveira de Souza, M. D. Bloedow, F. C. Rubo, R. M. de Figueiredo, G. Pessin, S. J. Rigo, "Investigation of different approaches to real-time control of prosthetic hands with electromyography signals," IEEE Sensors Journal, vol. 21 no. 18, pp. 20674-20684, DOI: 10.1109/JSEN.2021.3099744, 2021.

[12] X. Yan, H. Liu, G. Xin, H. Huang, Y. Jiang, Z. Guo, "Research on real-time elimination of ultra-wideband radar ranging abnormal value data," Geoscientific Instrumentation, Methods and Data Systems, vol. 10 no. 2, pp. 153-160, DOI: 10.5194/gi-10-153-2021, 2021.

[13] M. Wang, X. Wang, L.-T. Yang, X. Deng, L. Yi, "Multi-sensor fusion based intelligent sensor relocation for health and safety monitoring in BSNs," Information Fusion, vol. 54, pp. 61-71, DOI: 10.1016/j.inffus.2019.07.002, 2020.

[14] M. Chen, Y. Zhang, M. Qiu, N. Guizani, Y. Hao, "SPHA: smart personal health advisor based on deep analytics," IEEE Communications Magazine, vol. 56 no. 3, pp. 164-169, DOI: 10.1109/MCOM.2018.1700274, 2018.

[15] X. Wu, J. Duan, L. Zhang, S. M. AbouRizk, "A hybrid information fusion approach to safety risk perception using sensor data under uncertainty," Stochastic Environmental Research and Risk Assessment, vol. 32 no. 1, pp. 105-122, DOI: 10.1007/s00477-017-1389-9, 2018.

[16] G. Chen, Q. Pei, M. M. Kamruzzaman, "Remote sensing image quality evaluation based on deep support value learning networks," Signal Processing: Image Communication, vol. 83, article 115783,DOI: 10.1016/j.image.2020.115783, 2020.

[17] O. AlShorman, B. Alshorman, M. S. Masadeh, "A review of physical human activity recognition chain using sensors," Indonesian Journal of Electrical Engineering and Informatics, vol. 8 no. 3, pp. 560-573, DOI: 10.52549/ijeei.v8i3.2312, 2020.

[18] A.-A. AlZubi, A. Abugabah, M. al-Maitah, F. Ibrahim AlZobi, "DL multi-sensor information fusion service selective information scheme for improving the internet of things based user responses," Measurement, vol. 185, article 110008,DOI: 10.1016/j.measurement.2021.110008, 2021.

[19] T. Li, Z. Zhao, C. Sun, R. Yan, X. Chen, "Hierarchical attention graph convolutional network to fuse multi-sensor signals for remaining useful life prediction," Reliability Engineering & System Safety, vol. 215, article 107878,DOI: 10.1016/j.ress.2021.107878, 2021.

[20] R. Dinesh Jackson Samuel, E. Fenil, G. Manogaran, G. N. Vivekananda, T. Thanjaivadivel, S. Jeeva, A. Ahilan, "Real time violence detection framework for football stadium comprising of big data analysis and deep learning through bidirectional LSTM," Computer Networks, vol. 151, pp. 191-200, DOI: 10.1016/j.comnet.2019.01.028, 2019.

[21] M. L. Shuwandy, B. B. Zaidan, A. A. Zaidan, A. S. Albahri, A. H. Alamoodi, O. S. Albahri, M. Alazab, "mHealth authentication approach based 3D touchscreen and microphone sensors for real-time remote healthcare monitoring system: comprehensive review, open issues and methodological aspects," Computer Science Review, vol. 38, article 100300,DOI: 10.1016/j.cosrev.2020.100300, 2020.

[22] K. I. Mohammed, A. A. Zaidan, B. Zaidan, O. S. Albahri, M. A. Alsalem, A. S. Albahri, A. Hadi, M. Hashim, "Real-time remote-health monitoring systems: a review on patients prioritisation for multiple-chronic diseases, taxonomy analysis, concerns and solution procedure," Journal of Medical Systems, vol. 43 no. 7,DOI: 10.1007/s10916-019-1362-x, 2019.