Modeling and Evaluation of Attention Mechanism Neural Network Based on Industrial Time Series Data

Abstract

Chemical process control systems are complex, and modeling the controlled object is the first task in automatic control and optimal design. Most chemical process modeling experiments require test signals to be applied to the process, which may lead to production interruptions or cause safety accidents. Therefore, this paper proposes an improved transformer model based on a self-attention mechanism for modeling industrial processes. Then, an evaluation mechanism based on root mean square error (RMSE) and Kullback–Leibler divergence (KLD) metrics is designed to obtain more appropriate model parameters. The Variational Auto-Encoder (VAE) network is used to compute the associated KLD. Finally, a real nonlinear dynamic process in the petrochemical industry is modeled and evaluated using the proposed methodology to predict the time series data of the process. This study demonstrates the validity of the proposed transformer model and illustrates the versatility of using an integrated modeling, evaluation, and prediction scheme for nonlinear dynamic processes in process industries. The scheme is of great importance for the field of industrial soft measurements as well as for deep learning-based time series prediction. In addition, the issue of a suitable time domain for the prediction is discussed.