Batch-to-Batch Optimization Control of Fed-Batch Fermentation Process Based on Recursively Updated Extreme Learning Machine Models

Abstract

This paper presents a new method of batch-to-batch optimization control for a fed-batch fermentation process. A recursively updated extreme learning machine (ELM) neural network model is used to model a fed-batch fermentation process. ELM models have advantages over other neural networks in that they can be trained very fast and have good generalization performance. However, the ELM model loses its predictive abilities in the presence of batch-to-batch process variations or disturbances, which lead to a process–model mismatch. The recursive least squares (RLS) technique takes the model prediction error from the previous batch and uses it to update the model parameters for the next batch. This improves the performance of the model and helps it to respond to any changes in process conditions or disturbances. The updated model is used in an optimization control procedure, which generates an improved control profile for the next batch. The update of the RLS model enables the optimization control strategy to maintain a high final product quality in the presence of disturbances. The proposed batch-to-batch optimization control method is demonstrated on a simulated fed-batch fermentation process.

1. Introduction

Batch and fed-batch processes are commonplace in industrial fermentation processes. In a fed-batch process, a feed stream is used to supply additional substrate to the reactor during the reaction. The feed rate profile of this feed stream can be changed during the process, and the final product is removed at the end of the reaction. Introducing a feed stream helps to improve the control of reactant concentrations and, therefore, can be used to discourage undesired reactions from taking place and to improve product selectivity [1]. These benefits only transpire when a suitable feeding strategy is used. Therefore, determining the feeding strategy that yields the maximum amount of desired product is the main design objective for fed-batch reactors (i.e., the optimal control strategy) [2]. In the case of fermentations, this is usually where the rate of substrate being added to the reactor is equal to the rate at which it is used up by the organism [3]. The ever-increasing market competition and the stricter regulations on product quality and process efficiency mean that the optimization of fed-batch fermentations is becoming increasingly important [4]. An optimal control strategy will also help improve profits from the process [5].

The simplest way of optimizing the fed-batch process is to build an off-line control strategy using a model of the process. However, the presence of process–model mismatches (PMMs) and unknown disturbances renders the control strategy suboptimal when applied to the real fed-batch process [4]. Therefore, strategies have been proposed to mitigate this issue. Batch-to-batch optimization involves taking data from the previous batch and using them, with historical batch data, to optimize the performance of the following batch.

In order to develop an optimal control strategy, a suitable model of the process is required. There are three different types of models that can be used: mechanistic models, data-driven models, or hybrid models. A mechanistic model uses knowledge of the kinetics, stoichiometry, and mass and energy balances of the reaction system. Data-driven or “black-box” models are constructed from process operational data and used to describe the relationship between the manipulated variables and the output variables [2,6]. Hybrid models combine elements of data-driven and mechanistic models, taking advantages of both [7,8,9,10]. Von Stosch et al. [11] gives a detailed review of hybrid modeling in process systems engineering.

A common approach for developing a data-driven model is through using neural networks [12,13]. The most commonly used neural networks are multi-layer feedforward neural networks consisting of an input layer, an output later, and one or more hidden intermediate layers [14]. Each layer has several neurons, which are connected to neurons in the adjacent layers via weighted connections. Data are processed in these neurons as they pass through the network until they reach the output layer [15]. Fed-batch processes are nonlinear systems and are often modeled using neural networks due to their excellent approximation ability [16,17]. Xiong et al. [18] developed a batch-to-batch control strategy based on a control affine feedforward neural network, which used modeling errors from the previous batch to improve the model. The model prediction improved after each batch, and so process–model mismatch was reduced. Zhang et al. [4] developed a linearized model that was rebuilt after each batch by adding data from the recently completed batch to the pool of historical process data. The optimal control strategy using the updated model was able to respond to disturbances. Jewaratnam et al. [3] expanded on the updating linearized model by introducing a “sliding window” approach in which only data from a window of the most recent batches were used to update the model. The sliding window improved the convergence rate and stability following a disturbance.

ELM is a type of neural network developed by Huang et al. [19]. It differs from standard single-layer feedforward neural networks (SLFNs) in the way the network is trained. In standard SLFNs, hidden-layer weights must be solved iteratively using a training algorithm like back propagation, and a large number of training iterations are typically required [20]. Whereas, in ELM, the weights of the hidden layers are assigned randomly and are not repeatedly adjusted [21]. This allows the output-layer weights to be solved as a linear system of equations, which makes the training the ELM model significantly faster [22]. ELM also provides very high generalization performance [23]. Alli and Zhang [21] proposed using ELM in conjunction with recursive least squares (RLS) for batchwise model updating. Unknown disturbances or PMMs are detrimental to the ELM model prediction. RLS is, therefore, used to address any PMM. After each batch, RLS uses the latest model prediction error to update the weights of the ELM neural network output layer, allowing the ELM model to respond to disturbances. The method was shown to be successful in suppressing the effects of PMM and disturbances. Recursive methods have also been recently used in developing process monitoring methods. Li et al. [24] presented a recursive principal component analysis method for adaptive process monitoring. Yu and Zhao [25] developed a recursive exponential slow feature analysis method for adaptive process monitoring. Yu et al. [26] proposed a recursive cointegration approach for the adaptive monitoring of nonstationary industrial processes.

This paper proposes using the recursively updated ELM model in a batch-to-batch optimization control strategy. An ELM model is initially developed from historical process operation data. After each batch, the ELM model will be updated using RLS and used in an optimization framework to maximize the final biomass of a fed-batch fermentation in the next batch. Using a recursively updated model will allow the model to adapt to disturbances and reduce PMM with each iteration. A mechanistic model of a fed-batch fermentation will be used to simulate a real fed-batch process.

This paper is organized as follows: Section 2 presents the fed-batch fermentation process, ELM, batch-to-batch ELM model updating using RLS, and batch-to-batch optimization control. Section 3 presents the results and discussions. Section 4 concludes this paper.

2. Materials and Methods

2.1. Fed-Batch Fermentation Model

A mechanistic model for the production of baker’s yeast in a fed-batch fermenter is used in this study. The model was taken from Yüzgeç et al. [27]. Equations (1)–(12) outline the kinetic model.

Glucose uptake rate:

(1)$Q_{\mathrm{s}}=Q_{\mathrm{s},\mathrm{m}\mathrm{a}\mathrm{x}}{\displaystyle \frac{C_{\mathrm{s}}}{K_{\mathrm{s}}+C_{\mathrm{s}}}}\left(1-\exp \left(-{\displaystyle \frac{t}{t_{\mathrm{d}}}}\right)\right)$

Oxidation capacity:

(2)$Q_{\mathrm{o},\mathrm{l}\mathrm{i}\mathrm{m}}=Q_{o,\mathrm{m}\mathrm{a}\mathrm{x}}{\displaystyle \frac{C_{\mathrm{o}}}{K_{\mathrm{o}}+C_{\mathrm{o}}}}{\displaystyle \frac{K_i}{K_i+C_{\mathrm{e}}}}$

Specific growth rate limit:

(3)$Q_{\mathrm{s},\mathrm{l}\mathrm{i}\mathrm{m}}={\displaystyle \frac{{\mu }_{\mathrm{c}\mathrm{r}}}{Y_{\mathrm{x}/\mathrm{s}}^{\mathrm{o}\mathrm{x}}}}$

Oxidation glucose metabolism:

(4)$Q_{\mathrm{s},\mathrm{o}\mathrm{x}}=\min \left(\begin{array}{c}{Q}_{\mathrm{s}}\\ Q_{\mathrm{s},\mathrm{l}\mathrm{i}\mathrm{m}}\\ Q_{\mathrm{o},\mathrm{l}\mathrm{i}\mathrm{m}}/Y_{\mathrm{o}/\mathrm{s}}\end{array}\right)$

Reductive glucose metabolism:

(5)$Q_{\mathrm{s},\mathrm{r}\mathrm{e}\mathrm{d}}=Q_{\mathrm{s}}-Q_{\mathrm{s},\mathrm{o}\mathrm{x}}$

Ethanol uptake rate:

(6)$Q_{e,up}=Q_{\mathrm{e},\mathrm{m}\mathrm{a}\mathrm{x}}\left({\displaystyle \frac{C_{\mathrm{e}}}{K_{\mathrm{e}}+C_{\mathrm{e}}}}\right)\left({\displaystyle \frac{K_i}{K_i+C_{\mathrm{e}}}}\right)$

Oxidative ethanol metabolism:

(7)$Q_{\mathrm{e},\mathrm{o}\mathrm{x}}=\min \left(\begin{array}{c}{Q}_{\mathrm{e},\mathrm{u}\mathrm{p}}\\ \left(Q_{\mathrm{o},\mathrm{l}\mathrm{i}\mathrm{m}}-Q_{\mathrm{s},\mathrm{o}\mathrm{x}}{Y}_{\mathrm{o}/\mathrm{s}}\right)Y_{\mathrm{e}/\mathrm{o}}\end{array}\right)$

Ethanol production rate:

(8)$Q_{\mathrm{e},\mathrm{p}\mathrm{r}}=Q_{\mathrm{s},\mathrm{r}\mathrm{e}\mathrm{d}}{Y}_{\mathrm{e}/\mathrm{s}}$

Total specific growth rate:

(9)$\mu =Q_{\mathrm{s},\mathrm{o}\mathrm{x}}{Y}_{\mathrm{x}/\mathrm{s}}^{\mathrm{o}\mathrm{x}}+Q_{\mathrm{s},\mathrm{r}\mathrm{e}\mathrm{d}}{Y}_{\mathrm{x}/\mathrm{s}}^{\mathrm{r}\mathrm{e}\mathrm{d}}+Q_{\mathrm{e},\mathrm{o}\mathrm{x}}{Y}_{\mathrm{x}/\mathrm{e}}$

Carbon dioxide production rate:

(10)$Q_{\mathrm{c}}=Q_{\mathrm{s},\mathrm{o}\mathrm{x}}{Y}_{\mathrm{c}/\mathrm{s}}^{\mathrm{o}\mathrm{x}}+Q_{\mathrm{s},\mathrm{r}\mathrm{e}\mathrm{d}}{Y}_{\mathrm{c}/\mathrm{s}}^{\mathrm{r}\mathrm{e}\mathrm{d}}+Q_{\mathrm{e},\mathrm{o}\mathrm{x}}{Y}_{\mathrm{c}/\mathrm{e}}$

Oxygen consumption rate:

(11)$Q_{\mathrm{o}}=Q_{\mathrm{s},\mathrm{o}\mathrm{x}}{Y}_{\mathrm{o}/\mathrm{s}}+Q_{\mathrm{e},\mathrm{o}\mathrm{x}}{Y}_{\mathrm{o}/\mathrm{e}}$

Respiratory quotient:

(12)$RQ=Q_{\mathrm{c}}/Q_{\mathrm{o}}$

In the above equations, Q_i is the specific consumption or production rate of $i$ ($\mathrm{g}{\mathrm{g}}^{-1}{\mathrm{h}}^{-1}$); C_i is concentration of the ith component (g L⁻¹); K_e is the saturation constant for ethanol (g L⁻¹); K_i is the inhibition constant (g L⁻¹); K_o is the saturation constant for oxygen (g L⁻¹); K_s is the saturation constant for the substrate (glucose) (g L⁻¹); t is time (h); t_d is the time delay (h); the subscripts or superscripts c, cr, e, lim, max, o, ox, red, s, up, and x represent carbon dioxide, critic, ethanol, limitation, maximum, oxygen, oxidative, reductive, substrate (glucose), uptake, and biomass, respectively; and μ is the specific growth rate (h⁻¹).

The reactor was modeled as isothermal. The dynamic model is outlined in Equations (13)–(18).

(13)${\displaystyle \frac{d{C}_{\mathrm{s}}}{dt}}={\displaystyle \frac{F}{V}}\left(S_{\mathrm{o}}-C_{\mathrm{s}}\right)-\left({\displaystyle \frac{\mu }{Y_{\mathrm{x}/\mathrm{s}}^{\mathrm{o}\mathrm{x}}}}+{\displaystyle \frac{Q_{\mathrm{e},\mathrm{p}\mathrm{r}}}{Y_{\mathrm{e}/\mathrm{s}}}}+Q_{\mathrm{m}}\right)C_{\mathrm{x}}$

(14)${\displaystyle \frac{d{C}_{\mathrm{o}}}{dt}}=-Q_{\mathrm{o}}{C}_{\mathrm{x}}+k_L{a}_{\mathrm{o}}\left(C_{\mathrm{o}}^{\ast }-C_{\mathrm{o}}\right)-{\displaystyle \frac{F}{V}}{C}_{\mathrm{o}}$

(15)${\displaystyle \frac{d{C}_{\mathrm{e}}}{dt}}=\left(Q_{\mathrm{e},\mathrm{p}\mathrm{r}}-Q_{\mathrm{e},\mathrm{o}\mathrm{x}}\right)C_{\mathrm{x}}-{\displaystyle \frac{F}{V}}{C}_{\mathrm{e}}$

(16)${\displaystyle \frac{d{C}_{\mathrm{x}}}{dt}}=\mu C_{\mathrm{x}}-{\displaystyle \frac{F}{V}}{C}_{\mathrm{x}}$

(17)${\displaystyle \frac{dV}{dt}}=F$

(18)$k_L{a}_{\mathrm{o}}=113{\left({\displaystyle \frac{F_a}{A_{\mathrm{R}}}}\right)}^{0.25}$

where C_s, C_o, C_e, and C_x represent, respectively, the concentrations of glucose, oxygen, ethanol, and biomass; F and F_a stand for feed rate and air feed rate, respectively; k_La_o is the total volumetric mass transfer coefficient (h⁻¹); V is the volume of the content in the reactor; and A_R denotes the cross-sectional area of the reactor.

The values of the parameters used in the model are given in Table 1. The initial conditions used in the model are given in Table 2. A simulation based on this model was built in MATLAB [28] and was used to generate historical batch data and test new feed profiles obtained from the optimization process.

2.2. Extreme Learning Machine

An example of a single-hidden-layer feedforward neural network (SLFN) is shown in Figure 1. An input layer consisting of n input neurons, x, is connected to a hidden layer of $\tilde{N}$ neurons via a series of hidden-layer weights, w, and biases, b. The output values of the hidden-layer neurons are computed using these inputs, weights, and biases in an activation function. The value of the output-layer neurons is then calculated using the linear relationship between the hidden-layer outputs and the output-layer weights, β.

For the SLFN used in this investigation, each input neuron corresponds to the flowrate at the respective feed interval, and the single output neuron represents the final biomass concentration.

The extreme learning machine (ELM), proposed by Huang et al. [23], is a learning algorithm for an SLFN. They proved that the method was several orders of magnitude faster and could achieve improved generalization performance, when compared to other methods (such as back propagation). The main principle of ELM is that the hidden-layer weights and biases are chosen randomly.

For N distinct samples, (${\mathbf{x}}_j,{\mathbf{t}}_j$), where ${\mathbf{x}}_j={\left[x_{j1},x_{j2},\dots ,x_{jn}\right]}^{\mathrm{T}}\in R^n$is the $j$th set of input values and ${\mathbf{t}}_j={\left[t_{j1},t_{j2},\dots ,t_{jm}\right]}^{\mathrm{T}}\in R^m$is the $j$th set of target values, the hidden-layer output matrix of the neural network is given by $\mathbf{H}$.

(19)$\begin{array}{c}\mathbf{H}\left({\mathbf{w}}_1,\dots ,{\mathbf{w}}_{\tilde{N}},b_1,\dots ,b_{\tilde{N}},{\mathbf{x}}_1,\dots ,{\mathbf{x}}_N\right)\hfill \\ \hfill ={\left[\begin{array}{ccc}g\left({\mathbf{w}}_1·{\mathbf{x}}_1+b_1\right)& \dots & g\left({\mathbf{w}}_{\tilde{N}}·{\mathbf{x}}_1+b_{\tilde{N}}\right)\\ ⋮& \ddots & ⋮\\ g\left({\mathbf{w}}_1·{\mathbf{x}}_N+b_1\right)& \dots & g\left({\mathbf{w}}_{\tilde{N}}·{\mathbf{x}}_N+b_{\tilde{N}}\right)\end{array}\right]}_{N\times \tilde{N}}\end{array}$

where ${\mathbf{w}}_i={\left[w_{i1},w_{i2},\dots ,w_{in}\right]}^{\mathrm{T}},i=1,\dots ,\tilde{N}$ is the weight vector connecting the input neurons to the $i$th hidden-layer neuron, and $b_i$ is the bias for the $i$th hidden neuron.

Using ELM training, the hidden-layer weights and biases are chosen randomly and are used in the activation function $g\left(x\right)$, along with the input sample values, to compute $\mathbf{H}$. The output neurons are linear in nature for ELM such that,

(20)$\sum _{i=1}^{\tilde{N}}{\mathit{\beta }}_i{\mathbf{H}}_{ji}=\sum _{i=1}^{\tilde{N}}{\mathit{\beta }}_{i}g\left({\mathbf{w}}_i·{\mathbf{x}}_j+b_i\right)={\mathbf{o}}_j,$

where ${\mathit{\beta }}_i={\left[{\beta }_{i1},{\beta }_{i2},\dots ,{\beta }_{im}\right]}^{\mathrm{T}},i=1,\dots ,\tilde{N}$, is the weight vector connecting the $i$th hidden-layer neuron to the output neurons, and ${\mathbf{o}}_j$ is the network output for the $j$th sample.

So long as the activation function is infinitely differentiable, this SLFN can approximate the N samples with zero error ${\sum }_{j=1}^N\left|\left|{\mathbf{o}}_j-{\mathbf{t}}_j\right|\right|=0$, i.e., a properly trained network with correct ${\mathit{\beta }}_i$, ${\mathit{w}}_i$, and $b_i$ suggests the following:

(21)$\sum _{i=1}^{\tilde{N}}{\mathit{\beta }}_{i}g\left({\mathbf{w}}_i·{\mathbf{x}}_j+b_i\right)={\mathbf{t}}_j,$

This equation can be written more simply as follows:

(22)$\mathbf{H}\mathit{\beta }=\mathbf{T}$

$\mathit{\beta }={\left[\begin{array}{c}\begin{array}{c}{\mathit{\beta }}_1^T\\ ⋮\end{array}\\ {\mathit{\beta }}_{\tilde{N}}^T\end{array}\right]}_{\tilde{N}\times m}\mathrm{a}\mathrm{n}\mathrm{d}\mathbf{T}={\left[\begin{array}{c}{\mathbf{t}}_1^T\\ ⋮\\ {\mathbf{t}}_N^T\end{array}\right]}_{N\times m}$

The hidden-layer output matrix, $\mathbf{H}$, can be calculated simply as shown in Equation (19), and the matrix of target output values, $\mathbf{T}$, is already known. Therefore, training the SLFN involves finding the least-squares solution $\widehat{\mathit{\beta }}$ of the linear system given in Equation (22). The matrix of weights connecting the hidden layer to the output layer, $\beta$, is calculated as follows:

(23)$\widehat{\mathit{\beta }}={\mathbf{H}}^{†}\mathbf{T}$

where ${\mathbf{H}}^{†}$ is the Moore–Penrose generalized inverse of matrix $\mathbf{H}$.

Due to there often being an unequal number of hidden neurons, $\tilde{N}$, and distinct training samples, $N$, $\mathbf{H}$ is rarely a square matrix. Therefore, the hidden-layer output matrix, $\mathbf{H}$, is noninvertible, and so the Moore–Penrose generalized inverse is used to compute the system in Equation (22). Singular value decomposition (SVD) is the method used to solve the Moore–Penrose generalized inverse.

2.3. Recursive Least Squares

Recursive least squares (RLS) is used to update the ELM model parameters following each subsequent batch. It works in such a way that the parameters calculated from N observations are used in the estimation of parameters for N + 1 observations. This is more computationally efficient than simply recalculating the least squares after each observation from scratch [29].

Consider a system of historical data based on N pairs of input and output data,

$\mathbf{X}\left(N\right)=\left[\begin{array}{c}{\mathbf{x}}_1^{\mathrm{T}}\\ ⋮\\ {\mathbf{x}}_N^{\mathrm{T}}\end{array}\right]\mathbf{y}\left(N\right)=\left[\begin{array}{c}{y}_1\\ ⋮\\ y_N\end{array}\right]$

where ${\mathbf{x}}_N^{\mathrm{T}}$ is a column vector of system input values, and $y_N$ is the scalar system output for the $N$th measurement. The system is modeled by the following linear model:

(24)$\mathbf{y}=\mathbf{X}\mathsf{\theta }+\mathsf{\epsilon }$

The model parameters relating $\mathbf{y}$ to $\mathbf{X}$ can be estimated through least-squares estimation by minimizing the loss function as follows:

(25)$\min J\left(\mathsf{\theta }\right)={\left|\left|\mathsf{\epsilon }\right|\right|}^2$

where $\mathsf{\epsilon }=\mathbf{y}-\widehat{\mathbf{y}}$ are the estimation errors, and $\widehat{\mathbf{y}}=\mathbf{X}\mathsf{\theta }$ are the estimated values. The solution to the minimization is to find an estimation of the parameters, $\widehat{\mathsf{\theta }}$, that satisfies the following equation:

(26)$\mathbf{y}=\mathbf{X}\widehat{\mathsf{\theta }}$

By left multiplying both sides of the above equation by ${\mathbf{X}}^{\mathrm{T}}$ and then left multiplying both sides by (X^TX)⁻¹, the least-squares solution is given by the following:

(27)$\widehat{\mathsf{\theta }}={\left({\mathbf{X}}^{\mathrm{T}}\mathbf{X}\right)}^{-1}{\mathbf{X}}^{\mathrm{T}}\mathbf{y}$

Upon obtaining additional measurements of the system, the matrix $\mathbf{X}$ is expanded by a row, and the vector $\mathbf{y}$ is expanded by an element as follows:

$\mathbf{X}\left(N+1\right)=\left[\begin{array}{c}\mathbf{X}\left(N\right)\\ {\mathbf{x}}_{N+1}^{\mathrm{T}}\end{array}\right]\mathbf{y}\left(N+1\right)=\left[\begin{array}{c}\mathbf{y}\left(N\right)\\ y_{N+1}\end{array}\right]$

The model parameter estimation from Equation (27) can then be written as follows:

(28)$\begin{array}{c}\widehat{\mathsf{\theta }}\left(N+1\right)={\left[{\mathbf{X}}^{\mathrm{T}}\left(N+1\right)\mathbf{X}\left(N+1\right)\right]}^{-1}{\mathbf{X}}^{\mathrm{T}}\left(N+1\right)\mathbf{y}\left(N+1\right)\hfill \\ \hfill ={\left[{\mathbf{X}}^T\left(N\right)\mathbf{X}\left(N\right)+{\mathbf{x}}_{N+1}{\mathbf{x}}_{N+1}^{\mathrm{T}}\right]}^{-1}\left({\mathbf{X}}^T\left(N\right)\mathbf{y}\left(N\right)+{\mathbf{x}}_{N+1}{y}_{N+1}\right)\end{array}$

Assuming the matrix ${\mathbf{X}}^{\mathrm{T}}\left(N\right)\mathbf{X}\left(N\right)$ is positive definite, the recursive least squares can be solved from the following equations:

(29)$\widehat{\mathsf{\theta }}\left(N+1\right)=\widehat{\mathsf{\theta }}\left(N\right)+\mathbf{K}\left(N\right)e\left(N+1\right)$

(30)$e\left(N+1\right)=y_{N+1}-{{\mathbf{x}}_{N+1}}^{\mathrm{T}}\widehat{\mathsf{\theta }}\left(N\right)$

(31)$\mathbf{K}\left(N\right)=\mathbf{P}\left(N\right){{\mathbf{x}}_{N+1}}^{\mathrm{T}}{\left[1+{{\mathbf{x}}_{N+1}}^{\mathrm{T}}\mathbf{P}\left(N\right){\mathbf{x}}_{N+1}\right]}^{-1}$

(32)$\mathbf{P}\left(N+1\right)=\left[\mathbf{I}-\mathbf{K}\left(N\right){{\mathbf{x}}_{N+1}}^{\mathrm{T}}\right]\mathbf{P}\left(N\right)$

(33)$\mathbf{P}\left(0\right)=\mathbf{I}·P_0\left(P_0>100{\sigma }^2\right)$

For time-varying systems, it is necessary to introduce a forgetting factor, $\lambda$ (typically $0.9<\lambda \le 1$), which works to reduce the influence of old data. A smaller forgetting factor will “forget” the old data faster. Using a forgetting factor, Equations (31) and (32) become

(34)$\mathbf{K}\left(N\right)=\mathbf{P}\left(N\right){{\mathbf{x}}_{N+1}}^{\mathrm{T}}{\left[\lambda +{{\mathbf{x}}_{N+1}}^{\mathrm{T}}\mathbf{P}\left(N\right){\mathbf{x}}_{N+1}\right]}^{-1}$

(35)$\mathbf{P}\left(N+1\right)=\left[\mathbf{I}-\mathbf{K}\left(N\right){{\mathbf{x}}_{N+1}}^{\mathrm{T}}\right]\mathbf{P}\left(N\right)/\lambda$

Equations (29), (30), (34) and (35) need to be updated after each subsequent batch. The new ELM parameters (output-layer weights and bias) are taken from $\widehat{\mathsf{\theta }}\left(N+1\right)=\left[{\mathit{\beta }}^{\mathrm{T}},b_2\right]$, where b₂ is the bias for the output layer neuron.

2.4. Batch-to-Batch Optimization Control

The aim of batch-to-batch optimization control is to maximize the amount of biomass at the end of the batch by adjusting the substrate feed profile. In order to carry out the optimization, an ELM model linking the batch feed profile and the final biomass concentration at the end of a batch is developed. The ELM model is of the following form:

(36)$y=f\left(\mathbf{U}\right)$

where y is the final biomass concentration at the end of a batch; U = (u₁, u₂, …, u₈) is the feed profile for the batch; and f() is a nonlinear function represented by an ELM. Here, we divide the batch duration into 8 equal intervals, and the substrate feed rate is kept constant during each interval. Hence, the feed profile is a vector containing 8 elements.

For the calculation of the optimal feed profile for the current batch (the kth batch), U_k, the optimization uses the feed profile from the previous batch, U_k−1, as the initial search value. This feed profile is applied as the inputs to the ELM model in order to predict the final biomass concentration, which is used in a numerical optimization procedure. Therefore, the ELM model has 8 input neurons and one output neuron. The final volume is also calculated and multiplied with the predicted final biomass concentration to find the final amount of biomass. The optimization problem is solved using the MATLAB function, “fmincon”, from the MATLAB Optimization Toolbox and is outlined by Equation (37). Note that minimization is typically considered in optimization tools, so a negative sign is added to the predicted amount of the final biomass to convert the maximization problem into a minimization problem. The optimization solution is subject to the constraints that the feed at each interval cannot exceed 2600 L h⁻¹, and the final volume in the reactor should not exceed the maximum.

(37)$\underset{\mathbf{U}}{\min }J=-Vy=-Vf\left(\mathbf{U}\right)=-\left(V_0+\sum _{i=1}^8{u}_i{t}_s\right)f\left(\mathbf{U}\right)$

$\mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}:\left\{\begin{array}{c}0\le u_i\le 2600\left(\mathrm{L}{\mathrm{h}}^{-1}\right)\\ V<V_f\left(\mathrm{L}\right)\end{array}\right.$

In the above equation, V₀ is the initial volume of the content in the reactor at the start of a batch; V is the final volume of the content in the reactor at the end of a batch; U is the feed profile; and t_s is the interval length (or sampling time).

It should be noted that this optimization control strategy is not a conventional feedback control strategy in which a manipulated variable is continuously adjusted by a feedback controller. In this optimization control strategy, an ELM model of the batch process is used in an optimization framework to find the best feeding profile.

2.5. Batch-to-Batch Optimization Control Strategy Integrating ELM and RLS

In this section, the batch-to-batch optimizing control strategy integrating ELM and RLS for the fed-batch fermentation process is outlined. The RLS updates the parameters of the ELM model, i.e., the output-layer weights and bias, by recursively solving the least-squares problem. This will work to amend any plant–model mismatches and the effects of unknown disturbances. The output-layer weights and bias of the ELM for the previous batch are used as the initial guess for the RLS procedure. The parameters are updated iteratively in the RLS algorithm. Combining ELM with RLS is beneficial when used on real plant systems due to the historical process data often being inaccurate or insufficient. The updated ELM model can be then used for the optimization of the next batch. The optimization will benefit from the ELM model being up-to-date and accurate. Figure 2 gives the framework of the proposed approach, and the details are explained below.

To address the situation in which only limited historical process operation data are available, we started with a small number of historical batches. A set of 25 feed profiles, each consisting of 8 feed intervals, was created. Each interval had a duration of 2 h, and the feed rate was kept constant throughout the interval. These feed profiles were generated by adding uniformly distributed random variations (up to ±50 L h⁻¹) to a baseline profile to emulate how different operators might have different batch strategies based on their own experience. Figure 3 shows all the 25 feed profiles used. Due to the large number of curves, legends have not been added to the figure. It can be seen from Figure 3 that these feed profiles have a general increasing trend as the batch progresses. This is consistent with the general feed profiles for this process reported in previous studies [3,27,28,30]. Each feed profile was simulated in the fed-batch fermentation mechanistic model to find the corresponding final biomass concentrations. To represent real processes in which measurement noise always exists, normally distributed noise with zero mean and a standard deviation of 0.1 (g L⁻¹) was added to the final biomass concentrations. The data in this study were generated through simulation. In practical applications, the substrate feed rate can be measured using a flow meter, and the biomass concentration at the end of a batch can be either measured on-line or through laboratory analysis.

The historical batch data were normalized by scaling to zero mean and unit variance. They were then randomly partitioned into training, validation, and unseen testing data sets. The number of hidden neurons in the ELM model was determined using cross validation. The ELM model was trained using the training data set and with different numbers of hidden neurons. In this study, we consider the number of hidden neurons to be in the range of 1 to 60. The number of hidden neurons that resulted in a low combined root-mean-square error (RMSE) in the training and validation data was selected. Figure 4 shows how the modeling error changed with the number of hidden neurons. The final selected ELM model had 35 hidden neurons based on Figure 4. Note that, if the final selected number of hidden neurons is close to 60 (the upper end), then the upper end of the range should be extended. A plot of the predicted against the actual final biomass concentration, showing the performance of the trained ELM model on all 3 data sets, is given in Figure 5. Note that the values shown in the plot are normalized values.

A given batch feed profile was run in both the fed-batch simulation and the trained ELM model. In order for the feed profile to run in the fed-batch simulation, it was scaled back to the original scale. The prediction error of the ELM model was used in the RLS updating algorithm, i.e., Equations (29), (30), (34) and (35), to update the output-layer weights and bias of the ELM model. The updated model was then used in the optimization procedure to find an optimal feed profile for the next batch. The optimization yields a new feed profile that provides the maximum amount of the final biomass that the current model can achieve, subject to the constraints of the optimization. The process was repeated from batch to batch, using the new optimized feed profile with each batch. The reason for continuously updating the model is that it can then handle unknown disturbances. Also, in the case of limited historical batches for training the model, continuously updating the model will help it to adapt and improve its generalization capabilities.

As ELM models can be trained quickly when the number of data are not large, model redevelopment is feasible when the number of data are not large. However, when the number of data are large, which is the case in practical situations in which a batch process can run for years, re-developing an ELM model after each batch with a large number of data is costly. This issue can be addressed by recursively updating the ELM model from batch to batch using RLS.

3. Results and Discussions

3.1. Adaptive Modeling

To assess the RLS model update, a series of feed profiles, similar to the ones used to train the ELM model, were run through both the fed-batch simulation and the ELM model. The prediction error of the model was calculated after each batch. Recursive updating of the model parameters through RLS was performed after each batch. An unmeasured disturbance in the feed concentration was introduced from the 51st batch. The prediction error of the model for each subsequent batch is plotted in Figure 6, which also shows the results of this process when RLS model updating was not performed. Prior to the disturbance, the systems with and without RLS updating performed about equally with root-mean-square errors (RMSEs) of 0.075 and 0.079, respectively. The RMSE of the ELM model with batch-to-batch updating was only marginally lower than that of the ELM model with batch-to-batch updating. Even with the recursive model updating, the prediction error was unable converge to zero due to the added measurement noise. A small number of historical training batches (25 batches) were used to build the model; therefore, it is expected that the predictive capability of the ELM model will improve when more batches are performed and used to train the model through RLS updating. When Alli and Zhang [21] performed a similar experiment on a different case study, the prediction error remained steady and close to zero both with and without RLS model updating. This is more likely due to the relatively small measurement noise used in [21].

Following the disturbance, the system without the RLS updating continually overpredicted the final biomass concentration and was unable to correct itself. This is apparent in the prediction error remaining below zero. For the system with RLS model updating, the prediction error reduced back toward zero. However, the error does not truly converge to zero probably due to the added measurement noise, and it takes 30–40 batches for the prediction accuracy to return to a similar level to that of batches before the disturbance occurred. The RLS system was able to successfully modify the ELM model output-layer weights so that the prediction error for the following batches returned to near zero. Overall, Figure 6 shows that RLS model updating is able to improve the accuracy of the ELM model following a disturbance, but the performance for this system still has large room for improvement. A possible reason for this is that the RLS was not set up correctly, with suboptimal values used for the forgetting factor or P₀. The use of a forgetting factor is important as it causes the RLS algorithm to put more focus on the most recent batch data. This is significant, especially when there is a disturbance to the system, as it helps the updated model to better represent the current process. Another likely reason could be that the historical data used to train the model were not sufficient. The feed profiles could have been too varied or not varied enough, and the volume of data (25 batches) may not have been sufficient. This highlights a possible issue of ELM modeling and data-driven modeling as a whole: the accuracy of a model is reliant on it being provided with “good” data—which is not always possible, especially in industrial environments.

3.2. Batch-to-Batch Optimization with RLS Model Updating

The final biomass concentrations from both the ELM model and the fed-batch simulation for the batch-to-batch optimization with RLS model updating outlined in Section 2.5 is shown in Figure 7. A change in the substrate feed concentration (325 g L⁻¹ to 322 g L⁻¹) was introduced from batch 11 to represent an unknown disturbance. It should be noted that the disturbance is typically associated with raw material variations, and the disturbance typically lasts for a consecutive number of batches. For the first iteration (batch) of the process, the final biomass concentrations of the simulation and the model prediction were very similar as shown in Figure 8. This is because the first feed profile used in batch 1 came from the training data, and so the ELM model was well fitted for it. In the next iteration (batch), the feed profile was optimized using the updated ELM model. As a result, the final biomass concentration improved for both simulation and ELM model prediction. However, the ELM model significantly overpredicted and caused a large prediction error. This is due to the new optimized feed profile being different from the training data and so being outside the bounds of the ELM model’s predictive capabilities. For the next batch, the ELM model’s prediction was much closer to the result of simulation. This shows that the model was successfully updated through RLS so that it could better predict the final biomass concentration for the new optimized profiles. The prediction error continued to improve with each batch and converged to zero after six batches. The unmeasured disturbance to the feed concentration introduced from batch 11 caused the final biomass concentration to decrease. This caused the ELM model to overpredict again. However, RLS was again able to update the model parameters so that after about eight batches, the prediction error was back close to zero. The final biomass concentration following the disturbance was not as large as what was achieved prior to the disturbance. This was likely due to the physical limitations of the new conditions.

Figure 8a shows how the mass of biomass at the end of the batch changed with each iteration of the batch-to-batch optimization, with and without RLS model updating. The new optimized feed profile in batch 2 caused large model prediction errors for both systems. The system that used RLS model updating was able to use the prediction error to update the parameters of the ELM model and return the prediction error back to close to zero in the subsequent batches. However, the large prediction error for the system without RLS updating persisted as the ELM model was unable to adapt. The objective function in the optimization was then based on this inaccurate model and resulted in the new generated feed profile not being optimal for the true fermentation process. This resulted in the final amount of biomass being lower than that from the system with RLS updating, which used a more accurate model in the optimization. The performance also could not improve after the second batch as the model was not updated and so the optimization was being given the same problem to solve. The batch-to-batch optimization without RLS model updating was also unable to adapt to the disturbance introduced from batch 11 as shown in Figure 8. In contrast, the batch-to-batch optimization with the batchwise-updated ELM model was able to overcome the effect of disturbance and quickly converged to stable operation (after about five batches). It can be seen from Figure 8 that the final amount of biomass under the batch-to-batch optimization control was about 40 kg higher than that for the scheme with the fixed ELM model. Figure 9 shows the optimized feed profiles for batch 10 (before the introduction of disturbance) and batch 30 (after the effect of disturbance has been captured by the RLS model updating). It can be seen from Figure 9 that only some small adjustment in the feed profile during the second, third, and fifth stages (i.e., 2–6 h and 8–10 h) were required to overcome the effect of the unknown disturbance.

3.3. Improvement Suggestions

The new optimized feed profiles shown in Figure 9 differed substantially from the historical training feed profiles shown in Figure 3. Prior to any RLS model updating, the model struggled to predict the final biomass concentration for these new feed profiles, as shown by the large prediction error for batch 2 in Figure 8b. Rebuilding the model by including these feed profiles in the training data should improve its predictive capabilities.

In this study, 25 historical batches were used to build the model, which is a small number of data for model building. Simply using more historical batches to build the ELM model could improve its performance. However, using too much training data means the requirement of many batches of process operation data. This means more time and money need to be spent collecting the data. The smaller the number of batches used, the sooner the model can be built and implemented. Therefore, a balance must be found for a suitable number of historical batches.

The ELM model only predicted the final biomass concentration of the desired product and was used in the optimization to maximize the amount of this product at the end of the batch. The new optimized feed profiles may be causing the excessive production of undesirable products. In order to address this, the ELM model could be trained with two outputs in the output layer: the final desired product concentration and an undesired by-product concentration. The objective function could be adapted to maximize the amount of desired product whilst minimizing the amount of undesired by-product. Alternatively, two ELM models, one for each product, could be built and updated separately.

Baron and Zhang [30] showed that combining multiple ELM models, developed from bootstrap re-sampling copies of the original training data, improved the accuracy of the model. Additionally, the confidence bounds of the model predictions can be found and used in the optimization to improve the reliability of the optimal feed profile. Using a bootstrap-aggregated extreme learning machine (BA-ELM) with the RLS model updating could provide further improved results.

4. Conclusions

Batch-to-batch optimization control based on a recursively updated extreme learning machine (ELM) model of a fed-batch fermentation process is presented in this paper. ELM is a type of single-hidden-layer feedforward neural network and uses randomly assigned hidden-layer weights and biases, which makes model training very fast. Iterative learning in the form of batch-to-batch updating of the output-layer weights and bias through recursive least squares (RLS) improved the modeling performance. The predictive performance of the batchwise-updated ELM model was greater than that of the original ELM model, especially under the presence of an unknown disturbance. The batchwise-updated ELM model was also able to track the influence of unknown disturbances (variations in substrate feed concentration) by adapting the model to the new conditions. An offline optimization was performed on the updated model after each subsequent batch and was able to improve the final mass of biomass for the next batch. The batch-to-batch optimization policy was also able to respond to unknown disturbances (variations in substrate feed concentration) by altering the feed profile so that the batch product quality following the disturbance returned to a level similar to that achieved before the disturbance. It is shown that the batch-to-batch optimization with the updated ELM model could produce 40 kg more final biomass each batch than that without model updating when the feed concentration was disturbed from its nominal value of 325 g L⁻¹ to 322 g L⁻¹. The effectiveness of the proposed modeling and optimization method has been demonstrated on a simulated fed-batch fermentation process and shows how it can be used to handle process–model mismatch and unknown disturbances.

Author Contributions

Conceptualization, A.M. and J.Z.; methodology, A.M. and J.Z.; software, A.M.; validation, A.M.; formal analysis, A.M.; investigation, A.M.; resources, J.Z.; data curation, A.M.; writing—original draft preparation, A.M.; writing—review and editing, J.Z.; visualization, A.M.; supervision, J.Z.; project administration, J.Z.; funding acquisition, J.Z. All authors have read and agreed to the published version of the manuscript.

Data Availability Statement

Data will be made available on request.

Conflicts of Interest

The authors declare no conflicts of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. A single-hidden-layer feedforward neural network (SLFN) configured appropriately for training by ELM.

Figure 2. Flow diagram of the proposed batch-to-batch optimization control strategy integrating ELM and RLS: (a). data generation; (b). ELM modeling; (c). batch-to-batch optimization with ELM model updating using RLS.

Figure 3. Feed profiles of the historical batches used to build the ELM model.

Figure 4. RMSE of the model prediction for the training and validation data sets.

Figure 5. Predicted (ELM) vs. actual (fed-batch simulation) final biomass concentration (normalized values) for the training, validation, and unseen testing data sets.

Figure 6. Prediction error of the ELM model for recursive batches with and without RLS model updating. An unmeasured disturbance is introduced from the 51st batch.

Figure 7. Actual (fed-batch simulation) and predicted (ELM model) final biomass concentration for the ELM with RLS model updating system. An unmeasured disturbance is introduced from the 11th batch.

Figure 8. Final mass of biomass from the fed-batch simulation (a) and model prediction error (b) for the system with and without RLS model updating. An unmeasured disturbance is introduced from the 11th batch.

Figure 9. The optimized feed profiles for batch 10 and batch 30.

Table 1

Parameter values used in the fed-batch fermentation model.

Parameters	Value	Parameters	Value
$K_{\mathrm{e}}\left(\mathrm{g}{\mathrm{L}}^{-1}\right)$	0.1	$Y_{\mathrm{e}/\mathrm{o}}\left(\mathrm{g}{\mathrm{g}}^{-1}\right)$	1.1236
$K_{\mathrm{o}}\left(\mathrm{g}{\mathrm{L}}^{-1}\right)$	9.6 × 10−5	$Y_{\mathrm{e}/\mathrm{s}}\left(\mathrm{g}{\mathrm{g}}^{-1}\right)$	0.4859
$K_i\left(\mathrm{g}{\mathrm{L}}^{-1}\right)$	3.5	$Y_{\mathrm{x}/\mathrm{e}}\left(\mathrm{g}{\mathrm{g}}^{-1}\right)$	0.7187
$K_{\mathrm{s}}\left(\mathrm{g}{\mathrm{L}}^{-1}\right)$	0.612	$Y_{\mathrm{o}/\mathrm{s}}\left(\mathrm{g}{\mathrm{g}}^{-1}\right)$	0.3857
$Y_{\mathrm{x}/\mathrm{s}}^{\mathrm{o}\mathrm{x}}\left(\mathrm{g}{\mathrm{g}}^{-1}\right)$	0.585	$Y_{\mathrm{c}/\mathrm{e}}\left(\mathrm{g}{\mathrm{g}}^{-1}\right)$	0.6450
$Y_{\mathrm{x}/\mathrm{s}}^{\mathrm{r}\mathrm{e}\mathrm{d}}\left(\mathrm{g}{\mathrm{g}}^{-1}\right)$	0.050	$Y_{\mathrm{o}/\mathrm{e}}\left(\mathrm{g}{\mathrm{g}}^{-1}\right)$	0.8904
$Y_{\mathrm{c}/\mathrm{s}}^{\mathrm{o}\mathrm{x}}\left(\mathrm{g}{\mathrm{g}}^{-1}\right)$	0.5744	$S_o\left(\mathrm{g}{\mathrm{L}}^{-1}\right)$	325
$Y_{\mathrm{c}/\mathrm{s}}^{\mathrm{r}\mathrm{e}\mathrm{d}}\left(\mathrm{g}{\mathrm{g}}^{-1}\right)$	0.4620	$C_o^{\ast }\left(\mathrm{g}{\mathrm{L}}^{-1}\right)$	0.006
$Q_{\mathrm{o},\mathrm{m}\mathrm{a}\mathrm{x}}\left(\mathrm{g}{\mathrm{g}}^{-1}{\mathrm{h}}^{-1}\right)$	0.255	${\mu }_{\mathrm{c}\mathrm{r}}\left({\mathrm{h}}^{-1}\right)$	0.21
$Q_{s,\mathrm{m}\mathrm{a}\mathrm{x}}\left(\mathrm{g}{\mathrm{g}}^{-1}{\mathrm{h}}^{-1}\right)$	2.943	$A_{\mathrm{R}}\left({\mathrm{m}}^2\right)$	12.56
$Q_{\mathrm{e},\mathrm{m}\mathrm{a}\mathrm{x}}\left(\mathrm{g}{\mathrm{g}}^{-1}{\mathrm{h}}^{-1}\right)$	0.238	$t_{\mathrm{d}}\left(\mathrm{h}\right)$	2
$Q_{\mathrm{m}}\hspace{1em}\left(\mathrm{g}{\mathrm{g}}^{-1}{\mathrm{h}}^{-1}\right)$	0.03	$F_a\left({\mathrm{m}}^3{\mathrm{h}}^{-1}\right)$	9983.4451

Table 2

Initial conditions and settings used in the fed-batch fermentation model.

Parameters	Value	Parameters	Value
$C_{\mathrm{s}}\left(t=0\right)\left(\mathrm{g}{\mathrm{L}}^{-1}\right)$	7	$V\left(t=0\right)\left(\mathrm{L}\right)$	50,000
$C_{\mathrm{o}}\left(t=0\right)\left(\mathrm{g}{\mathrm{L}}^{-1}\right)$	7.8 × 10−3	$V_{\mathrm{m}\mathrm{a}\mathrm{x}}\hspace{1em}\hspace{1em}\hspace{1em}\left(\mathrm{L}\right)$	100,000
$C_{\mathrm{e}}\left(t=0\right)\left(\mathrm{g}{\mathrm{L}}^{-1}\right)$	0	$t_f\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\left(\mathrm{h}\right)$	16
$C_{\mathrm{x}}\left(t=0\right)\left(\mathrm{g}{\mathrm{L}}^{-1}\right)$	15	$t_{\mathrm{i}\mathrm{n}\mathrm{t}}\hspace{1em}\hspace{1em}\hspace{1em}\left(\mathrm{h}\right)$	2

References

1. Ochoa, S. Fed-Batch Fermentation—Design Strategies. Comprehensive Biotechnology; 3rd ed. Moo-Young, M. Pergamon: Oxford, UK, 2019; pp. 586-600. [DOI: https://dx.doi.org/10.1016/B978-0-444-64046-8.00093-8]

2. Bonvin, D. Optimal operation of batch reactors—A personal view. J. Process Control.; 1998; 8, pp. 355-368. [DOI: https://dx.doi.org/10.1016/S0959-1524(98)00010-9]

3. Jewaratnam, J.; Zhang, J.; Morris, J.; Hussain, A. Batch-to-batch iterative learning control using linearised models with adaptive model updating. Proceedings of the 2012 UKACC International Conference on Control; Cardiff, UK, 3–5 September 2012; pp. 271-276. [DOI: https://dx.doi.org/10.1109/CONTROL.2012.6334641]

4. Zhang, J.; Nguyan, J.; Morris, J.; Xiong, Z. Batch to batch iterative learning control of a fed-batch fermentation process using linearised models. Proceedings of the 2008 10th International Conference on Control, Automation, Robotics and Vision; Hanoi, Vietnam, 17–20 December 2008; pp. 745-750. [DOI: https://dx.doi.org/10.1109/ICARCV.2008.4795610]

5. Thomas, I.M.; Kiparissides, C. Computation of the near-optimal temperature and initiator policies for a batch polymerization reactor. Can. J. Chem. Eng.; 1984; 62, pp. 284-291. [DOI: https://dx.doi.org/10.1002/cjce.5450620217]

6. Souza, F.A.A.; Araújo, R.; Mendes, J. Review of soft sensor methods for regression applications. Chemom. Intell. Lab. Syst.; 2016; 152, pp. 69-79. [DOI: https://dx.doi.org/10.1016/j.chemolab.2015.12.011]

7. Tian, Y.; Zhang, J.; Morris, A.J. Modelling and optimal control of a batch polymerisation reactor using a hybrid stacked recurrent neural network model. Ind. Eng. Chem. Res.; 2001; 40, pp. 4525-4535. [DOI: https://dx.doi.org/10.1021/ie0010565]

8. Chen, L.; Hontoir, Y.; Huang, D.; Zhang, J.; Morris, A.J. Combining first principles with black-box techniques for reaction systems. Control. Eng. Pract.; 2004; 12, pp. 819-826. [DOI: https://dx.doi.org/10.1016/j.conengprac.2003.09.006]

9. Psichogios, D.C.; Ungar, L.H. A hybrid neural network-first principles approach to process modeling. AIChE J.; 1992; 38, pp. 1499-1511. [DOI: https://dx.doi.org/10.1002/aic.690381003]

10. Thompson, M.L.; Kramer, M.A. Modeling chemical processes using prior knowledge and neural networks. AIChE J.; 1994; 40, pp. 1328-1340. [DOI: https://dx.doi.org/10.1002/aic.690400806]

11. von Stosch, M.; Oliveira, R.; Peres, J.; Feyo de Azevedo, S. Hybrid semi-parametric modeling in process systems engineering: Past, present and future. Comput. Chem. Eng.; 2013; 60, pp. 86-101. [DOI: https://dx.doi.org/10.1016/j.compchemeng.2013.08.008]

12. Zhan, Y.; Zhu, J. Response surface methodology and artificial neural network-genetic algorithm for modeling and optimization of bioenergy production from biochar-improved anaerobic digestion. Appl. Energy; 2024; 355, 122336. [DOI: https://dx.doi.org/10.1016/j.apenergy.2023.122336]

13. Sun, P.; Chen, J.; Que, H. A priori knowledge-based dual hierarchical RNN for spatial-temporal process modeling: Using a multitubular reactor as a case study. IEEE Trans. Ind. Inform.; 2024; 20, pp. 899-910. [DOI: https://dx.doi.org/10.1109/TII.2023.3271741]

14. Himmelblau, D.M. Accounts of experiences in the application of artificial neural networks in chemical engineering. Ind. Eng. Chem. Res.; 2008; 47, pp. 5782-5796. [DOI: https://dx.doi.org/10.1021/ie800076s]

15. Misra, S.; Li, H. Comparative study of shallow and deep machine learning models for synthesizing in situ NMR T2 distributions. Machine Learning for Subsurface Characterization; Misra, S.; Li, H.; He, J. Gulf Professional Publishing: Houston, TX, USA, 2020; pp. 219-241. [DOI: https://dx.doi.org/10.1016/B978-0-12-817736-5.00008-9]

16. Ławryńczuk, M. Neural Modelling of a Yeast Fermentation Process Using Extreme Learning Machines; Springer: Berlin/Heidelberg, Germany, 2016; [DOI: https://dx.doi.org/10.1007/978-3-319-29357-8_2]

17. Kulkarni, S.G.; Chaudhary, A.K.; Nandi, S.; Tambi, S.S.; Kulkarni, B.D. Modeling and monitoring of batch processes using principal component analysis (PCA) assisted generalized regression neural networks (GRNN). Biochem. Eng. J.; 2004; 18, pp. 193-210. [DOI: https://dx.doi.org/10.1016/j.bej.2003.08.009]

18. Xiong, Z.; Xu, Y.; Zhang, J.; Dong, J. Batch-to-batch control of fed-batch processes using control-affine feedforward neural network. Neural Comput. Appl.; 2008; 17, pp. 425-432. [DOI: https://dx.doi.org/10.1007/s00521-007-0142-6]

19. Huang, G.-B.; Zhu, Q.-Y.; Siew, C.-K. Extreme learning machine: Theory and applications. Neurocomputing; 2006; 70, pp. 489-501. [DOI: https://dx.doi.org/10.1016/j.neucom.2005.12.126]

20. Sharma, N.; Deo, R. Wind speed forecasting in Nepal using self-organizing map-based online sequential extreme learning machine. Predictive Modelling for Energy Management and Power Systems Engineering; Deo, R.; Samui, P.; Roy, S.S. Elsevier: Amsterdam, The Netherlands, 2021; pp. 437-484. [DOI: https://dx.doi.org/10.1016/B978-0-12-817772-3.00014-8]

21. Alli, K.; Zhang, J. Adaptive Modelling of Fed-batch Processes with Extreme Learning Machine and Recursive Least Square Technique. Proceedings of the 12th International Conference on Agents and Artificial Intelligence; Valletta, Malta, 22–24 February 2020; SciTePress: Setúbal Municipality, Portugal, 2020; Volume 2, pp. 668-674. [DOI: https://dx.doi.org/10.5220/0008980506680674]

22. Huang, G.; Huang, G.-B.; Song, S.; You, K. Trends in extreme learning machines: A review. Neural Netw.; 2015; 61, pp. 32-48. [DOI: https://dx.doi.org/10.1016/j.neunet.2014.10.001]

23. Huang, G.-B.; Zhu, Q.-Y.; Siew, C. Extreme learning machine: A new learning scheme of feedforward neural networks. Proceedings of the IEEE International Conference on Neural Networks; Budapest, Hungary, 25–29 July 2004; pp. 985-990. [DOI: https://dx.doi.org/10.1109/IJCNN.2004.1380068]

24. Li, W.H.; Yue, H.H.; Valle-Cervantes, S.; Qin, S.J. Recursive PCA for adaptive process monitoring. J. Process Control.; 2000; 10, pp. 471-486. [DOI: https://dx.doi.org/10.1016/S0959-1524(00)00022-6]

25. Yu, W.K.; Zhao, C.H. Recursive exponential slow feature analysis for fine-scale adaptive processes monitoring with comprehensive operation status identification. IEEE Trans. Ind. Inform.; 2019; 15, pp. 3311-3323. [DOI: https://dx.doi.org/10.1109/TII.2018.2878405]

26. Yu, W.K.; Zhao, C.H.; Huang, B. Recursive cointegration analytics for adaptive monitoring of nonstationary industrial processes with both static and dynamic variations. J. Process Control.; 2020; 92, pp. 319-332. [DOI: https://dx.doi.org/10.1016/j.jprocont.2020.06.013]

27. Yüzgeç, U.; Türker, M.; Hocalar, A. On-line evolutionary optimization of an industrial fed-batch yeast fermentation process. ISA Trans.; 2008; 48, pp. 79-92. [DOI: https://dx.doi.org/10.1016/j.isatra.2008.09.001] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/18849027]

28. Zhang, J.; Feng, Y.; Al-Mahrouqi, M.H. Reliable optimal control of a fed-batch fermentation process using ant colony optimisation and bootstrap aggregated neural network models. Computer-Aided Chemical Engineering, Proceedings of the 21st European Symposium on Computer Aided Process Engineering; Chalkidiki, Greece, 29 May–1 June 2011; Pistikopoulos, E.N.; Georgiadis, M.C.; Kokossis, A.C. Elsevier: Amsterdam, The Netherlands, 2011; Volume 29, pp. 664-667.

29. Åström, K.J.; Wittenmark, B. Computer-Controlled Systems: Theory and Design; 3rd ed. Dover Publications: Mineola, NY, USA, 2011.

30. Baron, C.; Zhang, J. Reliable On-Line Re-Optimization Control of a Fed-Batch Fermentation Process Using Bootstrap Aggregated Extreme Learning Machine; Springer Nature: Cham, Switzerland, 2020; [DOI: https://dx.doi.org/10.5220/0006477601650176]