Dislocation Substructures Evolution and an Informer Constitutive Model for a Ti-55511 Alloy in Two-Stages High-Temperature Forming with Variant Strain Rates in β Region

Abstract

The high-temperature compression characteristics of a Ti-55511 alloy are explored through adopting two-stage high-temperature compressed experiments with step-like strain rates. The evolving features of dislocation substructures over hot, compressed parameters are revealed by transmission electron microscopy (TEM). The experiment results suggest that the dislocations annihilation through the rearrangement/interaction of dislocations is aggravated with the increase in forming temperature. Notwithstanding, the generation/interlacing of dislocations exhibit an enhanced trend with the increase in strain in the first stage of forming, or in strain rates at first/second stages of a high-temperature compressed process. According to the testing data, an Informer deep learning model is proposed for reconstructing the stress–strain behavior of the researched Ti-55511 alloy. The input series of the established Informer deep learning model are compression parameters (compressed temperature, strain, as well as strain rate), and the output series are true stresses. The optimal input batch size and sequence length are 64 and 2, respectively. Eventually, the predicted results of the proposed Informer deep learning model are more accordant with the tested true stresses compared to those of the previously established physical mechanism model, demonstrating that the Informer deep learning model enjoys an outstanding forecasted capability for precisely reconstructing the high-temperature compressed features of the Ti-55511 alloy.

1. Introduction

Due to its outstanding properties consisting of mechanical properties, anticorrosive performance, and thermal treatability, near-β titanium alloy is comprehensively applied in the crucial manufacture of load-bearing aircraft components [1,2]. Usually, hot deformation is necessarily utilized to improve the microstructures and further optimize the practical performance of titanium alloys [3,4,5]. The coupling effects of multiple forming parameters induce intricate evolving characteristics of microstructures and high-temperature flow behavior of titanium alloys [6,7,8,9,10]. Hence, investigations on the microstructural evolution and accurately modeling the true stress–strain characteristics of titanium alloys are significant.

To this day, numerous investigations have been devoted to exploring the microstructural evolution mechanisms of titanium alloys [11,12,13,14,15]. Some reports [16,17] revealed the substructural evolving features for multiple titanium alloys in thermal forming and detected that the substructural nucleated/migration mechanisms were substantially affected by processing parameters. Meanwhile, it was found that the evolution of substructures could exert a prominent effect on the nucleated/coarsening of dynamic recrystallization (DRX) [18,19,20,21]. Additionally, the transformation mechanisms of phases (i.e., α phase globularization [22,23], α phase conversion into β phase [24,25]) were intensively analyzed. As mentioned in previous investigations, intricate microstructural variation/interaction characteristics frequently emerge and notably affect the thermal forming features of titanium alloys.

Describing high-temperature flow characteristics of alloys is a current research subject and obtained tremendous achievements with various constitutive models [26,27,28,29,30,31,32,33]. First, multiple phenomenological models were constructed/improved for reproducing the thermal flow features of alloys [34,35,36,37,38]. Moreover, according to microstructural variations over processing parameters, multitudinous physical mechanism correlation models were constructed for reproducing the thermal flow characteristics of alloys [39,40,41,42]. Usually, the above two types of models can score decent prediction results, but it is challenging to formulate appropriate expressions and determine accurate material constants. Therefore, numerous machine learning models were established to simplify the conducting process and had overall superior forecasted results. For instance, the e-insensitive support vector regression (e-SVR) obtained decent results for forecasting flow characteristics [43,44]. Furthermore, complex artificial neural network (ANN) models were leveraged to predict the flow stress of titanium alloys, such as Ti600, Ti60, Ti40, and Ti-2Al-9.2Mo-2Fe β alloys [45,46,47]. Specifically, Ge et al. [48] leveraged the artificial neural network to propose the accurate constitutive model for the β-γ TiAl alloy. In recent years, various ANN-based deep learning models (DLMs) were developed to be applied in forecasting tasks, e.g., the recurrent neural network (RNN) [49,50] and long short-term memory (LSTM) [51,52,53]. However, the overfitting issue and long-term predicting performance degradation make them difficult to apply in practical usage. To tackle the problems, the Transformer-based Informer [54] deep learning model was proposed and showed an excellent capability in lithium-ion battery estimation [55]. Therefore, in this study, the two-stage high-temperature forming with variant strain rates in the β region of a Ti-55511 alloy is investigated. The Informer deep learning model was established for characterizing the microstructures and flow features of the Ti-55511 alloy.

Despite the comprehensive investigation of evolving characteristics of flow behaviors and microstructures for titanium alloys in thermal deformation at constant strain rates, systematic investigations of thermally compressed features of titanium alloys under variant strain rates remain lacking. Owing to the influences of sophisticated die structure as well as friction conditions between the die and component, the component commonly undergoes thermal forming with varying strain rates in the actual manufacturing process. Thereby, the stress–strain features for a Ti-55511 alloy in thermal compression with step-like strain rates were investigated. Furthermore, the evolving features of substructures are earnestly analyzed. Additionally, an Informer deep learning model is proposed for reconstructing the thermally compressed features of the Ti-55511 alloy.

2. Experimental Material and Procedure

The commercial near-β titanium alloy was employed in the present investigation. The chemical composition (wt.%) for the researched titanium alloy was 5.16Al-4.92Mo-4.96V-1.10Cr-0.98Fe-(bal.) Ti. Cylindrical specimens (Φ8 mm × 12 mm) for thermal compression were manufactured. The Gleeble-3500 device was employed for constructing the two-stage thermally compressed experiments. Figure 1 reveals the explicit experimental procedures. Distinctly, all forming processes contain two compressed stages (I as well as II). The compressed temperature ($T$) and the total strain (${\epsilon }_{\mathrm{total}}$) were consistent in two stages. Here, three compressed temperatures (890 °C, 920 °C as well as 950 °C) and the constant value of ${\epsilon }_{\mathrm{total}}$ (1.2) were adopted. Still, discrepant strain rates were exploited in each compressed stage. The representative complete compressed experimental step is that the specimen was thermally compressed under the strain rate of the first compression stage (${\dot{\epsilon }}_{\mathrm{I}}$) until the strain of stage I (${\epsilon }_{\mathrm{I}}$) was finished, and then thermal compression was executed under the strain rate of the second compression stage (${\dot{\epsilon }}_{\mathrm{II}}$). Correspondingly, three values of ${\epsilon }_{\mathrm{I}}$ (0.3, 0.6, as well as 0.9) were adopted.

Before thermal compression, each sample was heated to the compressed temperatures under 10 °C/s and remained at 300 s. When the thermal compressed process was finished, the compressed blocks were directly cooled utilizing water (about 25 °C). To dissect the evolving features of substructures in thermal compression, transmission electron microscopy (TEM) was adopted. To dissect the original microstructure, electron backscatter microscopy (EBSD) was chosen. For analyzing using TEM as well as EBSD, the thermally compressed samples were axially machined for acquiring cross-sections. Afterwards, these sections were ground, polished, and etched in a solution (10 mL HClO₄ + 70 mL C₄H₁₀O + 120 mL CH₃OH). Figure 2 displays the original grain structures, and most of the initial $\mathsf{\beta }$ grains are equiaxed grains.

3. High-Temperature Compression Features and Substructural Evolution

The prime hot flow features of the researched titanium alloy in double-stage hot compression with stepped-strain rates are displayed in Figure 3. Clearly, the high-temperature compression behaviors are markedly affected by compression parameters. As revealed in Figure 3a, the true stresses at the first and second stages of hot compression exhibit a diminishing tendency with rising compression temperature. One principal reason for this experimental result is that the DRX behavior dramatically proceeds as the compressed temperature (T) ascends [6]. Moreover, the visible evolution of substructures occurs with the elevated compression temperature, as depicted in Figure 4a,b. For the compressed temperature of 920 °C and strain rate of 0.01 s⁻¹, the formation of high-density dislocation clusters can be detected (Figure 4a). Then, the prominent work-hardening (WH) effect is inspired owing to the acute interaction of adjacent substructures, and the rise in true stress occurs quickly [6,16]. When the compressed temperature is elevated from 920 °C to 950 °C, the intensive migration/interaction of dislocations and grain boundary occurs, and the substructures are apparently consumed (Figure 4b). Then, the reinforced dynamic softening feature emerges with a rising incompression temperature, and a decrease in true stress appears. Furthermore, the true stress at the second stage of high-temperature compression exhibits a relative increasing trend along with the rise in the strain of the first-stage compression (${\epsilon }_{\mathrm{I}}$), as displayed in Figure 3b. This tested result is primarily ascribed to the weakened DRX development occurring at large values of ${\epsilon }_{\mathrm{I}}$, as the strain rate is transferred from a high value (${\dot{\epsilon }}_{\mathrm{I}}$ = 0.1 s⁻¹) to a low value (${\dot{\epsilon }}_{\mathrm{II}}$ = 0.001 s⁻¹) [6]. Meanwhile, the variations of ${\epsilon }_{\mathrm{I}}$ exerting a significant influence on the substructural evolution are depicted in Figure 4c. From Figure 4a,c, it can be detected that the generation/accumulation of substructures (subgrain, dislocation network, etc.) is promoted with increasing ${\epsilon }_{\mathrm{I}}$. Owing to the formation of high-density dislocation networks, the resistance of dislocation slippage, and grain boundary motion is raised, inducing the rise in true stress at the second stage of high-temperature compression.

4. The Informer Deep Learning Model for Forecasting Hot Flow Features of a Ti-55511 Alloy

In contrast to existing models with lengthy process limitations, the Transformer model demonstrates the operational potential for long sequence prediction, owing to its innovative architecture and self-attention mechanism [56]. Although the canonical self-attention mechanism is capable of processing large-scale data with impressive performance, the high computational complexity and significant memory consumption in stacking layers of the model impede its practical application. To address such a deficiency, optimized models such as the LogSparse Transformer model [57] and similar models [58] were proposed to reduce the original self-attention mechanism complexity, but their efficiency remained limited. Moreover, the Reformer model was embedded with locally sensitive hashing updated self-attention to reduce the complexity in the exceptionally long-term series for each layer [59]. In certain situations, the complexity growth rate of the Informer model was optimized to be linear, but the model could potentially experience degradation in practical long-term prediction [60]. More recently, a continuous-space attention mechanism was deployed in the Infinite Memory Transformer model to free the complexity from input length, but the prediction accuracy was decreased [61].

In summary, previous Transformer models focused on optimizing the complexity of the attention mechanism for each layer and obtained important findings. However, simultaneously cutting down the complexity and breaking the scalability bottleneck of stacking layers is rarely addressed. Therefore, the Informer deep learning model is proposed to address these limitations and accelerate its computing speed [54]. In the present research, the Informer deep learning model is applied as a practical method for forecasting the flow characteristics of the studied titanium alloy. Specifically, the Informer deep learning model leverages the proposed ProbSparse self-attention mechanism and distilling operation to reduce the memory usage and time complexity of the dependency alignment to $\mathcal{O}\left(L\log L\right)$ and the space complexity to $\mathcal{O}((2-\epsilon )L\log L)$. During the inference phase, the model utilizes a generative decoder form to avoid cumulative error spreading and optimize long-series output. The Informer deep learning model architecture is shown in Figure 5.

4.1. ProbSparse Self-Attention Mechanism

With inputs as query, key and value, the original self-attention mechanism is defined as [56],

(1)$\mathcal{A}\left(Q,K,V\right)=\mathrm{Softmax}\left(Q{K}^{\top }/\sqrt{d}\right)V$

where $Q\in {ℝ}^{L_Q\times d}, K\in {ℝ}^{L_K\times d}, V\in {ℝ}^{L_V\times d}$, and $d$ denotes the input dimension.

Derived by [62], the $i$-th query’s attention can be defined with kernel smoothing as,

(2)$\mathcal{A}\left(q_i,K,V\right)={\displaystyle \sum }_j\frac{k\left(q_i,k_j\right)}{{{\displaystyle \sum }}_{l}k\left(q_i,k_l\right)}{v}_j={\mathbb{E}}_{p\left(k_j\left|q_i\right.\right)}\left[v_j\right]$

where $q_i,k_i,v_i$ stand for the $i$-th row in $Q,K,V$, respectively, and $k\left(q_i,k_j\right)=\exp \left(q_i{k}_j^{\top }/\sqrt{d}\right)$. The part $p\left(k_j\left|q_i\right.\right)=k\left(q_i,k_j\right)/{\sum }_{l}k\left(q_i,k_l\right)$ is conducted to obtain the probability, which entails a large $\mathcal{O}\left(L_Q{L}_K\right)$ memory usage. Therefore, the Informer deep learning model proposed the query sparsity measurement to tackle this major defect of self-attention.

The similarity between $p$ and $q$ can be used to distinguish the importance, which can be conducted through Kullback–Leibler divergence as,

(3)$KL\left(q‖p\right)=\ln {\sum }_{l=1}^{L_K}{e}^{q_i{k}_l^{\top }/\sqrt{d}}-\frac{1}{L_K}{\sum }_{j=1}^{L_K}{q}_i{k}_j^{\top }/\sqrt{d}-\ln L_K$

The measurement of the $i$-th query is defined by dropping the constant as,

(4)$M\left(q_i,K\right)=\ln {\displaystyle \sum }_{j=1}^{L_K}{e}^{\frac{q_i{k}_j^{\top }}{\sqrt{d}}}-\frac{1}{L_K}{\displaystyle \sum }_{j=1}^{L_K}\frac{q_i{k}_j^{\top }}{\sqrt{d}}$

where the formula calculates the Log-Sum-Exp (LSE) and the arithmetic mean of all keys [63]. If $M\left(q_i,K\right)$ grows larger, the probability $p$ becomes more principal factor alterable, thus having a superior differentiating capability.

According to the above measurement, the ProbSparse self-attention mechanism can be further conducted by distributing keys to Top-u queries as

(5)$\mathcal{A}\left(Q,K,V\right)=\mathrm{Softmax}\left(\frac{\overline{Q}{K}^{\top }}{\sqrt{d}}\right)V$

where $\overline{Q}$ is the $q$-size sparse matrix. When $u=c\cdot \ln L_Q$, the layer memory usage is reduced to $\mathcal{O}\left(L_K\ln L_Q\right)$ due to the lessened calculation for each key.

Nevertheless, the query sparsity measurement needs quadratic $\mathcal{O}\left(L_Q{L}_K\right)$ calculation, and the LSE implement is not constantly numerically stable. Hence, an empirical approximation is conducted.

For each $q_i$, the discrete keys can be converted to continuous ones as vector $k_j$. In addition, the first term of the $M\left(q_i,K\right)$ becomes the LSE of the inner product of a fixed query $q_i$ and all the keys, and define

(6)$f_i\left(K\right)=\ln {\sum }_{j=1}^{L_K}{e}^{q_i{k}_j^{\top }/\sqrt{d}}$

From the Log-Sum-Exp network and relative studies [63,64], the convex function $f_i\left(K\right)$ combines linear $k_j$ for $q_i$, making $M\left(q_i,K\right)$ convex. Hence, the measurement can be conducted to a derivation form with each vector $k_j$ as follows,

(7)$\frac{\partial M\left(q_i,K\right)}{\partial k_j}=\frac{e^{q_i{k}_j^{\top }/\sqrt{d}}}{{\sum }_{j=1}^{L_K}{e}^{q_i{k}_j^{\top }/\sqrt{d}}}\cdot \frac{q_i}{\sqrt{d}}-\frac{1}{L_K}\cdot \frac{q_i}{\sqrt{d}}$

Let $\overrightarrow{\nabla }M\left(q_i\right)=\overrightarrow{0}$ to reach the minimum value; the condition can be listed as,

(8)$q_i{k}_1^{\top }+\ln L_K=\cdots =q_i{k}_j^{\top }+\ln L_K=\cdots =\ln {\sum }_{j=1}^{L_K}{e}^{q_i{k}_j^{\top }}$

The minimum value $\ln L_K$ can be obtained when $k_1=k_2=\cdots =k_{L_K}$. Therefore, the measurement can be written as

(9)$M\left(q_i,K\right)\ge \ln L_K$

Hence, by picking the largest inner-product ${\max }_j\left\{q_i{k}_j^{\top }/\sqrt{d}\right\}$, the inequation can be derived as

(10)$\left\{\begin{array}{c}\mathrm{M}(q_i,K)=\ln {\displaystyle {\displaystyle \sum }_{j=1}^{L_K}}{e}^{\frac{q_i{k}_j^{\top }}{\sqrt{d}}}-\frac{1}{L_K}{\displaystyle {\displaystyle \sum }_{j=1}^{L_K}}\left(\frac{q_i{k}_j^{\top }}{\sqrt{d}}\right)\\ \hspace{1em}\hspace{1em}\hspace{1em}\le \ln \left(L_K\cdot \underset{j}{\max }\left\{\frac{q_i{k}_j^{\top }}{\sqrt{d}}\right\}\right)-\frac{1}{L_K}{\displaystyle {\displaystyle \sum }_{j=1}^{L_K}}\left(\frac{q_i{k}_j^{\top }}{\sqrt{d}}\right)\hfill \\ \hspace{1em}\hspace{1em}\hspace{1em}=\ln L_K+\underset{j}{\max }\left\{\frac{q_i{k}_j^{\top }}{\sqrt{d}}\right\}-\frac{1}{L_K}{\displaystyle {\displaystyle \sum }_{j=1}^{L_K}}\left(\frac{q_i{k}_j^{\top }}{\sqrt{d}}\right)\hfill \end{array}\right.$

Eventually, by combining the above equations, the bound can be denoted as

(11)$\ln L_K\le M\left(q_i,K\right)\le {\max }_j\left\{q_i{k}_j^{\top }/\sqrt{d}\right\}-\frac{1}{L_K}{\sum }_{j=1}^{L_K}\left\{q_i{k}_j^{\top }/\sqrt{d}\right\}+\ln L_K$

where $q_i\in {ℝ}^d$ and $k_j\in {ℝ}^d$ are in the keys set $K$.

From the above deductions, the max–mean measurement can be defined as

(12)$\overline{M}\left(q_i,K\right)=\underset{j}{\max }\left\{\frac{q_i{k}_j^{\top }}{\sqrt{d}}\right\}-\frac{1}{L_K}{\displaystyle \sum }_{j=1}^{L_K}\frac{q_i{k}_j^{\top }}{\sqrt{d}}$

Specifically, a long-tail distribution pattern of the self-attention mechanism was observed by performing a qualitative assessment [54]. In this case, only a few dot product pairs contribute to the major attention. Hence, $\overline{M}\left(q_i,K\right)$ only requires $U=L_K\ln L_Q$ dot product pairs of random sampling, and the remaining pairs are filled with zero values. Therefore, the operation has a weaker sensitivity and remains numerically stable. Eventually, in practical application, the relatively equivalent input length $L_Q=L_K=L$ in self-attention computation can reduce the complexity to $\mathcal{O}\left(L\ln L\right)$.

4.2. Encoder

The Informer deep learning model utilizes the encoder architecture to extract the long-term dependency of input series, where the t-th input $X^t$ is reshaped as matrix $X_{\mathrm{en}}^{\mathrm{t}}\in {ℝ}^{LX\times d\mathrm{model}}$ [56]. The encoder is composed of multiple identical layers stacked on top of each other. Specifically, the architecture of a single stack in the encoder of the Informer deep learning model is given in Figure 6.

Due to the processing of the ProbSparse self-attention mechanism, the encoder is loaded with redundant value $V$ combinations. Hence, self-attention distilling is proposed to concentrate self-attention mechanisms for the next layer.

Based on the dilated convolution [65], the distilling operation feeds forwards the $\left(j+1\right)$-th layer as,

(13)$X_{j+1}^t=\mathrm{MaxPool}\left(\mathrm{ELU}\left(\mathrm{Conv}1\mathrm{d}\left({\left[X_j^t\right]}_{\mathrm{AB}}\right)\right)\right)$

where ${[\cdot ]}_{\mathrm{AB}}$ denotes the attention block, and $\mathrm{Conv}1\mathrm{d}\left(\cdot \right)$ generates a 1D convolutional filter with $\mathrm{ELU}\left(\cdot \right)$ activation function [66].

The max-pooling layer is added to reduce the total memory usage to $\mathcal{O}\left(\left(2-\epsilon \right)L\log L\right)$. Furthermore, a pyramid-like processing structure (shown in Figure 6) is established where inputs are halved to serve as the replication of the main stack and the distilling layers drop gradually. In this case, the operation has a better robustness, and the resulting dimensions of different layers are consistent.

4.3. Decoder

The canonical decoder structure is optimized with generative inference to mitigate the long-term speed descent. The decoder mechanism is defined as

(14)$X_{\mathrm{de}}^t=\mathrm{Concat}\left(X_{\mathrm{token}}^t,X_0^t\right)\in {ℝ}^{\left(L_{\mathrm{token}}+L_y\right)\times d_{\mathrm{model}}}$

where $X_{\mathrm{token}}^t\in {ℝ}^{L_{\mathrm{token}}\times d_{\mathrm{model}}}$ is the start token, and $X_0^t\in {ℝ}^{L_y\times d_{\mathrm{model}}}$ is the placeholder for target sequences.

Extended from dynamic decoding [67], the procedure is innovated to sample a $L_{\mathrm{token}}$ series in the input sequence as a start token then feed it to the decoder as $X_{\mathrm{de}}=\left\{X_{\mathrm{L}},X_0\right\}$. Afterwards, the decoder obtains outputs through a single forward procedure, and thus it can process with less time consumption than a normal encoder-decoder architecture.

4.4. Identification for the Parameters of the Informer Deep Learning Model

The inputs of the Informer deep learning model are temperature T = {890, 920, 950} °C, true strain $\epsilon$ = {0~1}, and strain rate $\dot{\epsilon }$ = {0.001, 0.01, 0.1, 1} s⁻¹. The input sequences were preprocessed by concatenating experimental data of true stress values under different temperatures, true strains, and strain rates. The corresponding temperature, true strain, and strain rate values were also concatenated in the sequences. Then, these sequences were applied as training inputs. The experimental data are shuffled using 7/10 of the total amount for training and the rest for testing and validating the model.

As discussed above regarding the architecture of the Informer deep learning model in Section 4.1, Section 4.2 and Section 4.3 and the features of general deep neural networks, the Informer deep learning model should first be established by tuning hyper-parameters such as learning rate, input batch size, dropout, etc. To obtain the optimal parameters, the correction coefficient $R$, average absolute relative error $AARE$, mean squared error $MSE$, and root-mean squared error $RMSE$ assessment criteria are employed for evaluating the results.

(15)$R=\frac{{\sum }_{i=1}^N\left(M_i-\stackrel{•}{M}\right)\left(P_i-\stackrel{•}{P}\right)}{\sqrt{{\sum }_{i=1}^N{\left(M_i-\stackrel{•}{M}\right)}^2{\sum }_{i=1}^N{\left(P_i-\stackrel{•}{P}\right)}^2}}$

(16)$AARE\left(\%\right)=\frac{1}{N}{\sum }_{i=1}^N\left|\frac{M_i-P_i}{M_i}\right|\times 100\%$

(17)$MSE={\sum }_{i=1}^N\frac{{\left(M_i-P_i\right)}^2}{N}$

(18)$RMSE=\sqrt{{\sum }_{i=1}^N\frac{{\left(M_i-P_i\right)}^2}{N}}$

where $N$ notes the total amount of result data, and $M_i$ and $P_i$ stand for the measured and predicted results when $\stackrel{•}{M}$ and $\stackrel{•}{P}$ are the mean values, respectively.

Generally, the accuracy and generalization ability of deep learning models are affected by various hyper-parameters. In the case of forecasting, the batch size of input sequences and the initial learning rate of the model play crucial roles. On the one hand, a larger batch size allows faster training but may result in worse model accuracy and an unstable training process [68]. On the other hand, a smaller batch size is beneficial for generalization but can lead to a longer computation time [69]. Additionally, both theoretical and empirical evidence have proven that the batch size and learning rate significantly impact the generalization ability and accuracy of the deep learning model [70,71,72]. To further explore the relationship between the two parameters and the results, experimental curves are displayed in Figure 7. The five curves represent the effect of the learning rate on validation loss under different batch sizes. Specifically, the learning rate is tested in a uniformly spaced range from 10⁻¹ to 10⁻⁶ with batch sizes of 8, 16, 32, 64, and 128, respectively. The model accuracy is evaluated by the validation loss.

It is clear that the validation loss of the Informer deep learning model drops to a minimum value and then starts to fluctuate when the learning rate increases from 10⁻⁴ to 2 × 10⁻³. As the learning rate further ascends, the fluctuation of validation loss becomes intense, and thus the optimal learning rate can be chosen as 1.2929 × 10⁻³. Specifically, the curve fluctuations in Figure 7 demonstrate an appropriate balance of model accuracy and training stability under the batch size of 64. Hence, the batch size is determined as 64.

In addition, it is important to note that the parameters of sequence length and label length also have a significant impact on accuracy. Based on experimental results, the optimal sequence length and label length are identified as two and one, respectively.

Eventually, the values of R, AARE, and RMRE can be computed as 0.9986, 4.191%, and 2.2016, respectively. According to the results, the performance of the Informer deep learning model is shown in Figure 8. It illustrates good consistency between the experimental data and the modeled results, demonstrating the great capability of the Informer deep learning model to describe the high-temperature deformation features of the researched titanium alloy.

4.5. Comparisons and Discussion

As shown in the above sections, the Informer deep learning model exhibits a strong forecasting ability for the true stress of the researched titanium alloy. According to the author’s previous investigation [6], a physical mechanism (PM) model was constructed for forecasting the true stress of the researched titanium alloy, i.e.,

(19)$\left\{\begin{array}{l}\sigma ={\sigma }_{\mathrm{y}}+{\sigma }_{\mathsf{\rho }}\\ {\sigma }_{\mathrm{y}}=1.589{(\dot{\epsilon }\exp (\frac{\mathrm{205,800}}{\mathrm{R}T}))}^{0.2052}\\ {\sigma }_{\mathsf{\rho }}=4.38\times {10}^{-10}\left(21.8847-0.0153T\right)\sqrt{{\rho }_i}\\ {\dot{\rho }}_i={\dot{\rho }}_i{}^{+}-{\dot{\rho }}_i{}^{\mathrm{DRV}}-{\dot{\rho }}_i{}^{\mathrm{DRX}}\\ {\dot{\rho }}_i{}^{+}=\frac{1}{2.86\times {10}^{-10}\mathrm{\Lambda }}\dot{\epsilon }\\ \frac{1}{\mathrm{\Lambda }}=\frac{1}{s}+\frac{1}{d_i}\\ s=\frac{F_{\mathrm{s}}}{\sqrt{{\rho }_i}}\\ F_{\mathrm{s}}=\begin{array}{c}4.1797\end{array}{(\dot{\epsilon }\exp (\frac{-6.4278}{\mathrm{R}T}))}^{-0.2843}\\ {\dot{d}}_{\mathrm{g}}=2.3866d^{\begin{array}{c}0.4468\end{array}}\\ {\dot{d}}_{\mathrm{x}}=-0.7803d^{0.0072}{\dot{X}}^{0.9906}\\ {\dot{d}}_{\mathrm{i}}={\dot{d}}_{\mathrm{x}}+{\dot{d}}_{\mathrm{g}}\\ {\dot{\rho }}_i{}^{\mathrm{DRV}}=47.0321{(\dot{\epsilon }\exp (\frac{0.04718}{\mathrm{R}T}))}^{\begin{array}{c}-0.1259\end{array}}\rho \\ {\dot{\rho }}_i{}^{\mathrm{DRX}}=\frac{0.7313{(\dot{\epsilon }\exp (\frac{-9.1687}{\mathrm{R}T}))}^{0.0137}\dot{X}{\rho }_i}{{(1-X)}^{2.0210}}\\ \dot{X}=\frac{6.0877M_{\mathrm{b}}P{[X(1-X)]}^{-1.4294}{\dot{\epsilon }}^{-2.6513}}{d^{0.5733}}\\ M_{\mathrm{b}}=\frac{1.54\times {10}^{-26}}{\mathrm{k}T}{[\dot{\epsilon }\exp (\frac{-0.1418}{\mathrm{R}T})]}^{0.0045}\\ \mathrm{P}=\frac{8.18\times {10}^{-20}{\rho }_{\mathrm{i}}\left(21.8847-0.0153T\right)}{2}\end{array}\right.$

where $\sigma$ is the flow stress, ${\sigma }_y$ is the short-range component, and ${\sigma }_{\mathsf{\rho }}$ is the dislocation interaction stress. $\dot{\epsilon }$ is the strain rate, R is the gas constant, T is the absolute temperature, ${\rho }_i$ is the dislocation density, ${\dot{\rho }}_i{}^{+}$ is the dislocation density emergence rate under WH, and ${\dot{\rho }}_i{}^{\mathrm{DRV}}$ and ${\dot{\rho }}_i{}^{\mathrm{DRX}}$ are dislocation density variation rate of DRV and DRX, respectively. $\mathrm{\Lambda }$ is the mean-free path of dislocation, and $d_i$ is the average grain size. $X$ is the DRX fraction and the rate $\dot{X}$. $M_{\mathrm{b}}$ is the grain boundary movement rate. $\mathrm{P}$ is the driving force. $D_{\mathrm{ob}}$ is the factor of self-diffusion, and $\delta$ is the grain boundary thickness.

Figure 9 unveils the comparative analysis of forecasting performances between the PM and Informer deep learning model. Compared to that of the PM model, the Informer deep learning model enjoys a smaller forecasting error of true stresses, particularly for the researched titanium alloy at lower compressed temperature (890 °C) or higher ${\dot{\epsilon }}_{\mathrm{I}}$/${\dot{\epsilon }}_{\mathrm{II}}$. To validate the forecasting capability, the correlation results of forecasted true stresses and tested ones are plotted in Figure 10. Clearly, the scatters of the PM constitutive model are more dispersed, while those of the Informer deep learning model are more centralized. Meanwhile, the values of R, AARE, and RMSE are determined, as noted in Table 1. Distinctly, the relative larger R as well as the smaller AARE and RMSE values imply that the established Informer deep learning model can accurately depict the hot compressed features of the Ti-55511 alloy.

5. Conclusions

The evolving characteristics of microstructures as well as flow behavior for a Ti-55511 alloy in two-stage thermal compression experiments with step-like strain rates are researched. The decisive conclusions are drawn as:

• (1). In high-temperature compression, the influences of forming parameters on the flow behaviors of the researched Ti-55511 alloy are significant. Flow stresses are reduced with the increase in compressed temperature. Notwithstanding, flow stresses at stage II of thermal compression display an increase trend with the descent of ${\epsilon }_{\mathrm{I}}$ or increase in ${\dot{\epsilon }}_{\mathrm{I}}$/${\dot{\epsilon }}_{\mathrm{II}}$;

• (2). The formation of high-density networks/clusters through dislocation concentration/interaction is suppressed with the increase in compressed temperature. Nevertheless, the dislocation nucleation/concentration is enhanced with the increase in ${\epsilon }_{\mathrm{I}}$;

• (3). The Informer deep learning model is developed to reconstruct the thermal compressed characteristics of the researched Ti-55511 alloy. The considerable agreement between the predicted true stresses and experimental results demonstrates the high prediction accuracy of the Informer deep learning model.

Author Contributions

Conceptualization, D.H.; Methodology, S.T., D.H. and B.Z.; Software, H.W.; Validation, Y.L.; Investigation, S.T. and B.Z.; Data curation, S.T. and H.W.; Writing—original draft, S.T.; Writing—review & editing, D.H. and Y.L.; Supervision, Y.L.; Funding acquisition, D.H. and Y.L. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The raw/processed data required to reproduce these findings cannot be shared at this time as the data also forms part of an ongoing study.

Conflicts of Interest

No conflict of interest exist in the submission of this manuscript, and the manuscript is approved by all authors for publication. I would like to declare on behalf of my co-authors that the work described was original research that has not been published previously, and it is not under consideration for publication elsewhere, in whole or in part. All authors listed have approved the manuscript that is enclosed.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures

Figure 1. Tested steps of the received titanium alloy: (a) type A: the strain rates altered from the relative low values ([Forumla omitted. See PDF.]) to high values ([Forumla omitted. See PDF.]); (b) type B: the strain rates altered from the relative high values ([Forumla omitted. See PDF.]) to low values ([Forumla omitted. See PDF.]).

Figure 2. EBSD map of original grain structures in the received titanium alloy.

Figure 3. Representative flow characteristic at variation of: (a) [Forumla omitted. See PDF.], (b) [Forumla omitted. See PDF.] [6].

Figure 4. TEM figures at: (a) [Forumla omitted. See PDF.] = 920 °C/[Forumla omitted. See PDF.] = 0.1 s−1/[Forumla omitted. See PDF.] = 0.36/[Forumla omitted. See PDF.] = 0.001 s−1, (b) [Forumla omitted. See PDF.] = 950 °C/[Forumla omitted. See PDF.] = 0.1 s−1/[Forumla omitted. See PDF.] = 0.36/[Forumla omitted. See PDF.] = 0.001 s−1, (c) [Forumla omitted. See PDF.] = 920 °C/[Forumla omitted. See PDF.] = 0.1 s−1/[Forumla omitted. See PDF.] = 0.6/[Forumla omitted. See PDF.] = 0.001 s−1.

Figure 4. TEM figures at: (a) [Forumla omitted. See PDF.] = 920 °C/[Forumla omitted. See PDF.] = 0.1 s−1/[Forumla omitted. See PDF.] = 0.36/[Forumla omitted. See PDF.] = 0.001 s−1, (b) [Forumla omitted. See PDF.] = 950 °C/[Forumla omitted. See PDF.] = 0.1 s−1/[Forumla omitted. See PDF.] = 0.36/[Forumla omitted. See PDF.] = 0.001 s−1, (c) [Forumla omitted. See PDF.] = 920 °C/[Forumla omitted. See PDF.] = 0.1 s−1/[Forumla omitted. See PDF.] = 0.6/[Forumla omitted. See PDF.] = 0.001 s−1.

Figure 5. Architecture of the Informer deep learning model.

Figure 6. The architecture of a single stack in the encoder of the Informer deep learning model.

Figure 7. Variations of validation loss under different tested batch sizes and learning rates.

Figure 8. Performance of the Informer deep learning model.

Figure 9. Comparisons of tested true stress and predicted results at: (a) T, (b) [Forumla omitted. See PDF.], (c) [Forumla omitted. See PDF.], (d) [Forumla omitted. See PDF.].

Figure 10. Correlation of tested true stresses and predicted values.

Table 1

Calculated assessment values of the Informer deep learning model and PM constitutive model.

Model	R	AARE(%)	RMSE
PM model [6]	0.9945	6.181%	3.7448
Informer deep learning model	0.9986	4.191%	2.0615

References

1. Lin, Y.C.; Pang, G.-D.; Jiang, Y.-Q.; Liu, X.-G.; Zhang, X.-Y.; Chen, C.; Zhou, K.-C. Hot compressive deformation behavior and microstructure evolution of a Ti-55511 alloy with basket-weave microstructures. Vacuum; 2019; 169, 108878. [DOI: https://dx.doi.org/10.1016/j.vacuum.2019.108878]

2. Tan, K.; Li, J.; Guan, Z.; Yang, J.; Shu, J. The identification of dynamic recrystallization and constitutive modeling during hot deformation of Ti55511 titanium alloy. Mater. Des.; 2015; 84, pp. 204-211. [DOI: https://dx.doi.org/10.1016/j.matdes.2015.06.093]

3. Bobbili, R.; Madhu, V. Constitutive modeling of dynamic flow behavior of Ti-5553 alloy. J. Alloys Compd.; 2019; 787, pp. 260-266. [DOI: https://dx.doi.org/10.1016/j.jallcom.2019.02.101]

4. Li, C.W.; Xie, H.; Mao, X.N.; Zhang, P.S.; Hou, Z.M. High Temperature Deformation of TC18 Titanium Alloy. Rare Metal. Mat. Eng.; 2017; 46, pp. 326-332.

5. Zhang, J.; Wang, Y. Tension Behavior of Ti–6.6 Al–3.3 Mo–1.8 Zr–0.29 Si Alloy over a Wide Range of Strain Rates. Mater. Lett.; 2014; 124, pp. 113-116. [DOI: https://dx.doi.org/10.1016/j.matlet.2014.03.042]

6. He, D.-G.; Su, G.; Lin, Y.-C.; Jiang, Y.-Q.; Li, Z.; Chen, Z.-J.; Yan, X.-T.; Xia, Y.-C.; Xie, Y.-C. Microstructural Variation and a Physical Mechanism Model for a Ti-55511 Alloy during Double-Stage Hot Deformation with Stepped Strain Rates in the β Region. Materials; 2021; 14, 6371. [DOI: https://dx.doi.org/10.3390/ma14216371] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34771895]

7. Quan, G.; Pu, S.; Wen, H.; Zou, Z.; Zhou, J. Quantitative Analysis of Dynamic Softening Behaviors Induced by Dynamic Recrystallization for Ti-10V-2Fe-2Al Alloy. High Temp. Mater. Process.; 2015; 34, pp. 549-561. [DOI: https://dx.doi.org/10.1515/htmp-2014-0106]

8. Liang, H.; Guo, H. The integrated influence on hot deformation of dual-phase titanium alloys incorporating dynamic recrystallization evolution and α/β phase transformation. Mater. Lett.; 2015; 151, pp. 57-60. [DOI: https://dx.doi.org/10.1016/j.matlet.2015.03.052]

9. Li, C.; Zhang, X.-Y.; Li, Z.-Y.; Zhou, K.-C. Hot Deformation of Ti-5Al-5Mo-5V-1Cr-1Fe Near β Titanium Alloys Containing Thin and Thick Lamellar α Phase. Mater. Sci. Eng. A; 2013; 573, pp. 75-83. [DOI: https://dx.doi.org/10.1016/j.msea.2013.02.033]

10. Kar, S.K.; Ghosh, A.; Fulzele, N.; Bhattacharjee, A. Quantitative microstructural characterization of a near beta Ti alloy, Ti-5553 under different processing conditions. Mater. Charact.; 2013; 81, pp. 37-48. [DOI: https://dx.doi.org/10.1016/j.matchar.2013.03.016]

11. Ning, Y.Q.; Xie, B.C.; Liang, H.Q.; Li, H.; Yang, X.M.; Guo, H.Z. Dynamic Softening Behavior of TC18 Titanium Alloy during Hot Deformation. Mater. Des.; 2015; 71, pp. 68-77. [DOI: https://dx.doi.org/10.1016/j.matdes.2015.01.009]

12. Lin, Y.C.; Zhao, C.-Y.; Chen, M.-S.; Chen, D.-D. A novel constitutive model for hot deformation behaviors of Ti–6Al–4V alloy based on probabilistic method. Appl. Phys. A; 2016; 122, 716. [DOI: https://dx.doi.org/10.1007/s00339-016-0248-8]

13. Kotkunde, N.; Krishna, G.; Shenoy, S.K.; Gupta, A.K.; Singh, S.K. Experimental and theoretical investigation of forming limit diagram for Ti-6Al-4 V alloy at warm condition. Int. J. Mater. Form.; 2017; 10, pp. 255-266. [DOI: https://dx.doi.org/10.1007/s12289-015-1274-3]

14. Yang, Z.; Xu, W.; Zhang, W.; Chen, Y.; Shan, D. Effect of power spinning and heat treatment on microstructure evolution and mechanical properties of duplex low-cost titanium alloy. J. Mater. Sci. Technol.; 2023; 136, pp. 121-139. [DOI: https://dx.doi.org/10.1016/j.jmst.2022.07.022]

15. Lin, Y.C.; Jiang, X.-Y.; Shuai, C.-J.; Zhao, C.-Y.; He, D.-G.; Chen, M.-S.; Chen, C. Effects of initial microstructures on hot tensile deformation behaviors and fracture characteristics of Ti-6Al-4V alloy. Mater. Sci. Eng. A; 2018; 711, pp. 293-302. [DOI: https://dx.doi.org/10.1016/j.msea.2017.11.044]

16. Wu, C.; Zhou, Y.J.; Liu, B. Experimental and simulated investigation of the deformation behavior and microstructural evolution of Ti6554 titanium alloy during an electropulsing-assisted microtension process. Mater. Sci. Eng. A; 2022; 838, 142745. [DOI: https://dx.doi.org/10.1016/j.msea.2022.142745]

17. Li, L.; Liu, J.; Ding, N.; Li, M. Substructure evolution in two phases based constitutive model for hot deformation of TC18 in α + β phase region. Chin. J. Aeronaut.; 2023; 36, pp. 573-588. [DOI: https://dx.doi.org/10.1016/j.cja.2023.02.007]

18. Li, C.; Huang, L.; Zhao, M.; Guo, S.; Su, Y.; Li, J. Characterization of hot workability of Ti-6Cr-5Mo-5V-4Al alloy based on hot processing map and microstructure evolution. J. Alloys Compd.; 2022; 905, 164161. [DOI: https://dx.doi.org/10.1016/j.jallcom.2022.164161]

19. Lu, T.; Dan, Z.-H.; Li, K.; Yi, D.-Q.; Zhou, L.; Chang, H. Hot deformation behaviors and dynamic recrystallization mechanism of Ti-35421 alloy in β single field. Trans. Nonferrous Met. Soc. China; 2022; 32, pp. 2889-2907. [DOI: https://dx.doi.org/10.1016/S1003-6326(22)65991-0]

20. Huang, L.; Li, C.-M.; Li, C.-L.; Hui, S.-X.; Yu, Y.; Zhao, M.-J.; Guo, S.-Q.; Li, J.-J. Research progress on microstructure evolution and hot processing maps of high strength β titanium alloys during hot deformation. Trans. Nonferrous Met. Soc. China; 2022; 32, pp. 3835-3859. [DOI: https://dx.doi.org/10.1016/S1003-6326(22)66062-X]

21. Abbasi, S.; Momeni, A.; Lin, Y.C.; Jafarian, H. Dynamic softening mechanism in Ti-13V-11Cr-3Al beta Ti alloy during hot compressive deformation. Mater. Sci. Eng. A; 2016; 665, pp. 154-160. [DOI: https://dx.doi.org/10.1016/j.msea.2016.04.040]

22. Kumar, V.A.; Murty, S.; Gupta, R.; Rao, A.G.; Prasad, M. Effect of boron on microstructure evolution and hot tensile deformation behavior of Ti-5Al-5V-5Mo-1Cr-1Fe alloy. J. Alloys Compd.; 2020; 831, 154672. [DOI: https://dx.doi.org/10.1016/j.jallcom.2020.154672]

23. Liu, H.; Wang, Q.; Zhang, J.; Xu, K.; Xue, Y. Effect of multi-pass deformation on hot flow behavior and microstructure evolution mechanism of Ti–6Al–4V alloy fabricated by hot isostatic pressing. J. Mater. Res. Technol.; 2022; 17, pp. 2229-2248. [DOI: https://dx.doi.org/10.1016/j.jmrt.2022.01.136]

24. Yu, Y.; Yan, H.; Chen, J.; Xia, W.; Su, B.; Ding, T.; Li, Z.; Song, M. Flow behavior and dynamic transformation of bimodal TC17 titanium alloy during high strain rate hot compression. J. Alloys Compd.; 2022; 912, 165260. [DOI: https://dx.doi.org/10.1016/j.jallcom.2022.165260]

25. Chen, X.; Tang, B.; Wei, B.; Zhang, X.; Li, J. Investigation on recrystallization behavior of Ti-47Al-1.5Re-X (Cr, Mn, V, Nb) alloy during hot deformation. Mater. Lett.; 2023; 331, 133484. [DOI: https://dx.doi.org/10.1016/j.matlet.2022.133484]

26. Mirzadeh, H. Constitutive Description of 7075 Aluminum Alloy During Hot Deformation by Apparent and Physically-Based Approaches. J. Mater. Eng. Perform.; 2015; 24, pp. 1095-1099. [DOI: https://dx.doi.org/10.1007/s11665-015-1389-1]

27. He, D.-G.; Lin, Y.C.; Wang, L.-H.; Wu, Q.; Zu, Z.-H.; Cheng, H. Influences of pre-precipitated δ phase on microstructures and hot compressive deformation features of a nickel-based superalloy. Vacuum; 2019; 161, pp. 242-250. [DOI: https://dx.doi.org/10.1016/j.vacuum.2018.12.043]

28. Khodashenas, H.; Mirzadeh, H.; Malekan, M.; Emamy, M. Constitutive Modeling of Flow Stress during Hot Deformation of Sn–Al–Zn–Cu–Mg Multi-Principal-Element Alloy. Vacuum; 2019; 170, 108970. [DOI: https://dx.doi.org/10.1016/j.vacuum.2019.108970]

29. Xia, Q.; Yuan, S.; Xiao, G.; Long, J.; Cheng, X. Meso-modelling study of the mechanical response and texture evolution of magnesium alloy during hot compression. Mater. Today Commun.; 2021; 27, 102469. [DOI: https://dx.doi.org/10.1016/j.mtcomm.2021.102469]

30. Long, J.; Xia, Q.; Xiao, G.; Qin, Y.; Yuan, S. Flow characterization of magnesium alloy ZK61 during hot deformation with improved constitutive equations and using activation energy maps. Int. J. Mech. Sci.; 2021; 191, 106069. [DOI: https://dx.doi.org/10.1016/j.ijmecsci.2020.106069]

31. Wen, D.; Gao, C.; Zheng, Z.; Wang, K.; Xiong, Y.; Wang, J.; Li, J. Hot tensile behavior of a low-alloyed ultrahigh strength steel: Fracture mechanism and physically-based constitutive model. J. Mater. Res. Technol.; 2021; 13, pp. 1684-1697. [DOI: https://dx.doi.org/10.1016/j.jmrt.2021.05.100]

32. Tang, C.; Liu, W.; Chen, Y.; Liu, X.; Deng, Y. Hot Deformation Behavior of a Differential Pressure Casting Mg-8Gd-4Y-Nd-Zr Alloy. J. Mater. Eng. Perform.; 2016; 26, pp. 383-391. [DOI: https://dx.doi.org/10.1007/s11665-016-2422-8]

33. Tian, X.; Chen, F.; Jiang, J.; Wu, G.; Cui, Z.; Qian, D.; Han, X.; Wang, B.; Wang, H.; Wang, H. et al. Experimental analyses and numerical modeling of the microstructure evolution of aluminum alloy using an internal state variable plasticity-based approach coupled with the effects of second phase. Int. J. Plast.; 2022; 158, 103416. [DOI: https://dx.doi.org/10.1016/j.ijplas.2022.103416]

34. Chen, X.-M.; Lin, Y.C.; Hu, H.-W.; Luo, S.-C.; Zhou, X.-J.; Huang, Y. An Enhanced Johnson–Cook Model for Hot Compressed A356 Aluminum Alloy. Adv. Eng. Mater.; 2021; 23, 2000704. [DOI: https://dx.doi.org/10.1002/adem.202000704]

35. Lin, Y.C.; Huang, J.; Li, H.-B.; Chen, D.-D. Phase transformation and constitutive models of a hot compressed TC18 titanium alloy in the α+β regime. Vacuum; 2018; 157, pp. 83-91. [DOI: https://dx.doi.org/10.1016/j.vacuum.2018.08.020]

36. He, D.; Chen, S.-B.; Lin, Y.C.; Xie, H.; Li, C. Hot tensile behavior of a 7046-aluminum alloy: Fracture mechanisms and constitutive models. Mater. Today Commun.; 2023; 34, 105209. [DOI: https://dx.doi.org/10.1016/j.mtcomm.2022.105209]

37. Pang, G.D.; Lin, Y.C.; Qiu, Y.L.; Jiang, Y.Q.; Xiao, Y.W.; Chen, M.S. Dislocation Density–Based Model and Stacked Auto-Encoder Model for Ti-55511 Alloy with Basket-Weave Microstructures Deformed in A+ β Region. Adv. Eng. Mater.; 2021; 23, 2001307. [DOI: https://dx.doi.org/10.1002/adem.202001307]

38. Wen, D.; Yue, T.; Xiong, Y.; Wang, K.; Wang, J.; Zheng, Z.; Li, J. High-temperature tensile characteristics and constitutive models of ultrahigh strength steel. Mater. Sci. Eng. A; 2021; 803, 140491. [DOI: https://dx.doi.org/10.1016/j.msea.2020.140491]

39. Yu, Z.; Ma, Q.; Su, X.; Lai, X.; Tibbenham, P. Constitutive modeling for large deformation behavior of thermoplastic olefin. Mater. Des.; 2010; 31, pp. 1881-1886. [DOI: https://dx.doi.org/10.1016/j.matdes.2009.10.059]

40. Fan, X.; Yang, H. Internal-state-variable based self-consistent constitutive modeling for hot working of two-phase titanium alloys coupling microstructure evolution. Int. J. Plast.; 2011; 27, pp. 1833-1852. [DOI: https://dx.doi.org/10.1016/j.ijplas.2011.05.008]

41. He, D.; Yan, X.-T.; Lin, Y.C.; Zhang, S.; Chen, Z.-J. Microstructure evolution and constitutive model for a Ni-Mo-Cr base alloy in double-stages hot compression with step-strain rates. Mater. Charact.; 2022; 194, 112385. [DOI: https://dx.doi.org/10.1016/j.matchar.2022.112385]

42. Chen, F.; Wang, H.; Zhu, H.; Zhu, H.; Ren, F.; Cui, Z. High-temperature deformation mechanisms and physical-based constitutive modeling of ultra-supercritical rotor steel. J. Manuf. Process.; 2019; 38, pp. 223-234. [DOI: https://dx.doi.org/10.1016/j.jmapro.2019.01.021]

43. He, D.-G.; Lin, Y.C.; Chen, J.; Chen, D.-D.; Huang, J.; Tang, Y.; Chen, M.-S. Microstructural evolution and support vector regression model for an aged Ni-based superalloy during two-stage hot forming with stepped strain rates. Mater. Des.; 2018; 154, pp. 51-62. [DOI: https://dx.doi.org/10.1016/j.matdes.2018.05.022]

44. Quan, G.-Z.; Zhang, Z.-H.; Zhou, Y.; Wang, T.; Xia, Y.-F. Numerical Description of Hot Flow Behaviors at Ti-6Al-2Zr-1Mo-1V Alloy By GA-SVR and Relative Applications. Mater. Res.; 2016; 19, pp. 1253-1269. [DOI: https://dx.doi.org/10.1590/1980-5373-mr-2016-0280]

45. Zhao, J.; Ding, H.; Zhao, W.; Huang, M.; Wei, D.; Jiang, Z. Modelling of the hot deformation behaviour of a titanium alloy using constitutive equations and artificial neural network. Comput. Mater. Sci.; 2014; 92, pp. 47-56. [DOI: https://dx.doi.org/10.1016/j.commatsci.2014.05.040]

46. Sun, Y.; Zeng, W.D.; Zhao, Y.Q.; Zhang, X.M.; Shu, Y.; Zhou, Y.G. Modeling constitutive relationship of Ti40 alloy using artificial neural network. Mater. Des.; 2011; 32, pp. 1537-1541. [DOI: https://dx.doi.org/10.1016/j.matdes.2010.10.004]

47. Mosleh, A.; Mikhaylovskaya, A.; Kotov, A.; Pourcelot, T.; Aksenov, S.; Kwame, J.; Portnoy, V. Modelling of the Superplastic Deformation of the Near-α Titanium Alloy (Ti-2.5 Al-1.8 Mn) Using Arrhenius-Type Constitutive Model and Artificial Neural Network. Metals; 2017; 7, 568. [DOI: https://dx.doi.org/10.3390/met7120568]

48. Ge, G.; Wang, Z.; Zhang, L.; Lin, J. Hot deformation behavior and artificial neural network modeling of β-γ TiAl alloy containing high content of Nb. Mater. Today Commun.; 2021; 27, 102405. [DOI: https://dx.doi.org/10.1016/j.mtcomm.2021.102405]

49. Hu, C.; Martin, S.; Dingreville, R. Accelerating phase-field predictions via recurrent neural networks learning the microstructure evolution in latent space. Comput. Methods Appl. Mech. Eng.; 2022; 397, 115128. [DOI: https://dx.doi.org/10.1016/j.cma.2022.115128]

50. Kautz, E.J. Predicting material microstructure evolution via data-driven machine learning. Patterns; 2021; 2, 100285. [DOI: https://dx.doi.org/10.1016/j.patter.2021.100285]

51. Khandelwal, S.; Basu, S.; Patra, A. A Machine Learning-based surrogate modeling framework for predicting the history-dependent deformation of dual phase microstructures. Mater. Today Commun.; 2021; 29, 102914. [DOI: https://dx.doi.org/10.1016/j.mtcomm.2021.102914]

52. Mei, H.; Lang, L.; Yang, X.; Liu, Z.; Li, X. Study on Constitutive Relation of Nickel-Base Superalloy Inconel 718 Based on Long Short Term Memory Recurrent Neural Network. Metals; 2020; 10, 1588. [DOI: https://dx.doi.org/10.3390/met10121588]

53. Benabou, L. Development of LSTM networks for predicting viscoplasticity with effects of deformation, strain rate and temperature history. J. Appl. Mech.; 2021; 88, 071008. [DOI: https://dx.doi.org/10.1115/1.4051115]

54. Zhou, H.; Zhang, S.; Peng, J.; Zhang, S.; Li, J.; Xiong, H.; Zhang, W. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting. Proceedings of the AAAI Conference on Artificial Intelligence; Vancouver, BC, Canada, 2–9 February 2021; Volume 35, pp. 11106-11115. [DOI: https://dx.doi.org/10.1609/aaai.v35i12.17325]

55. Zou, R.; Duan, Y.; Wang, Y.; Pang, J.; Liu, F.; Sheikh, S.R. A novel convolutional informer network for deterministic and probabilistic state-of-charge estimation of lithium-ion batteries. J. Energy Storage; 2023; 57, 106298. [DOI: https://dx.doi.org/10.1016/j.est.2022.106298]

56. Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention Is All You Need. Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017); Long Beach, CA, USA, 4–9 December 2017; Volume 30.

57. Li, S.Y.; Jin, X.Y.; Xuan, Y.; Zhou, X.Y.; Chen, W.H.; Wang, Y.X.; Yan, X.F. Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting. Adv. Neural Inf. Process. Syst.; 2019; 32.

58. Child, R.; Gray, S.; Radford, A.; Sutskever, I. Generating Long Sequences with Sparse Transformers. arXiv; 2019; arXiv: 1904.10509

59. Kitaev, N.; Kaiser, Ł.; Levskaya, A. Reformer: The Efficient Transformer. arXiv; 2020; arXiv: 2001.04451

60. Wang, S.N.; Li, B.Z.; Khabsa, M.; Fang, H.; Ma, H. Linformer: Self-Attention with Linear Complexity. arXiv; 2020; arXiv: 2006.04768

61. Martins, P.H.; Marinho, Z.; Martins, A.F.T. ∞-former: Infinite Memory Transformer-former: Infinite Memory Transformer. Proceedings of the the 60th Annual Meeting of the Association for Computational Linguistics; Dublin, Ireland, 22–27 May 2022; pp. 5468-5485. [DOI: https://dx.doi.org/10.18653/v1/2022.acl-long.375]

62. Tsai, Y.-H.H.; Bai, S.; Yamada, M.; Morency, L.-P.; Salakhutdinov, R. Transformer Dissection: An Unified Understanding for Transformer’s Attention via the Lens of Kernel. arXiv; 2019; [DOI: https://dx.doi.org/10.18653/v1/d19-1443] arXiv: 1908.11775

63. Calafiore, G.C.; Gaubert, S.; Possieri, C. Log-Sum-Exp Neural Networks and Posynomial Models for Convex and Log-Log-Convex Data. IEEE Trans. Neural Netw. Learn. Syst.; 2019; 31, pp. 827-838. [DOI: https://dx.doi.org/10.1109/TNNLS.2019.2910417]

64. Calafiore, G.C.; Gaubert, S.; Possieri, C. A Universal Approximation Result for Difference of Log-Sum-Exp Neural Networks. IEEE Trans. Neural Netw. Learn. Syst.; 2020; 31, pp. 5603-5612. [DOI: https://dx.doi.org/10.1109/TNNLS.2020.2975051] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/32167912]

65. Yu, F.; Koltun, V.; Funkhouser, T. Dilated residual networks. arXiv; 2017; arXiv: 1705.09914

66. Clevert, D.A.; Unterthiner, T.; Hochreiter, S. Fast and Accurate Deep Network Learning by Exponential Linear Units (Elus). arXiv; 2015; arXiv: 1511.07289

67. Devlin, J.; Chang, M.W.; Lee, K.; Toutanova, K. Bert: Pre-Training of Deep Bidirectional Transformers for Language Understanding. arXiv; 2018; arXiv: 1810.04805

68. Smith, L.N. A Disciplined Approach to Neural Network Hyper-Parameters: Part 1–Learning Rate, Batch Size, Momentum, and Weight Decay. arXiv; 2018; arXiv: 1803.09820

69. Masters, D.; Carlo, L. Revisiting small batch training for deep neural networks. arXiv; 2018; arXiv: 1804.07612

70. He, F.X.; Liu, T.L.; Tao, D.C. Control batch size and learning rate to generalize well: Theoretical and empirical evidence. Adv. Neural. Inf. Process. Syst.; 2019; 32.

71. Keskar, N.S.; Mudigere, D.; Nocedal, J.; Smelyanskiy, M.; Tang, P.T.P. On large-batch training for deep learning: Generalization gap and sharp minima. arXiv; 2016; arXiv: 1609.04836

72. Hoffer, E.; Hubara, I.; Soudry, D. Train longer, generalize better: Closing the generalization gap in large batch training of neural networks. Adv. Neural. Inf. Process. Syst.; 2017; 30.