Development and Evaluation of Transformer-Based Basic Fighter Maneuver Decision-Support Scheme for Piloting During Within-Visual-Range Air Combat

Abstract

In within-visual-range (WVR) air combat, basic fighter maneuvers (BFMs) are widely used. Air combat engagement database (ACED) is a dedicated database for researching WVR air combat. Utilizing the data in ACED, a Transformer-based BFM decision support scheme is developed to enhance the pilot’s BFM decision making in WVR air combat. The proposed Transformer-based model significantly outperforms the baseline long short-term memory (LSTM)-based model in accuracy. To augment the interpretability of this approach, Shapley Additive Explanation (SHAP) analysis is employed, exhibiting the rationality of the Transformer-based model’s decisions. Furthermore, this study establishes a comprehensive framework for evaluating air combat performance, validated through the utilization of data from ACED. The application of the framework in WVR air combat experiments shows that the Transformer-based model increases the winning rate in combat from 30% to 70%, the average percentage of tactical advantage time from 4.81% to 14.73%, and the average situational advantage time share from 17.83% to 25.19%, which substantially improves air combat performance, thereby validating its effectiveness and applicability in WVR air combat scenarios.

1. Introduction

1.1. Motivation

Air combat engagements can be classified into two scenarios based on distance: beyond-visual-range (BVR) and within-visual-range (WVR). In BVR air combat, aircraft employ radar-guided missiles and rely on onboard sensors or surveillance aircraft for mission deployments, emphasizing the importance of situational awareness and the ability to resist electronic interference [1,2]. WVR air combat involves the use of onboard guns or infrared-guided missiles, requiring greater aircraft maneuverability and rapid pilot responses, in which pilots often rely on basic fighter maneuvers (BFMs) to execute diverse tactical operations [3].

BFMs, rooted in aerodynamics and pursuit geometry, play a crucial role in WVR air combat. They significantly enhance a fighter’s maneuverability, allowing pilots to respond quickly [3,4,5], evade enemy weapon threats, and execute effective attacks. Since World War I, BFMs have been widely used in WVR, and their maturity and practical effectiveness have been validated in multiple studies [6,7]. However, considering the physiological stress and limited situational awareness faced by pilots in combat, the rational selection and execution of BFM in WVR remain challenging [8]. To address these challenges, the development of methods to support pilots in making informed BFM decisions is essential for improving the success rate of air combat missions. However, existing search-based BFM decision-making algorithms exhibit computational inefficiency [9,10], making them unsuitable for meeting the immediacy requirements of aerial combat. Similarly, machine-learning-based BFM decision-support approaches [11,12] achieve a win rate of no more than 60% in air combat simulation experiments, indicating their limited capacity to improve combat effectiveness. These limitations highlight the critical need for advanced, efficient BFM decision-support algorithms capable of significantly improving air combat performance, representing a pressing and unresolved research challenge in the field.

The primary weapons in WVR air combat are short-range missiles and onboard guns. Short-range missiles are characterized by high maneuverability, which allows them to autonomously track and engage targets. Consequently, aircraft can avoid performing complex maneuvers to gain a positional advantage for missile launches, reducing the demands for intricate BFM decision making for pilots [13]. On the other hand, launching onboard guns requires maintaining an ideal firing distance to the target while ensuring the aircraft’s nose is pointed at the enemy, which places greater demands on the aircraft’s maneuverability [14]. Therefore, in gun-based WVR air combat scenarios, pilots are required to execute more complex BFM decisions to secure a tactical advantage. In this study, we referenced the scenario settings of the AlphaDogfight competition, focusing on gun-based WVR air combat scenarios. This research aimed to propose a BFM decision-support model specifically designed for complex WVR combat scenarios.

1.2. Related Work

1.2.1. BFM Decision Scheme and Explainability Analysis

In WVR air combat, pilots need to determine the optimal BFM maneuver for their aircraft based on its historical states over a defined time window and the relative dynamics between friendly and adversarial forces. Accordingly, the BFM decision-support task can be framed as a time-series data classification problem, involving the analysis and classification of complex battlefield information to generate optimal BFM support commands. In recent years, data-driven approaches such as deep reinforcement learning [5], fuzzy decision trees [15,16,17], and deep learning [12,18,19,20] have been widely applied in this research area. In [12], an LSTM-based BFM decision-support scheme was proposed, and its effectiveness was validated. LSTM incorporates gating mechanisms to effectively manage long-term dependencies. However, the reliance of LSTM on adjacent temporal information poses challenges in capturing long-range global dependencies in BFM decision making. To address this limitation, the Transformer model proposed by Vaswani et al. [21] offers a novel approach for processing time-series data through its self-attention mechanism. Compared to LSTM, the Transformer efficiently handles long-range dependencies and significantly enhances training efficiency due to its parallel computation capabilities. Thus, the Transformer holds greater potential for research and application in BFM decision support in WVR air combat.

Despite their strong performance across various tasks, the ‘black box’ nature of deep learning models, including Transformers, poses significant challenges for applications requiring explainability. Therefore, the analysis of model explainability is particularly important. Techniques for interpreting neural networks have evolved to clarify the internal processes of AI algorithms, thereby enhancing transparency and fostering trustworthiness.

Currently, various explainability analysis methods have emerged, with Class Activation Mapping (CAM) [22] and Shapley Additive Explanations (SHAPs) [23] enhancing neural network explainability from different perspectives. These methods provide valuable insights for understanding and optimizing model decision-making mechanisms and have been widely applied in recent years [24,25,26,27,28]. CAM demonstrates superiority in elucidating attention mechanism distributions in deep learning models [29]. This method can flexibly adapt to different model architectures by directly applying the model’s internal parameters to interpret input images. However, it often requires adjustments to the model’s architecture, implying the need for retraining when applied to BFM decision-support models, which can be resource-intensive. In contrast, SHAPs accommodate various model types and data, providing a consistent interpretative framework across different models. This algorithm supports both global and local explainability analysis, making it more suitable for air combat scenarios. In WVR air combat, conducting SHAP-based explainability analysis on BFM decision-support models helps pilots understand the model’s preferences for different maneuver strategies and the underlying reasons, thereby enhancing the transparency and credibility of BFM decision support.

The goal of SHAP-based explainability analysis in neural networks is to reveal the logic and processes behind model decisions, ensuring that the Transformer-based BFM decision model aligns with cognitive demands in WVR air combat. However, SHAP analysis explains the rationality of the Transformer-based BFM decision model only at the algorithmic level, lacking an assessment of its practical effectiveness in complex air combat situations. Therefore, it is essential to establish an air combat situational assessment model to quantify the contribution of the Transformer-based BFM decision-support model to mission success through the analysis of air combat data. This model evaluates whether the BFM support model can assist pilots in gaining an advantage in a constantly evolving air combat environment, providing crucial evidence for validating the model’s practicality in air combat missions.

1.2.2. Situation Assessment of Air Combat

Air combat situational assessment can be categorized into parametric [30,31,32] and non-parametric methods [6,33,34,35,36]. Parametric methods primarily rely on Bayesian approaches to establish reasoning networks that produce air combat threat assessment models [30]. However, these methods involve substantial computational requirements and high model maintenance costs, making them unsuitable for this research context. Non-parametric methods, based on expert experience, focus on extracting situational assessment indicators to construct advantage functions through weighted summation, resulting in an overall air combat threat assessment model. Research based on non-parametric methods unfolds along two dimensions: the design and optimization of assessment indicators [6,33,34,35,36], and the determination and solution of the impact weights of situational indicators [37,38,39,40,41,42,43,44,45].

Traditional situational assessment indicators primarily consider spatial factors, including speed, altitude, and distance [33,34]. However, these methods often lead to overly simplistic indicator designs that struggle to comprehensively account for the complexities and uncertainties of the battlefield environment. Scholars like Shin have developed assessment models based on energy maneuver factors and speed overload relationships, facilitating collaborative maneuvers and role assignments in air combat tasks [6]. Furthermore, Zhao et al. focused on weapon engagement areas and combat geometry, extending situational assessments to various scenarios of BVR combat by developing advantage functions [35]. Gu et al. constructed a WVR air combat situational assessment model based on key situational factors affecting air-to-air missile engagement areas and kill probabilities, providing a basis for maneuver decision making in WVR combat [36]. However, their model inadequately considers the attack conditions and ranges of unguided weapons such as cannons, while the influence of fighter maneuver factors poses additional challenges, making it less suitable for WVR situational assessments. The above studies have designed and improved situational indicators, enhancing the accuracy and realism of situational assessment models. However, they are limited in their applicability to the cannon-based WVR air combat task scenarios in this study.

Regarding the determination and solution of situational indicator impact weights, the commonly used and mature techniques include the Analytic Hierarchy Process (AHP) [37,38,39], Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) [40,41,42], and Best–Worst Method (BWM) [43,44,45]. The AHP breaks down complex decision problems into multiple levels that include various criteria. Decision makers construct pairwise comparison matrices to assess the relative importance of each criterion, calculating weights and deriving final results through weighted aggregation. This method aids in breaking down complex issues, enhancing the structure of the decision-making process. However, the reliance on subjective judgment in pairwise comparison matrices may introduce bias, and the dimensions and computational complexity increase significantly with the number of factors. TOPSIS identifies an ideal and a negative ideal solution during the decision process, selecting the option closest to the ideal and furthest from the negative ideal. Compared to the AHP, TOPSIS offers a simpler calculation process, suitable for large-scale problems. However, it is sensitive to initial weights, leading to potential bias in results, making it less applicable in the dynamic and complex environment of air combat. BWM is an optimization technique for determining weights among criteria in multicriteria decision making [46]. Its core idea is to select the best and worst criteria as identified by decision makers and compare them with others to establish relative weights. This method reduces subjective judgment while improving weight consistency. BWM requires identifying only one optimal and one worst criterion, comparing others with these two, significantly reducing the number of pairwise comparisons, thereby minimizing the impact of subjective errors on overall weights. Given its ability to obtain stable weight values while significantly reducing computational complexity in environments defined by numerous evaluation indicators, BWM is better suited for the highly dynamic air combat context.

1.3. Overview of This Paper

This paper addresses the issues of the research conducted in [12] by proposing a Transformer-based BFM decision-support model. Although existing work has implemented an LSTM model to assist in BFM decision making, LSTM is better suited for handling short-term dependencies, which limits its effectiveness in managing complex global temporal dependencies. In contrast, the self-attention mechanism of the Transformer can effectively capture relationships at any point in a sequence, making it more suitable for BFM decision-support tasks. Additionally, this study conducted an explainability analysis of the neural network based on SHAP to validate whether the Transformer-based BFM decision model aligns with the cognitive requirements of WVR air combat. To evaluate the practical effectiveness of the Transformer-based BFM decision support in actual air combat, this study introduced a situational assessment model, enabling the quantitative analysis of air combat situations. Finally, this study proposed air combat performance metrics based on the situational assessment model and conduct 1vs1 WVR air combat experiments to quantitatively evaluate overall air combat performance, thereby validating the effectiveness of the BFM model’s decision support scheme.

In summary, the purpose of this study was to propose a Transformer-based BFM decision-support algorithm that meets the immediacy requirements of aerial combat by providing pilots with BFM decision instructions in real time and addresses the limitations of the existing LSTM-based algorithm to improve air combat win rates. Additionally, SHAP analysis and an air combat situation assessment model were introduced to validate the model’s effectiveness and performance in actual air combat. By introducing the Transformer model, this study aimed to provide pilots with more comprehensive and precise decision support to improve overall performance in WVR air combat. The main innovations and contributions of this paper are as follows:

1. A Transformer-based BFM decision-support model is proposed, with a decision-making time throughout the air combat engagement process that is significantly less than 1 s (not exceeding 0.006 s), meeting the immediacy requirements of aerial combat. Additionally, compared to the LSTM model in [12], the Transformer model more effectively captures global temporal dependencies and adapts to complex air combat scenarios, resulting in a significant improvement in the air combat win rate and providing more comprehensive decision support for pilots.

2. This study developed a situational assessment model for air combat, establishing multiple situational indicator models and designing indicator weights based on the BWM, enabling quantitative analysis of the advantages and disadvantages of both friendly and enemy situations. The effectiveness of the situational assessment model was validated using air combat data.

3. We proposed evaluation metrics for WVR air combat: This study conducted multiple sets of 1vs1 WVR air combat experiments and quantitatively assessed overall performance in air combat based on the situational assessment model, verifying the effectiveness of the Transformer-based BFM decision-support model in real air combat applications.

These innovations address the shortcomings identified in [12], enhancing the performance of the BFM decision-support model and providing a more comprehensive assessment of its effectiveness.

The remainder of this paper is organized as follows: Section 2 introduces the Transformer-based BFM decision-support scheme and the improved explainability analysis through SHAP analysis. Section 3 establishes an air combat performance evaluation framework. Section 4 introduces the contrast experiments on the LSTM-based and Transformer-based BFM decision-support scheme. Finally, Section 5 presents the conclusions.

2. Transformer-Based BFM Decision-Support Scheme

This section presents an explainable Transformer-based BFM decision-support scheme for WVR air combats. The input data should include all historical flight states utilized by pilots for making BFM decisions during combat engagements, consisting of 15 specific input features upon which BFM decision-making relies. The output data represent the composition of BFM types corresponding to the pilots’ decisions. The subsequent sections provide a comprehensive overview of the database and algorithmic models employed in this framework.

2.1. Data Preparation

This study focused on pilot maneuvering behavior in 1vs1 WVR air combat engagements with guns. Considering the specific environment and conditions of WVR air combat, the combat scenario is defined as follows: Both aircraft are of the same type, with comparable maneuverability and fire control capabilities. At the beginning of the engagement, both aircraft maintain the same initial altitude and Mach number, flying head-on at a distance of 12 km. Both pilots primarily employ BFM strategies during the combat. The data collection in ACED specifically targets WVR air combat engagements using guns, allowing for multiple gun attacks during the combat until one aircraft is shot down. The aircraft that is shot down is considered defeated.

The ACED relied on commercial air combat software that provided a simulated 1vs1 WVR air combat environment. In order to achieve higher authenticity in depicting real air combat environments, the hardware configurations shown in Figure 1 were adopted to simulate the aircraft cockpit environment. As shown in Figure 1, the screens and TRACK infrared (IR) camera or virtual reality (VR) locator were employed to simulate the aircraft cockpit. Furthermore, flight joysticks were provided to emulate the pilot’s control input. The hardware configurations supported pilot flight maneuver control and data recording (including flight controls, aircraft states, and weapon deployments) during air combats [12].

Based on the defined air combat scenario and high-fidelity flight simulation software, this study allocated 3 fighter pilots with different air combat engagement experiences to establish the ACED and conduct WVR air combat tests (This study was conducted in a simulated environment with professional fighter pilots participating voluntarily, and it did not involve sensitive personal data or impact participants’ welfare. Hence, it was granted an exemption from formal ethical approval by the Ethics Review Committee of Fudan University.) (see Table 1). In the flight, to better capture the engagement behavior of the pilots, this study selected pilot A, with the highest level of engagement experience, to operate blue aircraft (typically the winner) in the ACED. Pilot B (medium level) flew red aircraft in the ACED and WVR air combat tests. In the hardware configurations depicted in Figure 1, pilot A was capable of defeating in-service pilots in simulated WVR air combat. Meanwhile, pilot B exhibited comparable performance to in-service pilots in simulated WVR air combat. A pilot (novice level) conducted WVR air combat tests based on the BFM decision scheme (in Section 4) to verify the effectiveness of the scheme. In data collection, the pilots were required to perform a series of air combat engagements, and the flight data were recorded. In order to select a more suitable BFM during WVR air combat, the recorded flight data of the blue aircraft were utilized for training the BFM decision model. Experimental results indicated that the air combat capabilities of those 3 pilots aligned with their respective air combat levels [12]. The BFM decision model primarily focused on learning the BFM decision characteristics of pilot A.

In WVR air combat, pilots determine the appropriate BFM for their aircraft based on its state and the relative situation between our aircraft and enemy aircraft over a given historical time window. According to prior data analysis of ACED [12], BFM maneuvers are categorized into 6 types, with the duration of each maneuver not exceeding 30 s, which indicates that pilots rely on, at most, 30 s of historical information to make BFM maneuvering decisions (i.e., changing the BFM). Therefore, the BFM decision-support problem can be defined as predicting the type of BFM adopted by the pilot based on battlefield information within a 30 s time window. Based on this analysis, the BFM decision problem for decision support can be transformed into a classification problem using time-series data. The output data consist of BFM types for pilot decision making, as illustrated in Figure 2, including Break Turn, Split-S, Immelman Turn, High-Yo-Yo, Low-Yo-Yo, and Pure Pursuit, totaling 6 categories.

To calculate the data that pilots rely on for BFM decision making in air combat, this study referred to the aircraft kinematics and dynamics equations shown in Equation (1) and followed the methodologies in [10,11,47]. By comprehensively considering the state of their own aircraft and the relative situation between friendly and enemy aircraft, a total of 15 features were collected as input parameters for the BFM decision-making model, while ignoring the influence of other atmospheric factors.

(1)$\begin{array}{c}\hfill \begin{array}{c}\hfill \left\{\begin{array}{c}\dot{x}=vcos\theta cos\psi \hfill \\ \dot{y}=vcos\theta sin\psi \hfill \\ \dot{z}=vsin\theta \hfill \\ \dot{v}={\displaystyle \frac{Tcos\beta -D}{m}}-g·sin\theta \hfill \\ \dot{\theta }={\displaystyle \frac{Tcos\beta -D}{m}}-gcos\theta \hfill \\ \dot{\psi }={\displaystyle \frac{L+Tsin\beta }{mvcos\theta }}sin\varphi \hfill \end{array}\right.\end{array}\end{array}$

The input vector $\Phi \left(t\right)$ can be expressed as follows:

(2)$\begin{array}{c}\hfill \begin{array}{c}\hfill \Phi \left(t\right)=\{{\psi }_o,{\theta }_o,{\varphi }_o,N{y}_o,v_o,{\alpha }_o,{\beta }_o,H_o,v_c,x_r,y_r,z_r,{\psi }_r,{\theta }_r,{\varphi }_r\}.\end{array}\end{array}$

The input feature data can be categorized into two types: (a) the state variables of the own aircraft that influence pilot decision making in BFM, which are critical characteristics of the own aircraft, reflecting its flight attitude and performance, and can be directly accessed from ACED, and (b) the situational variables that influence pilot decision making in BFM. These variables describe the dynamic states of enemy aircraft relative to the own aircraft, providing information such as observed enemy aircraft positions and flight attitudes during air combat engagements. The specific parameter definitions are illustrated in Table 2.

2.2. Transformer-Based Model

In this section, the Transformer model is employed to construct a BFM decision-support scheme, and SHAP analysis is employed to enhance the credibility of the model.

2.2.1. Introduction of Transformer

The Transformer is a new technology in the field of NLP proposed by Vaswani et al. [21] in 2017. This model has a self-attention mechanism, which allows the direct capture of the dependencies between any 2 positions in the input sequence. The structure of the Transformer, illustrated in Figure 3, consists of stacked encoders and decoders. Each encoder consists of 2 main parts: a Multi-Head Attention layer and a Feed Forward neural network. The decoder has a similar structure with the encoder but includes an additional Multi-Head Cross-Attention layer between these 2 parts to attend to the encoder’s outputs.

Furthermore, the decoder’s Multi-Head Attention layer incorporates a masking mechanism, hence referred to as the Masked Multi-Head Attention layer. This masking prevents the model from utilizing future information when generating the current output. Hence, each layer in the decoder consists of 3 main parts: a Masked Multi-Head Attention layer, a Multi-Head Cross-Attention layer, and a Feed Forward neural network.

The Transformer model, due to its self-attention mechanism, demonstrates strong capabilities in handling long-term dependencies, including semantic feature extraction, and efficiently integrating task-relevant features. In view of BFM decision support, tactical choices are often influenced by multiple preceding and succeeding moments rather than merely the state at adjacent times. This implies that a pilot’s decision making must consider not only the states in nearby moments but also the global dependencies over a longer time interval. Compared to LSTM, which is more suited for short-term dependencies, the Transformer is algorithmically better aligned with the BFM decision-support task scenario. Moreover, the Transformer supports parallel computation, significantly enhancing operational efficiency. This enables its faster training speeds and better performance when faced with large datasets and complex tasks.

This study adopted the Transformer to improve the BFM decision-support scheme, designing the input parameters as a 15 × 31 parameter matrix, where the rows represent 15 input features, and the columns represent a 1 s time interval. The output data signify the type of BFM for the next second, represented by integers from 0 to 5. By adjusting the structure of the input and output layers, we ensure that they meet our task requirements. The subsequent sections cover the training, testing, and SHAP analysis of the Transformer model.

2.2.2. Model Training Process

In WVR air combat, the time window upon which BFM decisions by pilots rely was set at 30 s, with a decision-making frequency of 1 Hz [12]. Consequently, the input data were structured into a 15 × 31 parameter matrix, where rows represented 15 input features; columns represented time intervals of 1 s. The output data denoted the BFM types for the next second, represented by 6 integers ranging from 0 to 5. Initially, the Transformer model was trained using the aforementioned data, with 20% of the data randomly selected for testing. Following this, within the same experimental setup, the LSTM model was trained and tested using the same dataset for comparative analysis against the Transformer model.

During the network training process of the network, the basic parameters were first configured. Since the batch size, the learning rate, and the network training epochs mainly influence the convergence and stability of the training process and are independent of the model architecture, the hyperparameters optimized for the LSTM model were directly applied to the Transformer model. The batch size was set to 64, the learning rate was set to $1\times {10}^{-4}$, and the network training epochs were set to 500 [12]. The cross-entropy loss function was employed, with the Adam optimizer selected. This study focused on the experimental validation of the architecture-related hyperparameters of the Transformer, including the number of hidden layers ($d_{model}$), the number of heads in the Multi-Head Attention layer (h), and the number of network layers (N), to ensure optimal model performance. To determine the optimal hyperparameters of the network, a series of comparative experiments were conducted, with each set of hyperparameters undergoing 5 repeated tests to ensure result consistency. The model performance under different hyperparameters was evaluated based on test accuracy ($\mu$), as shown in Figure 4.

As illustrated in Figure 4, when (h = 8), the network architecture with (N = 4) and ($d_{model}$ = 32) achieved the highest test accuracy, reaching 97.31%. Moreover, the average test accuracy tended to decrease with increases in N and $d_{model}$. Ultimately, the network structure with the highest test accuracy was chosen as the definitive Transformer-based BFM decision model.

Additionally, the results illustrated in Figure 4 demonstrate that the test accuracy of different network structures consistently remained above 94%, exceeding the maximum test accuracy achieved by LSTM networks at 91.08%. This indicates that Transformer-based network structures exhibit superior performance in BFM decision making compared to LSTM-based models.

2.2.3. Final Model Results

The final hyperparameters of the Transformer model adopted in the BFM decision support scheme are presented in Table 3.

In the final Transformer-based BFM decision model obtained after training and testing, the test accuracy ($\mu$) of the model gradually improved with an increase in the number of training epochs, while the loss value continued to decrease (refer to Figure 5). Particularly, when the training epochs reached 100, the model achieved a test accuracy exceeding 90% on the test dataset, demonstrating the robust generalization ability of the model in BFM decision making when encountering new data.

The confusion matrix of this model is illustrated in Figure 6. From the graph, it can be observed that the test accuracy of the model exceeded 95% for all six types of BFM. Specifically, the test accuracy for Immelman Turn (95.53%) was the lowest, while the accuracy for the other five types of BFM all exceeded 96.90%, indicating the excellent performance of the BFM decision model across various BFMs.

The Transformer-based model achieved a significant improvement in test accuracy compared to the LSTM-based model. In addition to maintaining high test accuracy in Pure Pursuit, the Transformer-based model also demonstrated significant enhancements in the test accuracy of the other five types of BFM. Therefore, the Transformer-based BFM decision model contributes to assisting pilots in making more accurate BFM decisions during WVR air combat.

2.3. SHAP Analysis of Transformer Model

This section aims to enhance the interpretability of the Transformer-based BFM decision model through SHAP analysis. It comprehensively evaluates the contributions of various input features within the Transformer model to the BFM outputs. Through this comparative analysis, the reasons behind the Transformer model’s improvement in test accuracy are elucidated, thereby facilitating a deeper understanding of the model’s operational mechanism and the key factors that contribute to performance enhancement.

Figure 7 illustrates the results of the feature importance of the Transformer model, describing the extent to which the 15 input features influence the outputs. In Figure 7, six input features, including $x_r$, $v_c$, $v_o$, $H_o$, $N{y}_o$, $y_r$, contribute the most to influencing the model results (exceeding 8%). The results also indicate that the SHAP values of positional features $\{v_o,H_o,v_c,x_r,y_r,z_r\}$ are significantly higher than those of angular features $\{{\psi }_o,{\theta }_o,{\varphi }_o,{\alpha }_o,{\beta }_o,{\psi }_r,{\theta }_r,{\varphi }_r\}$, consistent with the analysis results of the LSTM network [12]. This further illustrates that positional features have a more significant impact on BFM outputs than angular features, implying that pilots’ BFM decisions primarily rely on information regarding the positions of both their own and enemy aircraft.

Figure 8 illustrates the global SHAP analysis for Break Turn and Split-S. Figure 8a presents the global SHAP analysis for Break Turn. The results in the figure provide a more precise interpretation of the input features relied upon for choosing Break Turn. $v_o$ has the most significant impact on the decision to execute a Break Turn. Furthermore, when the value of $v_o$ is high, its SHAP value decreases in absolute terms, indicating a lesser influence on the decision to execute Break Turn. Subsequently, $v_c$, $x_r$, $H_o$, and $z_r$ also significantly impact the decision to execute a Break Turn. These findings align with the fundamental understanding of Break Turn in WVR air combat.

Figure 8b presents the global SHAP analysis for Split-S, indicating that $H_o$ has the most significant impact on the decision to execute Split-S. Moreover, when the value of $H_o$ is higher, its SHAP value also increases, indicating that an increase in $H_o$ positively influences the decision to execute Split-S. This is because the Split-S maneuver involves a descending turn, thus requiring a higher altitude for execution. $x_r$, $v_c$, and $v_o$ also significantly influence the decision to execute Split-S. Furthermore, when these values are higher, their corresponding SHAP values also increase, indicating that their increase positively influences the decision to execute Split-S. These findings align with the basic understanding of Split-S in WVR air combat.

Figure 9 illustrates the global SHAP analysis for Immelman Turn and High-Yo-Yo. Figure 9a presents the global SHAP analysis for Immelmann Turn, indicating that $x_r$ has the most significant impact on the decision to execute an Immelmann Turn. Moreover, when the value of $x_r$ is higher, its SHAP value also increases, indicating that an increase in $x_r$ positively influences the decision to execute an Immelmann Turn. Additionally, $v_c$, $v_o$, $y_r$, $H_o$, and $z_r$, all five input features, significantly influence the decision to execute an Immelmann Turn. These results highlight that the decision to execute an Immelmann Turn is strongly influenced by variations in the positions and velocities of both their own and enemy aircraft. These findings align with the fundamental understanding of the Immelmann Turn in WVR air combat.

Figure 9b presents the global SHAP analysis for High-Yo-Yo, indicating that $x_r$ has the most significant impact on the decision to execute a High-Yo-Yo. Moreover, when the value of $x_r$ is higher, its SHAP value decreases, indicating that an increase in $x_r$ negatively influences the decision to execute a High-Yo-Yo. This is because High-Yo-Yo typically occurs when pursuing an enemy aircraft at high speed, approaching the point of overtaking, with both aircraft being close and $x_r$ being small. The decision to execute a High-Yo-Yo is also influenced by factors such as $v_c$, $v_o$, and $N{y}_o$. Although the aircraft’s ${\beta }_o$ has a relatively minor impact on the decision to execute a High-Yo-Yo, for certain extreme values of ${\beta }_o$, its SHAP value is very small, indicating a significant negative impact on the decision to execute a High-Yo-Yo. This is because at higher values of ${\beta }_o$, the aircraft’s maneuverability is generally poor, making it difficult to perform a High-Yo-Yo. These findings align with the basic understanding of High-Yo-Yo in WVR air combat.

Figure 10 illustrates the global SHAP analysis for Low-Yo-Yo and Pure Pursuit. Figure 10a presents the global SHAP analysis for Low-Yo-Yo, indicating that $y_r$ has the most significant impact on the decision to execute a Low-Yo-Yo. Moreover, when the value of $y_r$ is higher, its SHAP value also increases, indicating that an increase in $y_r$ positively influences the decision to execute a Low-Yo-Yo. The decision to execute a Low-Yo-Yo is also influenced by factors such as $H_o$, $v_o$, and $x_r$. Although $z_r$ has a relatively minor impact on the decision to execute a Low-Yo-Yo, for certain extreme values of $z_r$, its SHAP value has a large absolute value, significantly affecting the decision to execute a Low-Yo-Yo. These findings align with the basic understanding of Low-Yo-Yo in WVR air combat.

Figure 10b presents the global SHAP analysis for Pure Pursuit, indicating that $x_r$ has the most significant impact on the decision to execute a Pure Pursuit. Specifically, a significant SHAP value for $x_r$ occurs only when its value is relatively small, indicating a positive effect on Pure Pursuit execution. Pure Pursuit typically occurs when the own aircraft tails the enemy aircraft, placing the enemy within attack range and resulting in a small value of $x_r$. Conversely, a large value of $x_r$ signifies a long distance between the aircraft, rendering Pure Pursuit impractical. Additionally, factors such as $H_o$, $v_o$, and $v_c$ also significantly influence the decision of Pure Pursuit. Although $z_r$ has a minor impact, extreme values of $z_r$ can significantly affect the decision. These findings are consistent with the fundamental understanding of Pure Pursuit in WVR air combat.

The above SHAP analysis results indicate that the input features relied upon by the Transformer-based model for the classification of six BFM categories conform to the fundamental understanding of WVR air combat. Compared to the LSTM-based BFM decision model, the Transformer-based model extracts more influential feature information for BFM decision making, leading to improved test accuracy. The decision process and results of the Transformer-based BFM decision model established in this study are reasonable and can be used to assist pilots in BFM decision making.

3. Air Combat Performance Evaluation Framework

3.1. Metrics for Evaluating Air Combat Performance

This section combines the characteristics of WVR air combat, selecting five metrics for evaluating air combat performance: attack situation, positioning situation, maneuvering situation, energy situation, and intrinsic capability situation. These metrics were chosen to represent the influence of attack conditions, real-time engagement conditions, and static performance metrics on WVR air combat [48].

3.1.1. Attack Situation

This study focused on a WVR air combat scenario consistent with the AlphaDogfight competition settings, thus considering the onboard guns as the threat in the assessment of engagement situations. In WVR air combat utilizing onboard guns, the assessment of the engagement situation is primarily determined by whether the enemy aircraft enters the attack area of the own aircraft’s onboard guns and its specific position within that area. Leveraging the characteristics of the onboard guns, the model for the attack area was constructed as a conical, spherical region in three-dimensional space. The range of the boundaries defining the attack area was obtained through online computation, thereby facilitating the comprehensive analysis and evaluation of the engagement situation, which also met the requirements for the attack range of the guns in AlphaDogfight.

Considering the application of three-dimensional space in WVR air combat, the constructed model of the attack area of onboard guns is illustrated in Figure 11. In this illustration, points A and B, respectively, represent the centroids of the own aircraft and the enemy aircraft. The direction of the central line of the conical sphere aligns with the body axis of the own aircraft $O_b{x}_b$. $\varphi$ denotes the azimuth angle of the enemy aircraft, and R represents the distance between the sides.

The attack situation in WVR air combat is primarily related to the relative position and orientation of both sides. When the enemy aircraft is within the own aircraft’s attack area, the own aircraft can attack and inflict damage on the enemy aircraft effectively; when the enemy aircraft is outside the own aircraft’s attack area, the own aircraft’s onboard guns pose little threat to the enemy aircraft. Based on real-time data on the status of the enemy and the own aircraft, as well as the online calculation of $\varphi$ and R, the attack situation $S_{Attack}$ is constructed as follows:

(3)$\begin{array}{c}{S}_{Attack}=\left\{\begin{array}{cc}1\hfill & \mathrm{if}R\le R_{max}\mathrm{and}\varphi \le {\beta }_{max}\hfill \\ 0\hfill & \mathrm{other}\hfill \end{array}\right.\end{array}$

where $R_{max}$ denotes the maximum attack range of the onboard guns, determined empirically considering flight experience and the attack performance of onboard guns, set at 1.2 km; ${\beta }_{max}$ denotes the conical angle defining the boundary of the attack area, chosen as ${\displaystyle \frac{\pi }{6}}$ based on flight experience and aircraft performance.

3.1.2. Positioning Situation

In WVR air combat, positioning plays a crucial role. The primary objective of positioning is to secure an advantageous geometric position, enabling the creation of optimal attacking conditions. This advantageous geometric positioning allows for pre-emptive attacks ahead of the enemy. In WVR air combat utilizing onboard guns, the positioning situation primarily considers the enemy’s azimuth angle $\varphi$ and aspect angle q.

The smaller the $\varphi$, the smaller the deviation angle of the enemy aircraft from the axis direction of the own aircraft, resulting in a more advantageous positioning situation for the own aircraft. Therefore, the situation of the enemy’s azimuth angle $S_{\varphi }$ is constructed as follows:

(4)$\begin{array}{c}{S}_{\varphi }=\left\{\begin{array}{cc}0\hfill & \mathrm{if}\varphi \ge {\displaystyle \frac{\pi }{2}}\hfill \\ {\displaystyle \frac{1}{2}}-{\displaystyle \frac{\varphi -{\varphi }_s}{2(\frac{\pi }{2}-{\varphi }_s)}}\hfill & \mathrm{if}{\varphi }_s\le \varphi \le {\displaystyle \frac{\pi }{2}}\hfill \\ 1-{\displaystyle \frac{\varphi }{2{\varphi }_s}}\hfill & \mathrm{if}0\le \varphi \le {\varphi }_s\hfill \end{array}\right.\end{array}$

where ${\varphi }_s$ represents the maximum search azimuth angle, taken as ${\displaystyle \frac{\pi }{3}}$ [48].

In WVR air combat utilizing onboard guns, due to the stringent conditions required for onboard gun-firing, firing conditions typically occur only when in a tail-chase position [49]. Thus, the situation of the enemy’s aspect angle $S_q$ is constructed as follows:

(5)$\begin{array}{c}{S}_q=\left\{\begin{array}{cc}0\hfill & \mathrm{if}q\ge {\displaystyle \frac{\pi }{2}}\hfill \\ 2^{{\displaystyle \frac{\frac{\pi }{2}-q}{\frac{\pi }{2}}}}-1\hfill & \mathrm{if}0\le q<{\displaystyle \frac{\pi }{2}}\hfill \end{array}\right.\end{array}$

Considering the coupling relationship between the enemy’s azimuth angle and aspect angle [50], the positioning situation $S_{Place}$ is constructed as follows:

(6)$\begin{array}{c}{S}_{Place}={\omega }_{\varphi }{S}_{\varphi }+{\omega }_q{S}_q\left\{\begin{array}{cc}{\omega }_{\varphi }=0.75,{\omega }_q=0.25\hfill & \mathrm{if}0\le \varphi \le {\varphi }_s\hfill \\ {\omega }_{\varphi }=0.50,{\omega }_q=0.50\hfill & \mathrm{if}{\varphi }_s\le \varphi \le {\displaystyle \frac{\pi }{2}}\hfill \\ {\omega }_{\varphi }=0.40,{\omega }_q=0.60\hfill & \mathrm{if}{\displaystyle \frac{\pi }{2}}\le \varphi \le \pi \hfill \end{array}\right.\end{array}$

where ${\omega }_{\varphi }$ and ${\omega }_q$ are the weights of the enemy’s azimuth angle and aspect angle, respectively, satisfying ${\omega }_{\varphi }+{\omega }_q=1$.

3.1.3. Maneuvering Situation

The aircraft’s maneuverability is primarily demonstrated by its ability to change its orientation, speed, and altitude. In WVR air combat, the aircraft’s maneuverability directly affects the execution of subsequent tasks, including tail pursuit, gun firing, and defensive evasion, thereby altering the air combat situation for both sides. Therefore, the maneuvering situation mainly considers the impact of factors such as directional agility, speed, and residual power, leading to the construction of the maneuvering situation as follows:

(7)$\begin{array}{c}{S}_{MVR}={\omega }_{Agility}{S}_{Agility}+{\omega }_v{S}_v+{\omega }_{SEP}{S}_{SEP}\end{array}$

where ${\omega }_{Agility}$, ${\omega }_v$, and ${\omega }_{SEP}$ are the weights of the advantage function for the directional agility, speed, and unit residual power, respectively. Specifically, values of 0.2, 0.6, and 0.2 were assigned, respectively.

Directional agility is a crucial indicator measuring the speed at which the own aircraft nose can be directed toward the enemy aircraft. It can be represented as the time required for the aircraft to turn toward the target direction at the current speed. Directional agility is a key factor in achieving faster alignment of the own aircraft nose toward the enemy aircraft, thus enabling weapons to be fired ahead of the enemy [51]. Therefore, the relative directional agility between the own aircraft and enemy aircraft $R_{Agility}$ is constructed as follows:

(8)$\begin{array}{c}{R}_{Agility}={\displaystyle \frac{\omega (\pi -q)}{{\omega }_e\varphi }}\end{array}$

where $\omega$ and ${\omega }_e$ represent the angular velocities of the own aircraft and the enemy aircraft at the current speed, respectively. Based on the relative directional agility between the own aircraft and the enemy aircraft, the advantage function for the relative directional agility $S_{Agility}$ is constructed as [48]

(9)$\begin{array}{c}{S}_{Agility}=\left\{\begin{array}{cc}0\hfill & \mathrm{if}{R}_{Agility}<1\hfill \\ {\displaystyle \frac{e^{2(R_{Agility}-1)}-1}{e^{2(R_{Agility}-1)}+1}}\hfill & \mathrm{if}{R}_{Agility}\ge 1\hfill \end{array}\right.\end{array}$

In WVR air combat, increasing speed appropriately can allow the aircraft to achieve greater turn rates and smaller turn radii. However, as speed further increases, the aircraft faces issues such as increased energy consumption and decreased turn occupancy capability. Therefore, the optimal speed function $v_d$ is constructed as [52]

(10)$\begin{array}{c}{v}_d=\left\{\begin{array}{cc}v{\left({\displaystyle \frac{R}{R_{max}}}\right)}^3\hfill & \mathrm{if}R\ge R_{max}\hfill \\ v(1-{\left({\displaystyle \frac{R-R_{max}}{R_{max}}}\right)}^3)\hfill & \mathrm{if}R<R_{max}\hfill \end{array}\right.\end{array}$

Considering both the own aircraft speed and the enemy aircraft speed, we construct the advantage function for speed $S_v$ as [53] follows: When $v_d>1.5v_e$,

(11)$\begin{array}{c}{S}_v=\left\{\begin{array}{cc}{e}^{-{\displaystyle \frac{v_d-v}{v_d}}}\hfill & \mathrm{if}v>v_d\hfill \\ 1\hfill & \mathrm{if}1.5v_e<v\le v_d\hfill \\ -0.5+{\displaystyle \frac{v}{v_e}}\hfill & \mathrm{if}0.5v_e<v\le 1.5v_e\hfill \\ 0.1\hfill & \mathrm{if}v\le 0.5v_e\hfill \end{array}\right.\end{array}$

When $0.5v_e<v_d\le 1.5v_e$,

(12)$\begin{array}{c}{S}_v=\left\{\begin{array}{cc}{e}^{-{\displaystyle \frac{v_d-v}{v_d}}}\hfill & \mathrm{if}v>v_d\hfill \\ 0.4({\displaystyle \frac{v}{v_d}}+{\displaystyle \frac{v}{v_e}})\hfill & \mathrm{if}0.5v_e<v\le v_d\hfill \\ 0.1\hfill & \mathrm{if}v\le 0.5v_e\hfill \end{array}\right.\end{array}$

When $v_d\le 0.5v_e$,

(13)$\begin{array}{c}{S}_v=\left\{\begin{array}{cc}{e}^{-{\displaystyle \frac{v_d-v}{v_d}}}\hfill & \mathrm{if}v>1.5v_e\hfill \\ 0.4({\displaystyle \frac{v}{v_d}}+{\displaystyle \frac{v}{v_e}})\hfill & \mathrm{if}0.5v_e<v\le 1.5v_e\hfill \\ 0.1\hfill & \mathrm{if}v\le 0.5v_e\hfill \end{array}\right.\end{array}$

Specific excess power (SEP) can reflect the aircraft’s future maneuvering capability in the vertical direction [54]. In WVR air combat, SEP is advantageous for the aircraft to gain energy after maneuvering and occupy favorable positions, allowing the side with the SEP advantage to more flexibly choose maneuvers. The relative specific excess power between the own aircraft and the enemy aircraft ${SEP}_R$ can be expressed as

(14)$\begin{array}{c}{SEP}_R={\displaystyle \frac{av-a_e{v}_e}{gK}}\end{array}$

where a and $a_e$ represent the accelerations of the own aircraft and the enemy aircraft, respectively; g represents the gravitational acceleration, taken as $9.8$ m/s²; K represents the benchmark advantage reference factor for SEP, taken as 100. Based on ${SEP}_R$, we construct the advantage function for SEP $S_{SEP}$ as

(15)$\begin{array}{c}{S}_{SEP}=\left\{\begin{array}{cc}0\hfill & \mathrm{if}{SEP}_R\le 0\hfill \\ {\displaystyle \frac{e^{{SEP}_R}-e^{-{SEP}_R}}{e^{{SEP}_R}+e^{-{SEP}_R}}}\hfill & \mathrm{if}{SEP}_R>0\hfill \end{array}\right.\end{array}$

3.1.4. Energy Situation

In WVR air combat, the aircraft’s total energy comprises the sum of kinetic energy and potential energy. The side with a total energy advantage typically takes the initiative in WVR air combat [55]. The total energy of aircraft in air combat can be expressed by the following formula:

(16)$\begin{array}{c}E={\displaystyle \frac{1}{2}}m{v}^2+mgH\end{array}$

where m represents the mass of the aircraft, which is usually assumed to remain constant in WVR air combat; V and h, respectively, denote the current speed and altitude of the aircraft. Thus, combining the total energies of the own aircraft and the enemy aircraft, we construct the advantage function for energy situation $S_E$ as

(17)$\begin{array}{c}{S}_E=\left\{\begin{array}{cc}1\hfill & \mathrm{if}k\ge 0\hfill \\ 2^{2k-3}\hfill & \mathrm{if}0.5\le k\le 1.5\hfill \\ 0.5k\hfill & \mathrm{if}0\le k<0.5\hfill \end{array}\right.\end{array}$

where k represents the ratio of the total energy of the own aircraft to the total energy of the enemy aircraft, expressed as

(18)$\begin{array}{c}k={\displaystyle \frac{E}{E_e}}\end{array}$

where E represents the total energy of the own aircraft, and $E_e$ represents the total energy of the enemy aircraft.

3.1.5. Intrinsic Capability Situation

Based on the static performance indicators of the aircraft’s combat capabilities, the inherent capabilities were evaluated using the logarithmic method, as shown in the following formula [56]:

(19)$\begin{array}{c}Cap=c_e{c}_s{c}_o\left[ln\left(\sum _{j=1}^q{c}_{j,f}+1\right)+ln\left(\sum _{i=1}^p{c}_{i,d}\right)+ln{c}_m\right]\end{array}$

where $c_{j,f}$ represents the firepower strike capability parameter of the j-th equipped weapon, and $c_{i,d}$ represents the situational awareness capability parameter of the i-th mounted detection equipment. $c_e$, $c_s$, $c_o$, and $c_m$ denote the parameters for electronic warfare capability, battlefield survivability, and flight control and maneuverability of the aircraft, respectively. Based on the inherent static capability evaluation ratio between both sides in air combat, $Ca{p}_P/Ca{p}_T$, the inherent capability function is defined as

(20)$\begin{array}{c}{S}_{Cap}=\left\{\begin{array}{cc}1\hfill & \mathrm{if}Ca{p}_P/Ca{p}_T\ge 0\hfill \\ 0.75\hfill & \mathrm{if}1<Ca{p}_P/Ca{p}_T<1.5\hfill \\ 0.5k\hfill & \mathrm{if}Ca{p}_P/Ca{p}_T=1\hfill \\ 0.25\hfill & \mathrm{if}0.3<Ca{p}_P/Ca{p}_T<1\hfill \\ 0\hfill & \mathrm{if}0\le Ca{p}_P/Ca{p}_T\le 0.3\hfill \end{array}\right.\end{array}$

Considering that in the simulation and validation of WVR air combat, both aircraft are of the same type and have identical onboard status, situational awareness, and electronic warfare capabilities, as well as the same initial flight altitude and speed at the onset of combat, the inherent static capability evaluation ratio $Ca{p}_P/Ca{p}_T=1$. Therefore, the advantage function for intrinsic capability situation $S_{Cap}$ can be constructed as

(21)$\begin{array}{c}{S}_{Cap}=0.5\end{array}$

3.2. Algorithm for Air Combat Performance Evaluation

In WVR air combat, the situation between the own aircraft and the enemy aircraft changes rapidly, and the importance of different situations varies with changes in the relative situation between the own aircraft and the enemy aircraft. For example, when the own aircraft is in a tail-chase position, having an advantage in attack situations and positioning is beneficial for attacking the enemy aircraft and is more advantageous in surpassing the enemy aircraft in comprehensive situations; however, when the own aircraft is being tail-chased, having an advantage in maneuvering situations helps the own aircraft quickly break away from the pursuit of the enemy aircraft, which has a more significant impact on the comprehensive situation.

Therefore, to accurately assess the relative situation in WVR air combat, it was necessary to first classify the typical patterns of air combat situations and determine the weights of various air combat situation assessment metrics under different typical patterns, thereby constructing a dynamic air combat performance evaluation algorithm.

Since the primary task in WVR air combat is to achieve positional advantage through flying to place the enemy aircraft within the own aircraft’s attack area, this study considered the relative angular differences between the own aircraft and the enemy aircraft and classified air combat situations into 8 typical patterns [48]. These typical patterns are described using the enemy aircraft’s azimuth angle $\varphi$, enemy aircraft’s aspect angle q, and relative orientation flag $flag$, which are defined as follows:

(22)$\begin{array}{c}{S}_i=f(\varphi ,q,flag)(i=1,2,\dots ,8)\end{array}$

where the enemy aircraft’s azimuth angle $\varphi$ and aspect angle q can be calculated based on the velocity vectors of the own aircraft and the enemy aircraft, as well as the relative position vector of the enemy aircraft. The flight plane is defined as the plane containing the enemy aircraft’s relative position vector and the body axis of the own aircraft. When the velocity vectors of the own aircraft and the enemy aircraft are on the same side of the flight plane, $flag>1$; when the velocity vectors of the own aircraft and the enemy aircraft are on opposite sides of the flight plane, $flag\le 1$.

The schematic diagrams and corresponding parameter characteristics of these 8 typical patterns in WVR air combat can be found in Table 4.

Using the BWM in conjunction with expert knowledge [46], the relative importance among typical patterns of different air combat situations is determined. This calculation process is relatively straightforward, ensuring the reliability of the weight calculation results. The steps for calculating the weights of typical patterns of different air combat situations based on the BWM method are shown in Algorithm 1.

To solve for the weight matrix of the 8 typical patterns in air combat situations, as shown in Table 5.

Algorithm 1: Best–Worst Method (BWM)

In the comprehensive assessment process of WVR air combat, when determining the air combat situation at a specific moment, we used the BWM to establish the weights corresponding to the 5 advantage functions for each typical pattern, thereby obtaining the comprehensive air combat situation at that moment. The specific process was as follows: Firstly, determine the types of typical patterns based on the current flight data of both the own and enemy aircraft. Then, obtain the corresponding weights from Table 5, and finally calculate the comprehensive situation values of the own aircraft using Equation (23), where $w_j^i$ represents the weight of the j-th ($j=1,2,\dots ,5$) evaluation index under the i-th typical pattern, and $S_i$ represents the comprehensive air combat situation value of the i-th typical pattern. The natural range of the comprehensive air combat situation is [0,1]; a higher value of S indicates a greater comprehensive advantage for the own aircraft at that moment and a better overall state.

(23)$\begin{array}{c}{S}_i={\omega }_1^i{S}_{Attack}+{\omega }_2^i{S}_{Place}+{\omega }_3^i{S}_{WVR}+{\omega }_4^i{S}_E+{\omega }_5^i{S}_{Cap}(i=1,2,\dots ,8)\end{array}$

3.3. Validation of the Air Combat Performance Evaluation Algorithm

This section validates the effectiveness of the air combat performance evaluation algorithm using air combat data from ACED. WVR air combat data from ACED were selected to calculate the air combat situation assessment metrics for both the own and enemy aircraft, which are then applied to analyze the air combat performance evaluation algorithm. In the selected WVR air combat data, both the own and enemy aircraft maintain straight flight from 0 to 40 s and are oriented toward each other. The trajectories of both the own and enemy aircraft from 40 s until the end of the air combat are shown in Figure 12 (with the projection point of the own aircraft’s position at 40 s onto the $Z=0$ m plane as the coordinate origin).

Based on the data from both the own and enemy aircraft, the assessment metrics of the air combat situation for both sides can be calculated in real time, and the variations in the assessment metrics for both sides are shown in Figure 13. By combining the trajectories of both the own and enemy aircraft in Figure 12 and the variations in the air combat situation assessment metrics in Figure 13, the following observations were made:

In the initial stage, both the own and enemy aircraft adopt head-on flight, resulting in little difference between the attack situation, positioning situation, and energy situation of either side, but the maneuvering situation of the enemy aircraft is slightly better than that of the own aircraft. As the aircraft on both sides gradually approach and enter each other’s attack area, the attack situation of both sides changes from 0 to 1 simultaneously, but the differences in attack situation, positioning situation, and energy situation between both sides are still not significant, while the maneuvering situation of both sides fluctuates due to the maneuvering fluctuations on both sides. As the air combat progresses, the own aircraft quickly executes maneuvers after completing the first head-on encounter, gaining an advantage in positioning situation. However, due to the rapid execution of maneuvers accompanied by rapid energy consumption and speed reduction, the enemy aircraft maintains an advantage in maneuvering and energy situation. Subsequently, the own aircraft obtains advantageous attack positions through positional advantage and gains an advantage in attack situations in the subsequent phases. Finally, the own aircraft successfully utilizes the advantage in attack situation to destroy the enemy aircraft with onboard gun fire, thereby achieving victory in this air combat.

According to the air combat performance evaluation algorithm, the comprehensive situation values of both the own and enemy aircraft are as shown in Figure 14. By combining the comprehensive situation values of both the own and enemy aircraft with flight trajectories, the situation in the WVR air combat was analyzed and divided into six phases as follows:

• Phase 1: The time period is 0 to 22 s. During this period, both the own and enemy aircraft engage in head-on flight, with both aircraft directly pointing toward each other and continuously closing in. There is no significant difference between the own and enemy aircraft in terms of attack situation, positioning situation, or energy situation; due to the pilot’s control characteristics, the enemy aircraft’s maneuvering situation during head-on flight is slightly better than that of the own aircraft. Therefore, the enemy aircraft holds an advantage in the comprehensive situation, but the lead in the comprehensive situation value does not exceed 0.05 at most, and the gap in the comprehensive situation between the both sides is small.

• Phase 2: The time period is 22 to 47 s. At 22 s, both the own and enemy aircraft simultaneously enter each other’s attack area, and the attack situation changes simultaneously. During this period, both the own and enemy aircraft continue to engage in head-on flight, with both aircraft directly pointing toward the enemy aircraft and continuously closing in. There is still no significant difference between the own and enemy aircraft in terms of attack situation, positioning situation, or energy situation; due to the pilot’s control characteristics, the enemy aircraft’s maneuvering situation during head-on flight is slightly better than that of the own aircraft. Due to the change in the attack situation of both sides, the comprehensive situation values of both sides are higher compared to those in Phase 1. Although the enemy aircraft still maintains an advantage in the comprehensive situation, the lead in the comprehensive situation value does not exceed 0.05 at most, and the gap in the comprehensive situation between the both sides remains small.

• Phase 3: From 47 to 56 s, the flight trajectories of both the own and enemy aircraft in this phase are as shown in Figure 15. At 47 s, both the own and enemy aircraft complete the first head-on pass. Subsequently, the own aircraft quickly pulls up to execute an Immelmann Turn maneuver, completing the maneuver at 56 s. During the execution of this maneuver, the own aircraft’s speed and energy rapidly decrease, while the enemy aircraft maintains a higher speed to execute a turn, thus maintaining an advantage in maneuvering situation and energy situation, while also maintaining an advantage in the comprehensive situation.

• Phase 4: The time period is 56 to 70 s, and the flight trajectories of both own and enemy aircraft in this phase are shown in Figure 16. At 56 s, the own aircraft executes an Immelmann Turn maneuver, completes the turn, and points its nose toward the enemy aircraft. During this period, the own aircraft continues to maintain the direction and gradually approaches the enemy aircraft, while the enemy aircraft maintains a higher speed to execute a turn, completing the turn at 70 s, and pointing its nose toward the own aircraft, thus completing the second head-on pass. During this process, the enemy aircraft still maintains an advantage in maneuvering situation and energy situation, while the own aircraft maintains an advantage in attack situation and positioning situation. Due to the higher weights of the attack situation and positioning situation, the own aircraft maintains an advantage in the comprehensive situation.

• Phase 5: The time period is 70 to 82.5 s, and the flight trajectories of both the own and enemy aircraft in this phase are shown in Figure 17. At 70 s, both the own and enemy aircraft complete the second head-on pass. Subsequently, the own aircraft quickly pulls up to execute a turn, completing the turn and pointing its nose toward the enemy aircraft at 82.5 s. From Figure 17, it can be seen that the length of the own aircraft’s flight trajectory is significantly shorter than that of the enemy aircraft, indicating that the own aircraft’s speed and energy are lower during the turn, while the enemy aircraft maintains a higher speed to execute the turn. Therefore, the enemy aircraft maintains an advantage in maneuvering situation and energy situation while also maintaining an advantage in the comprehensive situation; the own aircraft maintains an advantage in positioning situation due to the rapid turn.

• Phase 6: The time period is 82.5 to 99 s, and the flight trajectories of both the own and enemy aircraft in this phase are shown in Figure 18. At 82.5 s, the own aircraft completes the turn and points its nose toward the enemy aircraft, then continues to pursue the enemy aircraft and fires the onboard guns. During this period, the enemy aircraft maintains an advantage in maneuvering situation and energy situation, while the own aircraft maintains an advantage in attack situation and positioning situation. The own aircraft also maintains an advantage in the comprehensive situation. During this process, the own aircraft continues to pursue the enemy aircraft, maintains a close distance, and continuously obtains guns firing windows. Although the enemy aircraft is fast, it cannot escape from the own aircraft’s pursuit and is eventually destroyed at 99 s, resulting in victory for the own aircraft.

The above analysis of the results shows that the introduction of five types of air combat situation evaluation metrics, including attack situation, positioning situation, maneuvering situation, energy situation, and intrinsic capability situation, comprehensively considers the factors influencing the comprehensive situation in WVR air combat. The weights for eight typical patterns of air combat situations are calculated separately, improving the accuracy of the air combat situation evaluation algorithm. The fluctuation in the comprehensive situation values of the own and enemy aircraft in the different phases of WVR air combat conforms to the actual air combat process. The evaluation results accurately quantify the superiority and inferiority of the situations of the own and enemy aircraft, thereby verifying the effectiveness of the proposed air combat situation evaluation algorithm.

4. WVR Air Combat Tests

4.1. Evaluation Metrics

This section introduces the evaluation metrics for WVR air combat, which aimed at accurately assessing pilot performance in WVR air combat. We subsequently conducted simulation verification to validate the effectiveness of the BFM decision model. In WVR air combat, although the winning rate is crucial for evaluating pilot performance, it does not fully represent their comprehensive air combat performance. In order to more accurately assess the feasibility and effectiveness of the BFM decision model, three additional evaluation metrics are proposed based on the air combat performance evaluation algorithm and actual air combat performance to quantify pilot performance.

The evaluation metrics for WVR air combat include winning rate $P_w$, average percentage of tactical advantage time $\overline{E}$, and average percentage of situation advantage time $\overline{E_S}$. Among these metrics, the winning rate reflects the final outcome of WVR air combat, while the average percentage of tactical advantage time represents the proportion of time spent maintaining a tactical advantage position in WVR air combat. These two indicators are primarily judged from the perspective of the combat outcome, but the actual performance during the engagement is also crucial. In Section 3, this paper proposes an evaluation framework for air combat that describes the state of aircraft at any moment during the engagement. Based on the air combat performance evaluation algorithm, we introduce a calculation for the average percentage of situational advantage time, representing both the aircraft’s overall status and the relative overall status of the enemy aircraft.

The specific calculation steps for winning rate $P_w$ and the average percentage of tactical advantage time $\overline{E}$ are detailed in [12]. The specific calculation steps for the average percentage of situation advantage time $\overline{E_S}$ are as follows:

The verification of the effectiveness of the air combat performance evaluation algorithm indicates that when the comprehensive situation value leads by more than 0.2, the leading side is in a clearly advantageous state. Therefore, states where the comprehensive situation value leads by more than 0.2 are defined as comprehensive situation advantage states. Thus, the average percentage of situation advantage time $\overline{E_S}$ can be expressed as follows:

(24)$\begin{array}{c}\overline{E_s}={\displaystyle \frac{\frac{{\sum }_{i=1}^N{t}_{S,i}}{T_i}}{N}}(i=1,2,\dots ,8)\end{array}$

where N represents the total number of sets of WVR air combat experiments; i represents the i-th group of WVR air combat experiments; $T_i$ represents the total duration of WVR air combat for the i-th group; $t_{S,i}$ represents the time a comprehensive situation advantage is maintained in the i-th group of WVR air combat; ${\displaystyle \frac{t_{S,i}}{T_i}}$ represents the percentage of situation advantage time in the i-th group of WVR air combat experiments.

4.2. Experiments: 1vs1 Air Combat Tests

To assess the effectiveness of the BFM decision model aimed at BFM decision-making, we conducted 3 sets of 1vs1 WVR air combat experiments, validated within the simulation framework adopted in ACED. Table 6 presents the basic settings of the 3 sets. Each set involved pilot C flying the own aircraft and pilot B flying the enemy aircraft, resulting in a total of 20 engagements across WVR air combat scenarios, and the tasks and characteristics of the pilots are shown in Table 1.

In set 1, both pilots directly maneuvered the aircraft in the WVR based on their own flight experience without the aid of any decision model. In set 2, the own pilot utilized the LSTM-based BFM decision model for WVR air combat assistance, while the enemy pilot engaged in direct piloting. Similarly, in set 3, the own pilot employed the Transformer-based BFM decision model for WVR air combat assistance, with the enemy pilot engaging in direct piloting. Following preliminary analysis of the results from ACED [12], the aircraft’s BFM decision time window was set to 30 s, with a decision frequency of 1 Hz. Therefore, in sets 2 and 3, the battlefield information of the past 30 s was processed as input data to the BFM-assisted decision model, and the model output the current BFM decision type. Based on the output, the pilots controlled the aircraft to execute the corresponding basic tactical maneuvers. This approach can fully utilize the real-time analysis capability of the BFM model to provide pilots with more accurate tactical recommendations.

4.2.1. Set 1: No BFM Decision Model

In set 1, the own pilot did not utilize the BFM decision model for BFM decision-making assistance. The total duration of all 20 WVR air combats was 56.58 min. The experimental results revealed that the own aircraft achieved a 30% winning rate, whereas the enemy aircraft attained a 70% winning rate, demonstrating a notable advantage for the enemy aircraft in WVR air combat.

The analysis depicted in Figure 19 illustrates the average percentage of tactical advantage time in set 1. On average, the own aircraft maintained tactical advantage for 4.81% of the time, whereas the enemy aircraft held tactical advantage for an average of 8.52%. Notably, the enemy aircraft consistently maintained a higher percentage of time with a tactical advantage. Within a single experimental group, the enemy aircraft’s tactical advantage time peaked at 30%, surpassing the own aircraft’s maximum of 20%. This observation underscores the significant advantage for the enemy aircraft in this regard.

Similarly, the analysis of the average percentage of situational advantage time in set 1, as depicted in Figure 20, revealed that the own aircraft maintained situational advantage for an average of 17.83% of the time, whereas the enemy aircraft maintained situational advantage for an average of 23.35% of the time. The enemy aircraft consistently had a higher percentage of time with a situational advantage.

4.2.2. Set 2: LSTM-Based BFM Decision Model

In set 2, the own pilot used the LSTM-based BFM decision model for BFM decision-making assistance. The total duration of all 20 WVR air combats was 51.12 min. The experimental results revealed that the own aircraft achieved a 55% winning rate, whereas the enemy aircraft attained a 45% winning rate, demonstrating a certain advantage for the own aircraft in WVR air combat.

The analysis of the percentage of time with a tactical advantage for set 2 is illustrated in Figure 21. The own aircraft maintained a tactical advantage for an average of 10.00% of the time, while the enemy aircraft maintained a tactical advantage for an average of 6.95% of the time. Notably, the own aircraft exhibited a higher percentage of time with a tactical advantage compared to the enemy aircraft.

Similarly, the analysis of the percentage of time with a situational advantage for set 2 is presented in Figure 22. The own aircraft maintained a situational advantage for an average of 19.37% of the time, whereas the enemy aircraft maintained situational advantage for an average of 19.87% of the time. The percentage of time with a situational advantage was comparable between the own and enemy aircraft. In a single experimental group, the own aircraft’s percentage of time with tactical advantage was generally higher than that of the enemy aircraft. In terms of the percentage of time with situational advantage in a single experimental group, the difference between the own aircraft and the enemy aircraft was relatively small, showing a tendency toward consistency.

4.2.3. Set 3: Transformer-Based BFM Decision Model

In set 3, the own pilot used the Transformer-based BFM decision model for BFM decision-making assistance. The total duration of all 20 WVR air combats was 40.47 min. The experimental results revealed that the own aircraft achieved a 70% winning rate, whereas the enemy aircraft attained a 30% winning rate, demonstrating a notable advantage for the own aircraft in WVR air combat.

The analysis depicted in Figure 23 illustrates the average percentage of tactical advantage time in set 3. On average, the own aircraft maintained a tactical advantage for 14.73% of the time, whereas the enemy aircraft held a tactical advantage for an average of 6.61%. The own aircraft consistently maintained a significantly higher percentage of time with a tactical advantage.

Similarly, the analysis of the average percentage of situational advantage time in S = set 3, as depicted in Figure 24, revealsedthat the own aircraft maintained situational advantage for an average of 25.19% of the time, whereas the enemy aircraft maintained situational advantage for an average of 17.04% of the time. The own aircraft consistently held a higher percentage of time with a situational advantage.

In a single experimental group, the percentage of time with a tactical advantage for the own aircraft was significantly higher than that of the enemy aircraft. Similarly, the percentage of time with a situational advantage for the own aircraft was also generally higher compared to that of the enemy aircraft. This analysis underscores the consistent superiority of the own aircraft in maintaining both tactical and situational advantages.

4.2.4. Comparative Analysis

The results of the three sets of 1vs1 WVR air combat experiments are shown in Table 7. For WVR air combat, the three evaluation metrics used were winning rate $P_w$, average percentage of tactical advantage time $\overline{E}$, and average percentage of situation advantage time $\overline{E_S}$. The superior performance of the own aircraft in WVR air combat is indicated by higher values in these metrics for the own aircraft and lower values for the enemy aircraft.

The results in Table 7 indicate that the BFM decision model effectively enhanced the performance of pilots in WVR air combat. The LSTM-based BFM decision model increased the own aircraft’s winning rate from 30% to 55%, while the Transformer-based BFM decision model boosted it to 70%.

Based on the network training results, the accuracy of LSTM on the test data was 91.08%, and Transformer achieved 97.31%. Both BFM decision models achieved test accuracies exceeding 90%, suitable for assisting pilots in WVR air combat decisions. The experiments demonstrated that both the LSTM and Transformer-based BFM decision models improved the pilots’ performance in WVR air combat. These results indicate that employing BFM decision models for BFM decision-making assistance during WVR air combat is effective, enhancing the efficiency and quality of pilot’s decisions.

Using the LSTM-based BFM decision model to assist pilots in air combat transforms the own side from having a significant disadvantage (30% winning rate) to a slight advantage (55% winning rate), notably enhancing combat performance. However, the Transformer-based BFM decision model, with its higher test accuracy, is more likely to make reasonable BFM decisions in complex WVR air combat environments, thus significantly outperforming the LSTM-based BFM decision model in improving air combat performance. Moreover, employing the Transformer-based BFM decision model shifts the winning rate from disadvantage to advantage in 1vs1 WVR air combat between pilots and shifts the tactical and situational advantage time from disadvantage to advantage. In contrast, when using the LSTM-based BFM decision model, the own situational advantage time remains in a disadvantaged state.

These results demonstrate that BFM decision models provide effective support for inexperienced pilots, assisting them in making rational BFM decisions and thereby enhancing their performance in WVR air combat. Particularly, through the use of Transformer-based BFM decision models, the efficiency and quality of BFM decision-making are effectively enhanced.

5. Conclusions

In this paper, a Transformer-based basic fighter maneuver (BFM) decision-support model was proposed to address the limitations of the existing long short-term memory (LSTM)-based models. The Transformer-based BFM decision model achieved an average accuracy of 97.31%, surpassing that of the LSTM-based BFM decision model. Subsequently, a Shapley Additive Explanation (SHAP) analysis was employed to enhance the interpretability of the Transformer-based BFM decision model. The results indicate that the decision-making process and outcomes of the Transformer-based BFM decision model are rational and can be utilized to aid pilots in BFM decision making.

Furthermore, this study introduced five metrics to evaluate air combat performance and established a comprehensive air combat performance evaluation framework. Through validation using data from air combat engagement database (ACED), this framework demonstrated the capability to accurately assess air combat performance. Subsequently, three evaluation metrics derived from the air combat performance evaluation framework and actual air combat performance were utilized in within-visual-range (WVR) air combat experiments. The experimental results clearly indicated that throughout the aerial combat engagement process, the decision-making time of the BFM model was significantly less than 1 s (not exceeding 0.006 s), meeting the immediacy requirements of aerial combat. Moreover, compared to the LSTM-based model, the win rate improved from 30% to 55%, while the Transformer-based model further increased the win rate from 30% to 70%, achieving a more substantial improvement in combat outcomes. Additionally, the Transformer-based model increased the average percentage of tactical advantage time from 4.81% to 14.73% and the average situational advantage time share from 17.83% to 25.19%, extremely improving air combat superiority. The simulation results further validate that the Transformer-based BFM decision-support scheme for piloting WVR air combat is effective.

However, the pilot data collected in this study were derived from commercial flight simulation software. While the software demonstrated a high degree of realism in simulating flight dynamics, avionics systems, and fire control systems, applying the model to actual air combat scenarios may still lead to reduced accuracy. Therefore, a future study could focus on training the model with more authentic pilot data to further improve the model’s applicability to actual scenarios.

Author Contributions

Conceptualization, Y.D.; software, C.W. and S.H.; resources, J.A.; data curation, C.W.; writing—original draft, S.H.; writing—review and editing, Y.Z. and Y.D. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

This study was conducted in a simulated environment with professional fighter pilots participating voluntarily, and it did not involve sensitive personal data or impact participants’ welfare. Hence, it was granted an exemption from formal ethical approval by the Ethics Review Committee of Fudan University.

Informed Consent Statement

Informed consent was obtained from all subjects involved in this study.

Data Availability Statement

The data presented in this study are available upon request from the corresponding author.

Conflicts of Interest

I hereby declare that there are no conflicts of interest associated with this work and the funders had no role in the design of this study; in the collection, analyses, or interpretation of data; in the writing of this manuscript; or in the decision to publish the results.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. Hardware configurations for establishing the ACED [12].

Figure 2. BFMs collected in the ACED [12]: (a) Break Turn, (b) Split-S, (c) Immelman Turn, (d) High-Yo-Yo, (e) Low-Yo-Yo, (f) Pure Pursuit.

Figure 3. The structure of the Transformer.

Figure 4. Test accuracy of Transformer model with different hyperparameters.

Figure 5. The evolution of test accuracy and loss values for the final Transformer model.

Figure 6. Confusion matrix of the final Transformer model.

Figure 7. Feature importance of different features.

Figure 8. Summary plots for Break Turn and Split-S.

Figure 9. Summary plots for Immelman Turn and High-Yo-Yo.

Figure 10. Summary plots for Low-Yo-Yo and Pure Pursuit.

Figure 11. Modeling the attack area for onboard guns.

Figure 12. Flight trajectories of both own and enemy aircraft in WVR air combat in ACED (blue represents the own aircraft; red represents the enemy aircraft).

Figure 13. Variation of assessment metrics for both own and enemy aircraft in WVR air combat.

Figure 14. Variation of comprehensive situation values for both own and enemy aircraft in WVR air combat.

Figure 15. Flight trajectories of both own and enemy aircraft in Phase 3 (blue represents the own aircraft, and red represents the enemy aircraft).

Figure 16. Flight trajectories of both own and enemy aircraft in Phase 4 (blue represents the own aircraft, and red represents the enemy aircraft).

Figure 17. Flight trajectories of both own and enemy aircraft in Phase 5 (blue represents the own aircraft, and red represents the enemy aircraft).

Figure 18. Flight trajectories of both own and enemy aircraft in Phase 6 (blue represents the own aircraft, and red represents the enemy aircraft).

Figure 19. Analysis of results: average percentage of tactical advantage time in set 1.

Figure 20. Analysis of results: average percentage of situation advantage time in set 1.

Figure 21. Analysis of results: average percentage of tactical advantage time in set 2.

Figure 22. Analysis of results: average percentage of situation advantage time in set 2.

Figure 23. Analysis of results: average percentage of tactical advantage time in set 3.

Figure 24. Analysis of results: average percentage of situation advantage time in set 3.

Table 1

Tasks and characteristics of pilots [12].

Symbol	Air Combat Level	Tasks	Air Combat Capabilities
A	High	Blue in the ACED	15 years of flight simulation experience, able to defeat in-service pilots in simulated WVR air combat
B	Medium	Red in the ACED and red in WVR air combat tests	5 years of flight simulation experience, comparable capability to in-service pilots in simulated WVR air combat
C	Novice	Blue in WVR air combat tests	1 year of flight simulation experience

Table 2

Tasks and characteristics of pilots [12].

Feature (a)	Description	Feature (b)	Description
${\psi }_o$	Own aircraft’s yaw angle	$v_c$	Closure velocity
${\theta }_o$	Own aircraft’s pitch angle	$x_r$	Relative position on x-axis
${\varphi }_o$	Own aircraft’s roll angle	$y_r$	Relative position on y-axis
$N{y}_o$	Own aircraft’s normal overload	$z_r$	Relative position on z-axis
$v_o$	Own aircraft’s velocity	${\psi }_r$	Relative yaw angle
${\alpha }_o$	Own aircraft’s angle of attack	${\theta }_r$	Relative pitch angle
${\beta }_o$	Own aircraft’s side slip angle	${\varphi }_r$	Relative roll angle
$H_o$	Own aircraft’s flight altitude

Table 3

Hyperparameters of the Transformer model.

Hyperparameter	Value
Learning rate	$1\times {10}^{-4}$
Optimizer	Adam
Loss function	Cross-entropy loss
Batch size	64
Training epochs	500
Heads (h)	8
Network layers (N)	4
Hidden layers ($d_{model}$)	32

Table 4

Typical patterns and their parameter characteristics in WVR air combat.

Typical Patterns	Parameter Characteristics	Schematic Diagram	Typical Patterns	Parameter Characteristics	Schematic Diagram
$S_1$	$\varphi \in [0,{\displaystyle \frac{\pi }{2}})$ $q\in [0,{\displaystyle \frac{\pi }{2}})$ $flag<0$	[Image omitted. Please see PDF.]	$S_5$	$\varphi \in [{\displaystyle \frac{\pi }{2}},\pi )$ $q\in [0,{\displaystyle \frac{\pi }{2}})$ $flag<0$	[Image omitted. Please see PDF.]
$S_2$	$\varphi \in [0,{\displaystyle \frac{\pi }{2}})$ $q\in [0,{\displaystyle \frac{\pi }{2}})$ $flag\ge 0$	[Image omitted. Please see PDF.]	$S_6$	$\varphi \in [{\displaystyle \frac{\pi }{2}},\pi )$ $q\in [0,{\displaystyle \frac{\pi }{2}})$ $flag\ge 0$	[Image omitted. Please see PDF.]
$S_3$	$\varphi \in [0,{\displaystyle \frac{\pi }{2}})$ $q\in [{\displaystyle \frac{\pi }{2}},\pi )$ $flag<0$	[Image omitted. Please see PDF.]	$S_7$	$\varphi \in [{\displaystyle \frac{\pi }{2}},\pi )$ $q\in [{\displaystyle \frac{\pi }{2}},\pi )$ $flag<0$	[Image omitted. Please see PDF.]
$S_4$	$\varphi \in [0,{\displaystyle \frac{\pi }{2}})$ $q\in [{\displaystyle \frac{\pi }{2}},\pi )$ $flag\ge 0$	[Image omitted. Please see PDF.]	$S_8$	$\varphi \in [{\displaystyle \frac{\pi }{2}},\pi )$ $q\in [{\displaystyle \frac{\pi }{2}},\pi )$ $flag\ge 0$	[Image omitted. Please see PDF.]

Table 5

Weight of typical patterns in air combat situations.

TypicalPatterns	Weight of Typical Patterns
	Attack Situation	Positioning Situation	Maneuvering Situation	Energy Situation	Intrinsic Capability Situation
$S_1$	0.452	0.128	0.187	0.105	0.128
$S_2$	0.428	0.143	0.172	0.086	0.171
$S_3$	0.145	0.478	0.146	0.105	0.126
$S_4$	0.127	0.465	0.186	0.093	0.129
$S_5$	0.091	0.115	0.452	0.161	0.181
$S_6$	0.113	0.149	0.398	0.170	0.170
$S_7$	0.065	0.116	0.424	0.185	0.210
$S_8$	0.082	0.096	0.423	0.184	0.215

Table 6

Basic settings of the 3 sets of WVR air combat experiments.

Experimental Set	Own Pilot	Enemy Pilot	Number of WVR Air Combat	Employment of BFM Decision Model by Own Pilot
1	C	B	20	No BFM Decision Model
2	C	B	20	LSTM-based BFM Decision Model
3	C	B	20	Transformer-based BFM Decision Model

Table 7

Results of experiments in 1vs1 air combat tests.

Experimental Set	Total Time (min)	${\mathit{P}}_{\mathit{w}}$ (%)		$\overline{\mathit{E}}$ (%)		$\overline{{\mathit{E}}_{\mathit{s}}}$ (%)
		Own	Enemy	Own	Enemy	Own	Enemy
1	56.58	30	70	4.81	8.52	17.83	23.35
2	51.12	55	45	10.00	6.95	19.37	19.87
3	40.47	70	30	14.73	6.61	25.19	17.04

References

1. Stillion, J.; Perdue, S. Air Combat Past, Present and Future. Rand, Project Air Force; Technical Report RAND: Santa Monica, CA, USA, 2008.

2. Zhang, G.; Hu, B.; Liu, X.; Jie, X. Influences of Electronic Interference on the Effectiveness of Beyond Visual Range Air Combat for Combat Aircraft. Electron. Opt. Control; 2011; 18, pp. 88-95.

3. Shaw, R.L. Fighter Combat; Naval Institute Press: Annapolis, MD, USA, 1985.

4. Nichols, S.O. 21st Century Air-to-Air Short Range Weapon Requirements; Technical Report Air Command and Staff College, Air University, Maxwell Air Force Base: Montgomery, AL, USA, 1998.

5. Pope, A.P.; Ide, J.S.; Mićović, D.; Diaz, H.; Rosenbluth, D.; Ritholtz, L.; Twedt, J.C.; Walker, T.T.; Alcedo, K.; Javorsek, D. Hierarchical Reinforcement Learning for Air-to-Air Combat. Proceedings of the 2021 International Conference on Unmanned Aircraft Systems (ICUAS); Athens, Greece, 15–18 June 2021; pp. 275-284. [DOI: https://dx.doi.org/10.1109/ICUAS51884.2021.9476700]

6. Shin, H.; Lee, J.; Kim, H.; Shim, D.H. An autonomous aerial combat framework for two-on-two engagements based on basic fighter maneuvers. Aerosp. Sci. Technol.; 2018; 72, pp. 305-315. [DOI: https://dx.doi.org/10.1016/j.ast.2017.11.014]

7. Dean, J.G. Autonomous Guidance of Unmanned Combat Air Vehicles in Basic Fighter Maneuvering. Mech. Aerosp. Eng. Diss.; 2021; 302.

8. An, X.; Gao, C.; Kou, Y.; Li, Z.; Li, Y. Autonomous air combat decision of 1vs1 based on BFM method. Syst. Eng. Electron.; 2020; 42, 2513.

9. Karli, M.; Efe, M.Ö.; Sever, H. Extracting basic fighter maneuvers from actual flight data. Lect. Notes Inf. Theory Vol.; 2017; 5, pp. 1-6. [DOI: https://dx.doi.org/10.18178/lnit.5.1.1-6]

10. McGrew, J.S.; How, J.P.; Williams, B.; Roy, N. Air-combat strategy using approximate dynamic programming. J. Guid. Control Dyn.; 2010; 33, pp. 1641-1654. [DOI: https://dx.doi.org/10.2514/1.46815]

11. Wang, Z.; Li, H.; Wu, H.; Wu, Z. Improving maneuver strategy in air combat by alternate freeze games with a deep reinforcement learning algorithm. Math. Probl. Eng.; 2020; 2020, 7180639. [DOI: https://dx.doi.org/10.1155/2020/7180639]

12. Wang, C.; Tu, J.; Yang, X.; Yao, J.; Xue, T.; Ma, J.; Zhang, Y.; Ai, J.; Dong, Y. Explainable Basic-Fighter-Maneuver Decision Support Scheme for Piloting Within-Visual-Range Air Combat. J. Aerosp. Inf. Syst.; 2024; 21, pp. 500-514. [DOI: https://dx.doi.org/10.2514/1.I011388]

13. Fan, H.; Liu, D. Development trends of short-range dogfight IR air to air missile. Infrared Laser Eng.; 2005; 34, 564.

14. Tan, M.; Zhu, W.; Liu, Y. UCAV escape maneuver decision based on maneuver library. Unmanned Syst. Technol.; 2020; 3, pp. 73-82.

15. Ernest, N.; Carroll, D.; Bogart, N.; Cohen, K. Perspectives on Genetic Fuzzy Based Artificial Intelligence for Cooperative Control of Unmanned Fighter Aircraft. Proceedings of the 2017 World Congress on Unmanned Systems Engineering; Cambridge, UK, 27–28 July 2017.

16. Ernest, N.; Cohen, K.; Kivelevitch, E.; Schumacher, C.; Casbeer, D. Genetic fuzzy trees and their application towards autonomous training and control of a squadron of unmanned combat aerial vehicles. Unmanned Syst.; 2015; 3, pp. 185-204. [DOI: https://dx.doi.org/10.1142/S2301385015500120]

17. Ernest, N.; Cohen, K.; Schumacher, C.; Casbeer, D. Learning of Intelligent Controllers for Autonomous Unmanned Combat Aerial Vehicles by Genetic Cascading Fuzzy Methods; Technical Report SAE Technical Paper: Warrendale, PA, USA, 2014.

18. Dong, Y.; Ai, J. Trial input method and own-aircraft state prediction in autonomous air combat. J. Aircr.; 2012; 49, pp. 947-954. [DOI: https://dx.doi.org/10.2514/1.C031671]

19. Dong, Y.; Huang, J.; Ai, J. Visual perception-based target aircraft movement prediction for autonomous air combat. J. Aircr.; 2015; 52, pp. 538-552. [DOI: https://dx.doi.org/10.2514/1.C032764]

20. Dong, Y. Deep learning-based opponent aircraft attitude detection in autonomous air combat. J. Aerosp. Inf. Syst.; 2019; 16, pp. 162-167. [DOI: https://dx.doi.org/10.2514/1.I010684]

21. Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. Adv. Neural Inf. Process. Syst.; 2017; 30, pp. 1-11.

22. Zhou, B.; Khosla, A.; Lapedriza, A.; Oliva, A.; Torralba, A. Learning Deep Features for Discriminative Localization. Proc. IEEE Conf. Comput. Vis. Pattern Recognit.; 2016; 2016, pp. 2921-2929.

23. Lundberg, S. A unified approach to interpreting model predictions. arXiv; 2017; arXiv: 1705.07874

24. Moujahid, H.; Cherradi, B.; Al-Sarem, M.; Bahatti, L.; Eljialy, A.B.A.M.Y.; Alsaeedi, A.; Saeed, F. Combining CNN and Grad-Cam for COVID-19 Disease Prediction and Visual Explanation. Intell. Autom. Soft Comput.; 2022; 32, pp. 723-745. [DOI: https://dx.doi.org/10.32604/iasc.2022.022179]

25. Hu, Z.; Ma, X.; Liu, Z.; Hovy, E.; Xing, E. Harnessing deep neural networks with logic rules. arXiv; 2016; arXiv: 1603.06318

26. Ummah, K.; Setiadi, H.; Pasaribu, H.M.; Anandito, D. A simple fight decision support system for BVR air combat using fuzzy logic algorithm. Int. J. Aviation Sci. Eng.; 2020; 1, pp. 1-5. [DOI: https://dx.doi.org/10.47355/avia.v1i1.8]

27. Zhou, X.; Wen, H.; Li, Z.; Zhang, H.; Zhang, W. An interpretable model for the susceptibility of rainfall-induced shallow landslides based on SHAP and XGBoost. Geocarto Int.; 2022; 37, pp. 13419-13450. [DOI: https://dx.doi.org/10.1080/10106049.2022.2076928]

28. Zheng, Q.; Wang, Z.; Zhou, J.; Lu, J. Shap-CAM: Visual Explanations for Convolutional Neural Networks Based on Shapley Value. Eur. Conf. Comput. Vis. Proc.; 2022; 2022, pp. 459-474.

29. Russakovsky, O.; Deng, J.; Su, H.; Krause, J.; Satheesh, S.; Ma, S.; Huang, Z.; Karpathy, A.; Khosla, A.; Bernstein, M. et al. Imagenet large scale visual recognition challenge. Int. J. Comput. Vis.; 2015; 115, pp. 211-252. [DOI: https://dx.doi.org/10.1007/s11263-015-0816-y]

30. Xu, X.; Yang, R.; Fu, Y. Situation Assessment for Air Combat Based on Novel Semi-Supervised Naive Bayes. J. Syst. Eng. Electron.; 2018; 29, pp. 768-779.

31. Yang, A.W.; Li, Z.W.; Xu, A.; Xi, Z.F.; Chang, Y.Z. Threat Level Assessment of the Air Combat Target Based on Weighted Cloud Dynamic Bayesian Network. Flight Dyn.; 2020; 38, pp. 87-94.

32. Liu, M.; Guo, Y.; Long, J.; Wei, X.; Wang, X. Target Threat Assessment Method Based on Modified Weighting-Cloud Reasoning. China Autom. Congr. Proc.; 2023; 2023, pp. 1199-1204.

33. Wu, A.; Yang, R.; Liang, X.; Zhang, J.; Qi, D.; Wang, N. Visual range maneuver decision of unmanned combat aerial vehicle based on fuzzy reasoning. Int. J. Fuzzy Syst.; 2022; 24, pp. 519-536. [DOI: https://dx.doi.org/10.1007/s40815-021-01158-y]

34. Shi, Z.; Liang, X.; Zhang, J.; Liu, L. Situation assessment for air combat based on missile attack zone. Fire Control Command. Control; 2018; 43, pp. 89-93.

35. Zhao, K.; Huang, C. Air Combat Situation Assessment for UAV Based on Improved Decision Tree. Chin. Control Decis. Conf. Proc.; 2018; 2018, pp. 1772-1776.

36. Gu, J.; Liu, W.; Jiang, W. WVR air combat situation assessment model based on weapon engagement zone and kill probability. Syst. Eng. Electron.; 2015; 37, pp. 1306-1312.

37. Zhang, Y. Threat measurement and sequencing of air raid targets based on entropy method fused with AHP. J. Telemetry Track. Command; 2020; 41, pp. 57-64.

38. Fang, C.; Kou, Y.; Xu, A.; Deng, S.; Peng, M. A VIKOR method for threat assessment in air combat based on AHP-CRITIC combination weighting. Electro-Opt. Control; 2021; 28, pp. 24-28.

39. Li, H.; Liu, B.; Song, R. Air Attack Target Threat Assessment Based on Combination Weighting. Int. J. Adv. Netw. Monit. Control; 2022; 7, pp. 92-99. [DOI: https://dx.doi.org/10.2478/ijanmc-2022-0020]

40. Qu, S.; Yang, Z.; Xu, T.; Liu, J. Improved TOPSIS Based on IAHP and Entropy for Threat Assessment. J. Phys. Conf. Ser.; 2020; 1624, 032015. [DOI: https://dx.doi.org/10.1088/1742-6596/1624/3/032015]

41. Gong, H.; Yu, X.; Zhang, Y.; Liu, F. Dynamic Threat Assessment of Air Multi-Target Based on DBN-TOPSIS Method. China Autom. Congr. Proc.; 2021; 2021, pp. 6902-6907.

42. Yin, Y.; Zhang, R.; Su, Q. Threat assessment of aerial targets based on improved GRA-TOPSIS method and three-way decisions. Math. Biosci. Eng; 2023; 20, pp. 13250-13266. [DOI: https://dx.doi.org/10.3934/mbe.2023591]

43. Yang, C.; Wang, Q.; Zhu, J.; Li, A. Evaluation Method of Radio Frequency Stealth Performance of Aircraft Swarm in Air Combat Mode. Syst. Eng. Electron.; 2020; 42, 2811.

44. Torğul, B.; Demiralay, E.; Paksoy, T. Training aircraft selection for department of flight training in fuzzy environment. Decis. Mak. Appl. Manag. Eng.; 2022; 5, pp. 264-289. [DOI: https://dx.doi.org/10.31181/dmame0311022022t]

45. Bongo, M.F.; Seva, R.R. Evaluating the performance-shaping factors of air traffic controllers using fuzzy DEMATEL and fuzzy BWM approach. Aerospace; 2023; 10, 252. [DOI: https://dx.doi.org/10.3390/aerospace10030252]

46. Rezaei, J. Best-worst multi-criteria decision-making method. Omega; 2015; 53, pp. 49-57. [DOI: https://dx.doi.org/10.1016/j.omega.2014.11.009]

47. Ma, Y.; Ma, X.; Song, X. A case study on air combat decision using approximated dynamic programming. Math. Probl. Eng.; 2014; 2014, 183401. [DOI: https://dx.doi.org/10.1155/2014/183401]

48. Ma, J.; Bi, W.; Zhang, A.; Lan, Y.; Tang, C. Close-range Air Combat Situation Assessment Based on Fuzzy Dynamic Weight. Control and Decision; 2023; 39, pp. 2995-3005.

49. Du, H.; Cui, M.; Han, T.; Wei, Z.; Tang, C.; Tian, Y. Air Combat Maneuver Decision Making Based on Multi-Objective Optimization and Reinforcement Learning. J. Beijing Univ. Aeronaut. Astronaut.; 2018; 44, pp. 2247-2256. [DOI: https://dx.doi.org/10.13700/j.bh.1001-5965.2018.0132]

50. He, B.; Bai, L.; Wen, P. One-on-One Air Combat Control Method Based on Situation Assessment and DDPG Algorithm. Adv. Aeronaut. Sci. Eng.; 2024; 15, pp. 179-187.

51. Sun, C. Evolution of Air Combat Winning Mechanism and Future Trends of Fighter Aircraft Development. Acta Aeronaut. Et Astronaut. Sin.; 2021; 42, pp. 8-20.

52. Wang, Y.; Ding, D.; Zhang, P.; Xie, L.; Zhang, X. Research on Adaptive Situation Assessment Method for UCAV Close Air-combat. Unmanned Syst. Technol.; 2023; 6, pp. 85-94. [DOI: https://dx.doi.org/10.19942/j.issn.2096-5915.2023.04.41]

53. Li, Y.; Han, T.; Sun, C.; Wei, Z. An Optimization Method of Air Combat Situation Assessment Function Based on Inverse Reinforcement Learning. Fire Control Command. Control; 2019; 44, pp. 101-106.

54. Schuck, T.M.; Blasch, E.; Gagne, O.B. Information Maneuverability and the Transformation of the Warfighting Environment. Proceedings of the 2021 IEEE Aerospace Conference (50100); Big Sky, MT, USA, 6–13 March 2021; pp. 1-7.

55. Zhang, H.; Li, G.; Ding, Q.; Kong, F. Research on Situation Assessment in BVR Air Combat. Electr. Opt. Control; 2010; 17, pp. 9-13.

56. Yang, A.; Li, Z.; Li, B.; Xi, Z.; Gao, C. Air Combat Situation Assessment Based on Dynamic Variable Weight. Acta Armamentarii; 2021; 42, 1553.