Adaptive Navigation Performance Evaluation Method for Civil Aircraft Navigation Systems with Unknown Time-Varying Sensor Noise

Abstract

During civil aviation flights, the aircraft needs to accurately monitor the real-time navigation capability and determine whether the onboard navigation system performance meets the required navigation performance (RNP). The airborne flight management system (FMS) uses actual navigation performance (ANP) to quantitatively calculate the uncertainty of aircraft position estimation, and its evaluation accuracy is highly dependent on the position estimation covariance matrix (PECM) provided by the airborne integrated navigation system. This paper proposed an adaptive PECM estimation method based on variational Bayes (VB) to solve the problem of ANP misevaluation, which is caused by the traditional simple ANP model failing to accurately estimate PECM under unknown time-varying noise. Combined with the 3D ANP model proposed in this paper, the accuracy of ANP evaluation can be significantly improved. This enhancement contributes to ensured navigation integrity and operational safety during civil flight.

1. Introduction

The implementation of performance-based navigation (PBN) of civil aircrafts in recent years has greatly enhanced the capacity of civil airspace. With the development of airborne navigation systems, especially global navigation satellite systems (GNSSs), the implementation of required navigation performance (RNP) with higher performance requirements has become a trend for future civil aviation air traffic management [1,2,3,4].

RNP is an advanced civil aviation navigation procedure that defines the essential capability of the airborne navigation system in the form of a performance threshold tunnel when operating in specified airspace. The flight management system (FMS) requires fault detection, fault isolation, and system reconfiguration capabilities, and it uses hardware redundancy and model-based analytic redundancy methods for accurate fault identification [5,6,7]. The FMS also needs to accurately evaluate a civil aircraft’s navigation capability in real-time to determine if it meets the current RNP threshold. An incorrect evaluation result will cause misjudgement by the crew, which will lead to serious threats to flight safety.

The FMS uses the probability circle radius (95%) of estimated position uncertainty (EPU) to represent the actual navigation performance (ANP) [8]. The existing ANP evaluation method focuses on the improvement of the EPU model [9,10,11]. As shown in Figure 1, the evaluation accuracy of ANP depends on the precision of position uncertainty estimation provided by the FMS.

Airborne integrated navigation systems usually use empirical positioning error models of each navigation sensor for position calculation to ensure computational efficiency [12]. However, the performance of the primary navigation source during civil flight is time-varying and unpredictable [13]. In this case, the use of an empirical error model will lead to inaccurate calculation of the position error covariance matrix (PECM) by the integrated navigation system, which in turn leads to inaccuracy of the ANP value [14].

Accurate estimation of the time-varying PECM is a prerequisite for ANP calculations. The exact solution of the time-varying PECM caused by inaccurate and unknown noise cannot be obtained in-flight, and it is more common to estimate the PECM using approximation methods. There are numerous approximation methods, including the maximum likelihood method, maximum a posterior method, expectation–maximization method, variational Bayesian methods, etc. [15]. Variational Bayesian (VB) methods are a combination of Bayesian reasoning and variational calculus. They can perform approximate posterior inference at a low computational cost by choosing an appropriate conjugate prior distribution [16,17]. However, the existing VB-based estimator can estimate an unknown and slowly varying measurement noise by choosing the appropriate conjugate prior distribution [18]. But when the estimator uses an inaccurate process noise matrix, it will introduce inaccuracy in the PECM estimation; therefore, the tracking capability for time-varying unknown system noise needs to be improved.

This paper aims to solve the ANP misestimation cost due to unknown time-varying sensor noise. An adaptive 3D ANP evaluation method based on variational Bayesian was proposed. The main contributions of this paper are as follows: (1) A novel VB-based PECM estimator was constructed to accurately estimate both the predicted error covariance matrix and the measurement noise matrix in integrated navigation systems. This approach effectively addresses the issue of inaccurate estimation caused by unknown noise in the integrated navigation system input. (2) The fading memory index weighting is used to improve the tracking capability of time-varying noise, and the accuracy of the actual navigation performance of the integrated navigation system is further improved. (3) To address the limitations of the traditional ANP model, a 3D ANP model was developed that was specifically designed to overcome issues such as missing evaluation dimensions and model inaccuracy. Finally, a simulation was performed to prove the algorithm.

The structure of the proposed airborne integrated navigation system and ANP evaluation is shown in Figure 2. The FMS uses the inertial reference system (IRS) as the reference system and the GNSS and land-based radio system as auxiliary systems. The accurate PECM can be obtained using the proposed VB-based estimator, and by applying PECM to the 3D ANP model, the FMS can output a 3D-ANP evaluation result for civil aircrafts during RNP flight.

2. Traditional Airborne ANP Evaluation with Kalman Filter

The traditional airborne ANP evaluation is based on the Kalman filter. The discrete-time linear state-space model of the airborne integrated system is as follows:

(1)$\left\{\begin{array}{c}{X}_k={\Phi }_{k,k-1}{X}_{k-1}+W_{k-1}\\ Z_k=H_k{X}_k+V_k\end{array}\right.$

Gaussian system noise $W_{k-1}~N(0,Q_{k-1})$ and Gaussian measurement noise $V_k~N(0,R_k)$ are irrelevant.

The process of integrated navigation based on the Kalman filter is as follows:

• (1). Time update:

  (2)${\widehat{X}}_{k|k-1}={\Phi }_{k,k-1}{\widehat{X}}_{k-1|k-1}$

• (2). Measurement update:

  (3)$P_{k|k-1}={\Phi }_{k/k-1}{P}_{k-1}{\Phi }_{k/k-1}^T+Q_{k-1}$

  (4)$K_k=P_{k|k-1}{H_k}^T{(H_k{P}_{k|k-1}{H_k}^T+R_k)}^{-1}$

  (5)$P_{k|k}=(I-K_k{H}_k)P_{k|k-1}{(I-K_k{H}_k)}^T+K_k{R}_k{K}_k^T$

$P_{k|k}$ is the estimated error matrix (EEM) of the Kalman filter, which includes the position estimate error (PEE) of the airborne integrated navigation system.$P_{k|k-1}$ is the one-step prediction error matrix. FMS uses the PECM obtained according to $P_{k|k}$ as the ANP calculation input. PECM $\Lambda$ for the horizontal plane can be expressed as follows:

(6)$\Lambda =A{\Lambda }_m{A}^{-1}$

where:

(7)${\Lambda }_m=\left[\begin{array}{cc}{\sigma }_{mx}^2& 0\\ 0& {\sigma }_{my}^2\end{array}\right]$

$A$ is the similarity transformation matrix of $\Lambda$. Then, the ANP in the horizontal plane can be expressed as follows:

(8)$\mathrm{ANP}=\mathrm{k}\times {\mathrm{l}}_{\mathrm{major}}$

${\mathrm{l}}_{\mathrm{major}}=\max [{\sigma }_{mx},{\sigma }_{my}]$ is the major axis of the PEE ellipses, and k is the scale factor of the 95% probability elliptic integral:

(9)$k={\displaystyle \frac{0.4852}{rati{o}^3}}+1.9625$

The Kalman filter is the optimal recursive linear variance estimation algorithm with accurate $Q_k$ and $R_k$ values. However, active FMS empirically sets $Q_k$ and $R_k$ according to the sensor error characteristics. As shown in Figure 3, such customized parameters cannot reflect the changing characteristics of uncertain noise [19]. The use of inaccurate $Q_k$ and $R_k$ values will result in substantial estimation errors in the PECM, leading to misevaluation of ANP. Therefore, a novel VB-based PECM approximation method will be proposed.

3. Adaptive Navigation Performance Evaluation Method Based on Adaptive PECM Estimation

3.1. PECM Approximation Based on VB with Unknown Time-Varying Noise

As shown in Figure 4, to obtain a 3D ANP safety boundary, the PECM of the airborne integrated navigation system with uncertain and time-varying noise needs to be estimated more accurately. The variational Bayesian approximation is used to determine $X_k$, $R_k$, and $P_{k|k-1}$ to obtain an accurate estimation of $P_{k|k}$.

$X_k$, $R_k$, and $P_{k|k-1}$ are independent, and the variational approximation of joint posterior PDF is as follows:

(10)$p(X_k,P_{k|k-1},R_k|Z_{1:k})\approx q(X_k)q(P_{k|k-1})q(R_k)$

The posterior PDF of $p(·)$ is represented within $q(·)$. We use the Kullback–Leibler (KL) divergence to describe the approach degree of the approximate posterior distribution and true posterior distribution.

(11)$\begin{array}{l}{D}_{KL}[p(X_k,P_{k|k-1},R_k|Z_{1:k})||q(X_k)q(P_{k|k-1})q(R_k)]=\\ {\displaystyle \int {\displaystyle \int {\displaystyle \int q(X_k)q(P_{k|k-1})q(R_k)\log \frac{q(X_k)q(P_{k|k-1})q(R_k)}{p(X_k,P_{k|k-1},R_k|Z_{1:k})}d{X}_{k}d{P}_{k|k-1}d{R}_k}}}\end{array}$

$q(X_k),q(P_{k|k-1}),q(R_k)$ can be acquired by minimizing the KL divergence $D_{KL}$.

(12)$[q(X_k),q(P_{k|k-1}),q(R_k)]=\arg \min (D_{KL})$

Taking the derivative of $q(X_k),q(P_{k|k-1}),q(R_k)$ results in the following:

(13)$q(X_k)\propto {\displaystyle \int q(X_k)}\log p(Z_k,X_k,P_{k|k-1},R_k|Z_{1:k})d{X}_k$

(14)$q(P_{k|k-1})\propto {\displaystyle \int q(X_k)}\log p(Z_k,X_k,P_{k|k-1},R_k|Z_{1:k})d{P}_{k|k-1}$

(15)$q(R_k)\propto {\displaystyle \int q(R_k)}\log p(Z_k,X_k,P_{k|k-1},R_k|Z_{1:k})d{R}_k$

The above Equations (13) to (15) cannot be directly solved analytically due to mutual coupling. By calculating the expectation of these equations, we can obtain the following:

(16)$\log q(\delta )=E_{{\Psi }^{-\delta }}[\log p(\Psi ,Z_{1:k})]+C_{\delta }$

Within $\Psi \triangleq (X_k,P_{k|k-1},R_k)$, $E(·)$ represents the expectation calculation, $\delta$ is an arbitrary element of $\Psi$, ${\Psi }^{-\delta }$ indicates the set of elements of $\Psi$ except for $\delta$, and $C_{\delta }$ is a constant term associated with $\delta$.

For a linear system, the PDF of $p(X_k|Z_{1:k},P_{k|k-1})$, $p(Z_k|X_k,R_k)$, $P_{k|k-1}$, and $R_k$ are Gaussian. $N(·;\mu ,\sigma )$ denotes a Gaussian distribution with a mean of $\mu$ and a covariance of $\sigma$.

(17)$p(X_k|Z_{1:k},P_{k|k-1})=N(X_k;{\widehat{X}}_{k|k-1},P_{k|k-1})$

(18)$p(Z_k|X_k,R_k)=N(Z_k;H_k{X}_k,R_k)$

In variational Bayesian approximation, the inverse Wishart distribution is usually used as the conjugate distribution of the Gaussian distribution to reduce the iterative computation. Therefore, the prior distribution of $P_{k|k-1}$ and $R_k$ is selected as the inverse Wishart distribution to derive the posterior distribution of $X_k$. $IW(·;\theta ,\omega )$ denotes an inverse Wishart distribution with degrees of freedom (DOF) $\theta$ and scale matrix $\omega$.

(19)$p(P_{k|k-1}|Z_{1:k-1})=IW(P_{k|k-1};y_{k|k-1},Y_{k|k-1})$

(20)$p(R_k|Z_{1:k-1})=IW(R_k;u_{k|k-1},U_{k|k-1})$

The joint posterior PDF Equation (10) can be rewritten as follows:

(21)$p(X_k,P_{k|k-1},R_k|Z_{1:k})=p(Z_k|X_k,R_k)p(X_k|Z_{1:k},P_{k|k-1})p(P_{k|k-1}|Z_{1:k})p(Z_{1:k})$

By substituting Equations (19) and (20) into Equation (21), we obtain the following:

(22)$\begin{array}{c}p(X_k,P_{k|k-1},R_k|Z_{1:k})=N(Z_k;h(X_k),R_k)N(X_k;{\widehat{X}}_{k|k-1},P_{k|k-1})\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\times IW(P_{k|k-1};y_{k|k-1},Y_{k|k-1})IW(R_k;u_{k|k-1},U_{k|k-1})p(Z_{1:k-1})\end{array}$

The $\log p(\Psi ,Z_{1:k})$ in Equation (16) can be derived as follows:

(23)$\begin{array}{ll}\log p(\Psi ,Z_{1:k})& =-0.5\log \left|R_k\right|-0.5{(Z_k-H_k{X}_k)}^T{R}_k^{-1}(Z_k-H_k{X}_k)\\ & -0.5\log \left|P_{k|k-1}\right|-0.5{(X_k-{\widehat{X}}_{k|k-1})}^T{P}_{k|k-1}^{-1}(X_k-{\widehat{X}}_{k|k-1})\\ & -0.5(m+u_{k|k-1}+1)\log \left|R_k\right|-tr(U_{k|k-1}{R}_k^{-1})\\ & -0.5(n+y_{k|k-1}+1)\log \left|P_{k|k-1}\right|-tr(Y_{k|k-1}{P}_{k|k-1}^{-1})+C_{\psi }\end{array}$

where n is the dimension of $X_k$, and m is the dimension of $Z_k$.

Then, an iterative approximation of $q(X_k),q(P_{k|k-1}),q(R_k)$ can be obtained. Let $\Psi =P_{k|k-1}$, deploying Equations (23) to (16):

(24)$\begin{array}{ll}\log q^{(i+1)}(P_{k|k-1})& =-0.5(n+y_{k|k-1}^{(i)}+2)\log \left|P_{k|k-1}\right|\\ & -tr\left[(Y_{k|k-1}^{(i)}+M_k^{(i)})P_{k|k-1}^{-1}\right]+C_P\end{array}$

(25)$M_k^{(i)}=(X_k-{\widehat{X}}_{k|k-1}^{(i)}){(X_k-{\widehat{X}}_{k|k-1}^{(i)})}^T$

After i + 1 iterative operations, $q(P_{k|k-1})$ can be updated as follows:

(26)$q^{(i+1)}(P_{k|k-1})=IW(P_{k|k-1};y_{k|k-1}^{(i)},Y_{k|k-1}^{(i)})$

(27)$y_{k|k-1}^{(i+1)}=y_{k|k-1}^{(i)}+1$

(28)$Y_{k|k-1}^{(i+1)}=Y_{k|k-1}^{(i)}+M_k^{(i)}$

Similarly, let $\Psi =R_k$:

(29)$\begin{array}{ll}\log q^{(i+1)}(R_k)& =-0.5(m+u_{k|k-1}^{(i)}+2)\log \left|R_k\right|\\ & -tr\left[(U_{k|k-1}^{(i)}+A_k^{(i)})R_k^{-1}\right]+C_R\end{array}$

(30)$A_k^{(i)}=(Z_k-H_k{X}_k^{(i)}){(Z_k-H_k{X}_k^{(i)})}^T$

The $q(R_k)$ value can be updated as follows:

(31)$q^{(i+1)}(R_k)=IW(R_k;u_k^{(i)},U_k^{(i)})$

(32)$u_k^{(i+1)}=u_k^{(i)}+1$

(33)$U_k^{(i+1)}=U_k^{(i)}+A_k^{(i)}$

Let $\Psi =X_k$:

(34)$\begin{array}{ll}\log q^{(i+1)}(X_k)& =-0.5{(Z_k-h(X_k^{(i)}))}^T{[R_k^{-1}]}^{(i+1)}(Z_k-h(X_k^{(i)}))\\ & -0.5{(X_k-{\widehat{X}}_{k|k-1}^{(i)})}^T{[P_{k|k-1}^{-1}]}^{(i+1)}(X_k-{\widehat{X}}_{k|k-1}^{(i)})+C_X\end{array}$

where:

(35)${[R_k^{-1}]}^{(i+1)}=(u_k^{(i+1)}-m-1){[U_k^{(i+1)}]}^{-1}$

(36)${[P_{k|k-1}^{-1}]}^{(i+1)}=(y_k^{(i+1)}-n-1){[Y_k^{(i+1)}]}^{-1}$

The $q(X_k)$ value can be updated as follows:

(37)$\begin{array}{ll}{q}^{(i+1)}(X_k)& ={\displaystyle \frac{1}{C_X}}{p}^{(i+1)}(Z_k|X_k)p^{(i+1)}(X_k|Z_{1:k-1})\\ & =N(X_k;{\widehat{X}}_{k|k}^{(i+1)},P_{k|k}^{(i+1)})\end{array}$

(38)$C_X={\displaystyle \int p^{(i+1)}(Z_k|X_k)p^{(i+1)}(X_k|Z_{1:k-1})}d{X}_k$

Deploying Equations (2) to (5), the $P_{k|k}^{(i+1)}$ after iterative operations can be obtained as follows:

(39)$K_k^{(i+1)}=P_{k|k-1}^{(i+1)}{H}_k^T{(H_k{P}_{k|k-1}^{(i+1)}{H}_k^T+R_k^{(i+1)})}^{-1}$

(40)$P_{k|k}^{(i+1)}=(I-K_k{H}_k)P_{k|k-1}^{(i+1)}{(I-K_k{H}_k)}^T+K_k{R}_k^{(i+1)}{K}_k^T$

After all, the PECM $\Lambda$ can be obtained from $P_{k|k}^{(i+1)}$.

To achieve timely adjustment of the posterior approximation, a fading memory index weighting is used in this paper:

(41)${\displaystyle \sum _{i=1}^k{\beta }_i}=1,{\beta }_i={\beta }_{i-1}b,(0<b<1)$

(42)$\left\{\begin{array}{c}{d}_k={\displaystyle \frac{1-b}{1-b^k}}\\ {\beta }_k=d_k{b}^{i-1}\end{array}\right.$

where b is the forgetting factor, usually takes a value of between 0.95 and 0.99. The previous approximate posteriors are spread through the fading memory index weighting, and the prior parameters of $q(P_{k|k-1})$, $q(R_k)$ can be given as follows:

(43)$\left\{\begin{array}{c}{y}_{k|k-1}=(1-{\beta }_k)(y_{k-1|k-1}-n-1)+n+1\\ Y_{k|k-1}=(1-{\beta }_k)Y_{k-1|k-1}\end{array}\right.$

(44)$\left\{\begin{array}{c}{u}_{k|k-1}=(1-d_k)(u_{k-1|k-1}-m-1)+m+1\\ U_{k|k-1}=(1-d_k)U_{k-1|k-1}\end{array}\right.$

After N iterations, the VB approximations of the expected posterior PDFs are obtained:

(45)$\left\{\begin{array}{c}q(P_{k|k-1})\approx q^{(N)}(P_{k|k-1})=IW(P_{k|k-1};y_{k|k-1}^{(N)},Y_{k|k-1}^{(N)})\\ q(R_k)\approx q^{(N)}(R_k)=IW(R_k;u_k^{(N)},U_k^{(N)})\\ q(X_k)\approx q^{(N)}(X_k)=N(X_k;{\widehat{X}}_{k|k}^{(N)},P_{k|k}^{(N)})\end{array}\right.$

3.2. Three-Dimensional ANP Evaluation Model

The traditional ANP evaluation model mainly focuses on the aircraft navigation performance along the track. Based on the assumption of three-dimensional independence and uniformity, the traditional model can only perform approximate calculations, and the evaluation results are not accurate. A 3D ANP evaluation model is proposed to accurately calculate the ANP in both horizontal and vertical planes.

PECM ${\Lambda }_{3D}$ for civil aircrafts in 3D space can be expressed as follows:

(46)${\Lambda }_{3D}=A{\Lambda }_{3Dm}{A}^{-1}$

where:

(47)${\Lambda }_{3Dm}=\left[\begin{array}{ccc}{\sigma }_{mx}^2& 0& 0\\ 0& {\sigma }_{my}^2& 0\\ 0& 0& {\sigma }_{mz}^2\end{array}\right]$

The probability P that the real position falls into the three-dimensional normal distribution integral ellipsoid domain is as follows:

(48)$P={\displaystyle \underset{\Omega }{\iiint }\frac{1}{{(2\pi )}^{3/2}{\left(\det {\Lambda }_{3Dm}\right)}^{1/2}}\exp \left(-\frac{1}{2}{E}^T{\Lambda }_{3Dm}^{-1}E\right)}$

For a given probability, the ANP calculation process can be converted into the calculation of the three-axis length of the integral ellipsoid Ω.

The probability integration can be simplified to the following:

(49)$P=2[\phi (w)-{\displaystyle \frac{w}{{(2\pi )}^{1/2}}}\exp (-{\displaystyle \frac{w^2}{2}})]-1$

where $w=R/{\sigma }_{mx}$, and $\phi (w)$ is the standard normal distribution function. For a given allowable error $\xi$, the ANP can be obtained via iterative numerical integration.

The flowchart of the proposed adaptive VB-based 3D ANP evaluation method is shown in Figure 5. The main process includes (1) the time update; (2) the initialization of VB iterative parameters based on fading exponential factors; (3) the approximation of $P_{k|k-1},R_k,X_k$, which was obtained through N-times VB iterations; (4) the 3D ANP calculation based on the PECM.

4. Experimental Setup

To verify the effectiveness of the algorithm proposed in this paper, the performance of the following algorithms is compared, including the traditional ANP evaluation method based on KF with a nominal $R_k$ and $Q_k$ noise matrix(KFNN), traditional ANP evaluation method based on KF with a true $R_k$ and $Q_k$ noise matrix(KFTN), ANP evaluation method based on VBKF for estimating only $R_k$ (VBKFR), and the proposed 3D ANP evaluation method based on the VB-PECM estimator for estimating $R_k$ and $Q_k$.

The integrated navigation system loosely adopts the IRS/GNSS position combination. The simulation experiments are carried out in the MATLAB environment. The standard deviation of the GNSS positioning error for each leg is set as follows: (1) 10 m (take off, landing), (2) 20 m (departure, climb, descent, approach), (3) 50 m (en-route). The bias of airborne accelerometers is set as 10⁻⁴ g, and the bias of gyros is set as 0.01°/h [20,21,22].

The RNP flight procedure from Shanghai Hongqiao International Airport (ZSSS) to Chengdu Shuangliu International Airport (ZUUU) is executed in the simulation. The aircraft’s flight trajectory is dynamic, taking full account of the aircraft’s maneuverability, comprising five phases: takeoff, climb, steady flight, turn, descent and landing. The initial position is 31.1967° N, 121.335° E at an altitude of 8 m; the initial heading angle is 175°; and the arrival airport is 30.58° N, 103.948° E at an altitude is 495 m. The simulation duration is 10,505 s. The fly track simulation output is shown in Figure 6.

5. Results

Figure 7 shows the comparison results of the position-estimated error of the FMS in the east, north, and vertical directions of the different algorithms.

To evaluate the position estimate accuracy of the proposed algorithm, the root mean square error (RMSE) of position error is chosen as an evaluation metric. The formula for calculating RMSE is as follows:

(50)$\mathrm{RMSE}=\sqrt{{\displaystyle \frac{1}{{\mathrm{N}}_{\mathrm{S}}}}{\displaystyle \sum _{\mathrm{i}=1}^{{\mathrm{N}}_{\mathrm{S}}}{({\widehat{\mathit{x}}}_{\mathrm{i}}-{\overline{\mathit{x}}}_{\mathrm{i}}^{\mathrm{true}})}^2}}$

where ${\mathrm{N}}_{\mathrm{S}}$ is the number of data points, $\widehat{\mathit{x}}$ is the position estimated value of FMS, and ${\overline{\mathit{x}}}_{\mathrm{i}}^{\mathrm{true}}$ is the position true value of FMS obtained from the simulated track generator.

Table 1 shows the position error RMSE of each algorithm in three directions.

Table 2 and Table 3 show the mean and standard deviation of the position error for each algorithm in three directions.

As can be seen from the position error curve and the statistical table of position errors, the proposed VB-PECM method can significantly improve the precision of integrated navigation. The addition of forgetting factor makes the tracking performance of measurement noise better, reducing the influence of time-varying noise on the navigation results.

The comparison of the traditional ANP method with KFNN, the traditional ANP method with VB-PECM, and the proposed 3D ANP method with VB-PECM that was proposed in this paper is shown in Figure 8.

To compare the success rate and precision of each ANP evaluation method, the F-score and mean absolute percentage error (MAPE) statistics are introduced as the evaluation metrics.

(51)$\left\{\begin{array}{c}F-Score=(1+\tau ){\displaystyle \frac{\mathrm{Pr}ecision\cdot \mathrm{Re}call}{{\tau }^2\cdot \mathrm{Pr}ecision+\mathrm{Re}call}}\\ \mathrm{Pr}ecision={\displaystyle \frac{TP}{TP+FP}}\\ \mathrm{Re}call={\displaystyle \frac{TP}{TP+FN}}\end{array}\right.$

The F-score represents the comprehensive efficiency of the ANP evaluation, TP is the number of correct evaluations, FP is the number of misevaluations, FN is the number of missed evaluations, and $\tau$ is the harmonization factor of precision and recall. When $\tau =1$, the F-score can fully represent the accuracy of each ANP model.

MAPE denotes the approximability of ANP to the true position error.

(52)$MAPE={\displaystyle \frac{1}{n}}{\displaystyle \sum _{t=1}^n\left|\frac{E_t-A_t}{E_t}\right|}$

where n is the number of data points, $E_t$ is the true position error of FMS, and $A_t$ is the ANP results in each direction.

As can be seen, because the traditional ANP with KFNN uses a nominal $R_k$ and $Q_k$ noise matrix, the ANP evaluation result is a smooth curve, which has a large error gap compared to the real navigation positioning error. The traditional ANP with VB-PECM and the proposed ANP with VB-PECM accurately perceived the time-varying unknown noise in the integrated navigation system due to the PECM estimator. The ANP model proposed in this paper further improved the accuracy of the ANP evaluation. Table 4 and Table 5 show the F-1score and MAPE of each ANP method in three directions.

Table 6 and Table 7 show the mean and standard deviation of ANP for each algorithm in three directions.

As can be seen from the ANP curve and the statistical table of ANP, the proposed 3D ANP model with VB-PECM has the lowest MAPE and the highest F1-score. This means the results of the ANP algorithm proposed in this paper are closer to the real navigation error, and the 3D model proposed in this paper makes the ANP evaluation in three directions more accurate. The statistical results of the mean and variance of each algorithm also show that the proposed algorithm has a higher performance.

In order to verify the effectiveness of the proposed algorithm, Monte Carlo simulations were carried out under the RNP flight procedure from Beijing Capital Airport (ZBAA) to Shanghai Hongqiao International Airport (ZSSS). The initial position is 40.073333° N, 116.598333° E at an altitude of 30.4 m, the initial heading angle is 175°, and the end point is 31.196667° N, 121.335000° E at an altitude is 8 m. The standard deviation of GNSS positioning error for each leg is set to be the same as the previous simulation conditions. The F1-score and MAPE of each ANP method in three directions are shown in Table 8 and Table 9.

Based on the above simulation results, it can be seen that the proposed VB-PECM filter has stronger adaptability to time-varying unknown noise. The proposed VB-PECM filter not only improves the accuracy of positioning but also the estimation accuracy of the PECM. Combined with the 3D ANP model proposed in this paper, the ANP evaluation is more effective. The evaluation accuracy in the east direction is 60.7% higher than the traditional ANP with KFNN, and it is 38.8% higher than the traditional ANP with VB-PECM. In the north direction, it is 55.8% higher than the traditional ANP with KFNN and 39.7% higher than the traditional ANP with VB-PECM. In the vertical direction, it is 56.9% higher than the traditional ANP with KFNN and 31.7% higher than the traditional ANP with VB-PECM.

6. Conclusions

During RNP operation, it is important to monitor the actual navigation performance of FMS accurately. We aimed to address the problem that the traditional ANP calculation method has regarding nominal noise matrices, where the Kalman filter cannot be sensitive to time-varying unknown noise, which will lead to inaccurate evaluation results.

Firstly, this paper proposed a PECM estimator based on the variational Bayesian method. Accurate estimation of the PECM is achieved via iterative approximation of the predicted error covariance matrix and the measurement noise matrix. Meanwhile, the addition of fading memory index weighting endows the VB-PECM with a better tracking performance for time-varying noise. On this basis, the 3D ANP calculation model is constructed to solve the problem of missing evaluation dimensions and inaccurate evaluation of the traditional model. The simulation results show that the proposed ANP calculation method in this paper can effectively improve the evaluation accuracy of ANP during RNP operation with time-varying unknown noise. Thus, the proposed algorithm has a certain theoretical reference value for ensuring the flight safety of civil aircrafts under RNP operation.

Author Contributions

Conceptualization, Y.D. and J.L.; methodology, Y.D. and J.L.; validation, Y.D. and J.L.; formal analysis, Y.D.; investigation, Z.L., Y.D., Q.Z. and R.L.; data curation, Y.D. and Z.L.; writing—original draft preparation, Y.D.; writing—review and editing, Y.D. and J.L.; visualization, J.L.; supervision, J.L. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding author.

Conflicts of Interest

The authors declare no conflicts of interest.

Abbreviations

RNP	Required navigation performance
ANP	Actual navigation performance
FMS	Flight management system
PECM	Position estimation covariance matrix
IRS	Inertial reference system

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. Schematic diagram of ANP definition.

Figure 2. Proposed airborne integrated navigation and ANP evaluation system.

Figure 3. ANP misevaluation caused by inaccurate PECM.

Figure 4. 3D ANP tunnel during RNP flight.

Figure 5. Three-dimensional ANP evaluation based on adaptive PECM estimation.

Figure 6. Fly track simulation output.

Figure 7. Comparison between navigation positioning errors.

Figure 8. Comparison between different ANP evaluation results.

Table 1

Comparison of position error RMSE of each algorithm.

Position Error (RMSE)	East/m	North/m	Vertical/m
KFNN	11.20	11.95	21.52
KFTN	1.69	1.45	2.09
VBKFR	5.12	4.76	13.45
VB-PECM	2.53	1.83	5.98

Table 2

Comparison of position error mean of each algorithm.

Mean	East/m	North/m	Vertical/m
KFNN	−0.1	3.8	−4.45
KFTN	−0.18	0.73	−0.51
VBKFR	−0.33	1.53	−2.62
VB-PECM	−0.15	0.8	−1.11

Table 3

Comparison of position standard deviation of each algorithm.

Standard Deviation	East/m	North/m	Vertical/m
KFNN	11.2	11.3	21.05
KFTN	1.63	1.25	2.02
VBKFR	5.11	4.58	13.12
VB-PECM	2.53	1.65	5.87

Table 4

Comparison of F1-score of the results of each ANP algorithm.

F1-Score	East	North	Vertical
Traditional ANP with KFNN	80.3%	81.2%	66.3%
Traditional ANP with VB-PECM	96.5%	94.1%	89.8%
Proposed ANP with VB-PECM	99.7%	99.7%	98.9%

Table 5

Comparison of F1-score and MAPE of the results of each ANP algorithm.

MAPE	East/m	North/m	Vertical/m
Traditional ANP with KFNN	7.54	6.79	11.14
Traditional ANP with VB-PECM	4.57	4.99	7.13
Proposed ANP with VB-PECM	2.83	2.93	4.88

Table 6

Comparison of ANP mean of each algorithm.

Mean	East/m	North/m	Vertical/m
Traditional ANP with KFNN	11.36	11.36	16.51
Traditional ANP with VB-PECM	6.62	6.76	10.16
Proposed ANP with VB-PECM	4.92	4.85	6.18

Table 7

Comparison of ANP standard deviation of each algorithm.

Standard Deviation	East/m	North/m	Vertical/m
Traditional ANP with KFNN	0.6	0.6	0.87
Traditional ANP with VB-PECM	0.19	0.20	0.78
Proposed ANP with VB-PECM	0.15	0.15	0.38

Table 8

F1-score comparison of each ANP algorithm results in Monte Carlo simulations.

F1-Score	East	North	Vertical
Traditional ANP with KFNN	77.3%	75.1%	62.3%
Traditional ANP with VB-PECM	90.1%	89.7%	86.3%
Proposed ANP with VB-PECM	96.2%	96.5%	95.8%

Table 9

MAPE comparison of each ANP algorithm results in Monte Carlo simulations.

MAPE	East/m	North/m	Vertical/m
Traditional ANP with KFNN	8.34	7.12	12.37
Traditional ANP with VB-PECM	5.35	5.21	7.81
Proposed ANP with VB-PECM	3.27	3.14	5.33

References

1. International Civil Aviation Organization (ICAO). Doc 9750: 2016–2030 Global Air Navigation Plan; ICAO: Montréal, QC, Canada, 2016.

2. Joint Planning and Development Office (JPDO). Concept of Operations for the Next Generation Air Transportation System; JPDO: Washington, DC, USA, 2013.

3. Civil Aviation Administration of China (CAAC). Statistical Bulletin on the Development of Civil Aviation Industry of 2019; CAAC: Beijing, China, 2020.

4. SESAR Consortium. The ATM Target of Operations; SESAR Consortium: Brussels, Belgium, 2007.

5. Lyu, P.; Wang, B.; Lai, J.; Bai, S. A factor graph optimization method for high-precision IMU based navigation system. IEEE Trans. Instrum. Meas.; 2023; 72, pp. 1-12. [DOI: https://dx.doi.org/10.1109/TIM.2023.3291779]

6. Li, J.; Zhang, G.; Zhang, X.; Zhang, W. Integrating dynamic event-triggered and sensor-tolerant control: Application to USV-UAVs cooperative formation system for maritime parallel search. IEEE Trans. Intell. Transp. Syst.; 2024; 5, pp. 3986-3998. [DOI: https://dx.doi.org/10.1109/TITS.2023.3326271]

7. Zhang, J.; Ma, L.; Zhao, J.; Zhu, Y. Fault Detection and Fault-Tolerant Control for Discrete-Time Multiagent Systems With Sensor Faults: A Data-Driven Method. IEEE Sens. J.; 2024; 14, pp. 22601-22609. [DOI: https://dx.doi.org/10.1109/JSEN.2024.3404006]

8. International Civil Aviation Organization (ICAO). Doc 9613: Performance Based Navigation (PBN) Manual; ICAO: Montréal, QC, Canada, 2013.

9. Chamlou, R. TIS-B: Calculation of navigation accuracy category for position and velocity parameters. Proceedings of the 23rd Digital Avionics Systems Conference; Salt Lake City, UT, USA, 28 October 2004; Computer Science; D.3-1-1.D.3-13

10. Zhang, Q.; Zhang, J.; Zhu, Y.; Liu, W. Navigation Accuracy Category-Position Models and Estimate Position Uncertainty Calculations for TIS-B System. Chin. J. Aeronaut.; 2009; 22, pp. 419-425.

11. Zhao, H.; Xu, X.; Zhang, J.; Zhu, Y.; Yang, C.; Hong, S. Lateral Flight Technical Error Estimation Model for Performance Based Navigation. Chin. J. Aeronaut.; 2011; 24, pp. 329-336. [DOI: https://dx.doi.org/10.1016/S1000-9361(11)60039-3]

12. Pleninger, S.; Hospodka, J.; Pilmannová, T.; Topková, T.; Lukeš, P.; Steiner, J.; Kraus, J. Jamming of GNSS receiver on b737 MAX aircraft and its impact on ADS-b technology. 2020 New Trends in Civil Aviation; IEEE: Prague, Czech Republic, 2020; pp. 123-128.

13. Karasalo, M.; Hu, X. An optimization approach to adaptive Kalman filtering. Automatica; 2011; 47, pp. 1785-1793. [DOI: https://dx.doi.org/10.1016/j.automatica.2011.04.004]

14. Zhang, L.; Wang, S.; Selezneva, M.S.; Neusypin, K.A. A new adaptive Kalman filter for navigation systems of carrier-based aircraft. Chin. J. Aeronaut.; 2022; 35, pp. 416-425. [DOI: https://dx.doi.org/10.1016/j.cja.2021.04.014]

15. Huang, Y.; Zhang, Y.; Wu, Z.; Li, N.; Chambers, J. A novel adaptive Kalman filter with inaccurate process and measurement noise covariance matrices. IEEE Trans. Autom. Control.; 2017; 63, pp. 594-601. [DOI: https://dx.doi.org/10.1109/TAC.2017.2730480]

16. Chang, G.; Chen, C.; Zhang, Q.; Zhang, S. Variational Bayesian adaptation of process noise covariance matrix in Kalman filtering. J. Frankl. Inst.; 2021; 358, pp. 3980-3993. [DOI: https://dx.doi.org/10.1016/j.jfranklin.2021.02.037]

17. He, J.; Sun, C.; Zhang, B.; Wang, P. Variational Bayesian-based maximum correntropy cubature Kalman filter with both adaptivity and robustness. IEEE Sens. J.; 2020; 21, pp. 1982-1992. [DOI: https://dx.doi.org/10.1109/JSEN.2020.3020273]

18. Xu, H.; Duan, K.Q.; Yuan, H.D. Black box variational inference to adaptive Kalman filter with unknown process noise covariance matrix. Signal Process.; 2020; 169, pp. 524-550. [DOI: https://dx.doi.org/10.1016/j.sigpro.2019.107413]

19. FAA Aeronautical Center. Assessing the Validity of Using Actual Navigation Performance (ANP) Information for Supporting Designated Flight Inspection Operations; FAA Aeronautical Center: Oklahoma City, OK, USA, 2008.

20. Radio Technical Commission for Aeronautics (RTCA). RTCA DO-236C, Minimum Aviation System Performance Standards: Re-quired Navigation Performance for Area Navigation; Radio Technical Commission for Aeronautics (RTCA): Washington, DC, USA, 2014.

21. Radio Technical Commission for Aeronautics (RTCA). RTCA DO-229C, Minimum Operational Performance Standards for Global Positioning System/Wide Area Augmentation System Airborne Equipment; Radio Technical Commission for Aeronautics (RTCA): Washington, DC, USA, 2001.

22. Aeronautical Radio Inc. (ARINC). ARINC 712-7, Airborne ADF System; ARINC: Annapolis, MD, USA, 1997.