Low-Complexity Constrained Recursive Kernel Risk-Sensitive Loss Algorithm

Abstract

The constrained recursive maximum correntropy criterion (CRMCC) combats the non-Gaussian noise effectively. However, the performance surface of maximum correntropy criterion (MCC) is highly non-convex, resulting in low accuracy. Inspired by the smooth kernel risk-sensitive loss (KRSL), a novel constrained recursive KRSL (CRKRSL) algorithm is proposed, which shows higher filtering accuracy and lower computational complexity than CRMCC. Meanwhile, a modified update strategy is developed to avoid the instability of CRKRSL in the early iterations. By using Isserlis’s theorem to separate the complex symmetric matrix with fourth-moment variables, the mean square stability condition of CRKRSL is derived, and the simulation results validate its advantages.

1. Introduction

The constrained adaptive filters (CAFs) [1], where the weight is subject to linear constraints, have been widely studied in the field of adaptive signal processing. The original research of CAFs was derived from the antenna array processing, which employed the linearly-constrained minimum-variance (LCMV) criterion to estimation the direction of the antenna array [2]. CAFs have since been successfully applied to adaptive beamforming [3], system identification [4], channel equalization [5], and blind multiuser detection [6].

The simplest linearly-constrained adaptive filter, named constrained least mean-square (CLMS) [2], is developed from the LCMV criterion, and its mean square performance is analyzed based on a decomposable symmetric matrix in [7]. Due to stochastic gradient optimization, the CLMS has a simple structure with low computational complexity. However, its performance is highly influenced by the step size and correlated input. To improve the convergence speed, the constrained fast least-squares (CFLS) algorithm [8], linear-equality-constrained recursive least-squares (CRLS) algorithm [9] and its relaxed version are proposed at the expense of high computational complexity. Furthermore, the reduced-complexity constrained recursive least-squares algorithm based on the dichotomous coordinate descent (CRLS-DCD) iterations [10] and low-complexity constrained affine-projection (CAP) algorithm [11] with the data-selection method are proposed for reducing the computational complexity effectively. Other types of improved constrained filters [12,13,14] have been widely used, which make a trade-off between the computational complexity and filtering performance, i.e., convergence speed and filtering accuracy. All the mentioned constrained algorithms above are developed based on the mean square error (MSE) criterion [15] with good performances under Gaussian assumptions. However, with non-Gaussian cases, the filtering accuracy of these algorithms will decline sharply.

Therefore, the maximum correntropy criterion (MCC) [16,17], generalized MCC (GMCC) [18] and minimum error entropy (MEE) [19] criteria from information theoretic learning (ITL) [20] become alternative criteria, showing strong robustness to non-Gaussian signals. Adding the linear constraints on the weights of MCC and GMCC, the constrained MCC (CMCC) [21] and constrained GMCC (CGMCC) [22] are developed by using stochastic gradient optimization. CMCC and CGMCC display good filtering performances in the presence of single-peak heavy-tailed noise. However, when coping with multi-peak noise, their performances will decline. Due to the symmetry of the errors, MEE counteracts the influence of multi-peak noise effectively. Adding the linear constraints into MEE, a gradient-based constrained MEE (CMEE) algorithm [23] with a sliding window is proposed at the expense of higher complexity but with better accuracy than CMCC and GMCC in the presence of multi-peak noise. Except for the ITL-based constrained filters, there also exist other criteria [24,25] to show good performance under a non-Gaussian environment. A constrained least mean M-estimation (CLMM) algorithm based on an improved M-estimation loss function has been proposed in [24]. Inspired by the boundedness of the gradient of the lncosh function, a constrained least lncosh adaptive filtering algorithm (CLLAF) has been developed in [25]. These constrained algorithms show good performances under different non-Gaussian noise. However, the gradient-based constrained adaptive filters need to make a trade-off between accuracy and convergence speed through an adjustable step size. Hence, the constrained recursive MCC (CRMCC) algorithm [26] is developed, which not only improves the accuracy but also accelerates the convergence.

Recently, an advanced ITL-based criterion, named kernel risk-sensitive loss (KRSL) [27], has been proposed by introducing a beneficial risk-sensitive parameter to regulate the shape of its performance surface. Compared with MCC, the KRSL is more “convex”, which conduces better accuracy and faster convergence. Based on the KRSL, a gradient-based constrained mixture KRSL algorithm [28] is proposed, which indicates a higher filtering accuracy than CMCC.

In this paper, thanks to the advantage of KRSL criterion, a novel constrained recursive KRSL (CRKRSL) algorithm is proposed by using an average approximation method [29]. Thus, the proposed CRKRSL algorithm is converted to a variable step size gradient-based algorithm, which has a lower complexity than the traditional constrained recursive algorithms. Meanwhile, due to the instability of CRKRSL at the initial update stage, we use a gradient algorithm with a fixed step size to replace its initial update. Moreover, the convergence condition of mean square stability regarding iteration times is derived by decomposing a symmetric matrix with fourth-order variables. Simulation results indicate the advantages of CRKRSL on filtering accuracy and computational complexity.

The rest of the paper is organized as follows. The constrained KRSL loss and the CRKRSL algorithm are presented in Section 2. Stability analysis of CRKRSL is given in Section 3. Simulation results and discussion of CRKRSL are shown in Section 4. Finally, the conclusion is given in Section 5.

2. CRKRSL Algorithm

2.1. Notations

Throughout this paper, $\mathbb{R}$ denotes the real field, ${\mathbb{R}}^m$ denotes an m-dimensional real-valued vector space, ${\mathbb{R}}^{n\times m}$ denotes $n\times m$ matrix where its entries belong to $\mathbb{R}$; ${(·)}^{\mathrm{T}}$ represents the transpose operation; $\mathbb{E}[·]$ is the expectation operation; $\parallel ·\parallel$ denotes the Euclidean norm; and $O(·)$ represents the computational complexity of algorithm.

2.2. KRSL Loss

As a nonlinear similarity measure of variable $\mathcal{X},\mathcal{Y}\in \mathbb{R}$, the KRSL [27] is defined as

(1)$\begin{array}{cc}\hfill S(\mathcal{X},\mathcal{Y})& =\frac{1}{\gamma }\mathbb{E}[exp(\frac{\gamma }{2}{\parallel \phi \left(\mathcal{X}\right)-\phi \left(\mathcal{Y}\right)\parallel }^2\left)\right]\hfill \\ \hfill & =\frac{1}{\gamma }\mathbb{E}[exp(\gamma (1-\kappa (\mathcal{X}-\mathcal{Y}))\left)\right],\hfill \end{array}$

where $\gamma \in [0,+\infty )$ is a risk-sensitive parameter. $\phi (·)$ is the corresponding mapping variable in reproducing kernel Hilbert spaces (RKHS) [30], satisfying ${\phi }^{\mathrm{T}}\left(\mathcal{X}\right)\phi \left(\mathcal{Y}\right)=\kappa (\mathcal{X}-\mathcal{Y})$ with Gaussian kernel

(2)$\begin{array}{c}\hfill \kappa (\mathcal{X}-\mathcal{Y})=exp\left(-\frac{{\parallel \mathcal{X}-\mathcal{Y}\parallel }^2}{2{\sigma }^2}\right).\end{array}$

Generally, in terms of the unknown joint distribution between $\mathcal{X}$ and $\mathcal{Y}$, we adopt a finite sampled data pair ${\{{\tilde{x}}_l,{\tilde{y}}_l\}}_{l=1}^N$ to approximate the expected loss, which is given by

(3)$\begin{array}{c}\hfill \widehat{S}(\mathcal{X},\mathcal{Y})=\frac{1}{\gamma N}{\displaystyle \sum _{l=1}^N}exp(\gamma (1-\kappa ({\tilde{x}}_l-{\tilde{y}}_l))).\end{array}$

2.3. Constrained KRSL Loss

When applied to constrained adaptive filtering, the optimization problem with KRLS loss becomes

(4)$\begin{array}{cc}\hfill & {\displaystyle \underset{\mathbf{w}}{min}}\frac{1}{\gamma }{\displaystyle \sum _{l=1}^n}exp(\gamma (1-\kappa (d_l-{\mathbf{w}}^{\mathrm{T}}{\mathbf{u}}_l)))\hfill \\ \hfill & \begin{array}{cc}\hfill s.t.& \hfill {\mathbf{C}}^{\mathrm{T}}\mathbf{w}=\mathbf{f},\end{array}\hfill \end{array}$

where ${\{{\mathbf{u}}_l,d_l\}}_{l=1}^N\in {\mathbb{R}}^m\times \mathbb{R}$ are the input–output training data pair; $\mathbf{w}\in {\mathbb{R}}^m$ is the weight vector and $e_l=d_l-{\mathbf{w}}^{\mathrm{T}}{\mathbf{u}}_l$ is the corresponding error; $\mathbf{C}\in {\mathbb{R}}^{m\times q}$ and $\mathbf{f}\in {\mathbb{R}}^q$ are the constrained matrix and vector, respectively. By constructing the Lagrange function, the constrained problem is transformed to minimize the following constrained KRSL loss:

(5)$\begin{array}{c}\hfill J_c\left(n\right)=\frac{{\sigma }^2}{\gamma }{\displaystyle \sum _{l=1}^n}exp\left(\gamma (1-\kappa (d_l-{\mathbf{w}}^{\mathrm{T}}{\mathbf{u}}_l))\right)+{\mathbf{\theta }}_n^{\mathrm{T}}(\mathbf{f}-{\mathbf{C}}^{\mathrm{T}}\mathbf{w})\end{array}$

with ${\mathbf{\theta }}_n\in {\mathbb{R}}^q$ being the Lagrange multiplier.

2.4. Proposed CRKRSL Algorithm

Taking the gradient of $J_c\left(n\right)$ to zero at instant n, one has

(6)$\begin{array}{c}\hfill \frac{\partial J_c\left(n\right)}{\partial {\mathbf{w}}_{\mathbf{n}}}={\displaystyle \sum _{l=1}^n}\varphi \left(e_l\right)(d_l-{\mathbf{w}}_n^{\mathrm{T}}{\mathbf{u}}_l){\mathbf{u}}_l+\mathbf{C}{\mathbf{\theta }}_n=\mathbf{0}\end{array}$

with $\varphi \left(e_l\right)=exp\left(\gamma (1-\kappa \left(e_l\right))\right)\kappa \left(e_l\right)$.

Define

(7)$\begin{array}{cc}\hfill {\mathbf{U}}_n^{-1}& ={\displaystyle \sum _{l=1}^{n-1}}\varphi \left(e_l\right){\mathbf{u}}_l{\mathbf{u}}_l^{\mathrm{T}}+\varphi \left(e_n\right){\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}\hfill \end{array}$

(8)$\begin{array}{cc}\hfill {\mathbf{d}}_n& ={\displaystyle \sum _{l=1}^{n-1}}\varphi \left(e_l\right){\mathbf{u}}_l{d}_l+\varphi \left(e_n\right){\mathbf{u}}_n{d}_n.\hfill \end{array}$

Then, the constrained solution is derived as

(9)$\begin{array}{c}\hfill {\mathbf{w}}_n={\mathbf{U}}_n{\mathbf{d}}_n+{\mathbf{U}}_n\mathbf{C}{\mathbf{\theta }}_n\end{array}$

with ${\mathbf{\theta }}_n={\left({\mathbf{C}}^{\mathrm{T}}{\mathbf{U}}_n\mathbf{C}\right)}^{-1}(\mathbf{f}-{\mathbf{C}}^{\mathrm{T}}{\mathbf{U}}_n{\mathbf{d}}_n)$. By the matrix inversion lemma [31], Equation (7) is further rewritten by

(10)$\begin{array}{c}\hfill {\mathbf{U}}_n={\mathbf{U}}_{n-1}-{\mathbf{g}}_n{\mathbf{u}}_n^{\mathrm{T}}{\mathbf{U}}_{n-1},\end{array}$

where the gain is given by

(11)$\begin{array}{cc}\hfill {\mathbf{g}}_n& =\frac{{\mathbf{U}}_{n-1}{\mathbf{u}}_n}{{\varphi }^{-1}\left(e_n\right)+{\mathbf{u}}_n^{\mathrm{T}}{\mathbf{U}}_{n-1}{\mathbf{u}}_n}.\hfill \end{array}$

Reorganizing Equation (11), we get another form of ${\mathbf{g}}_n$ by

(12)$\begin{array}{cc}\hfill {\mathbf{g}}_n& =\varphi \left(e_n\right)({\mathbf{U}}_{n-1}{\mathbf{u}}_n-{\mathbf{g}}_n{\mathbf{u}}_n^{\mathrm{T}}{\mathbf{U}}_{n-1}{\mathbf{u}}_n)\hfill \\ \hfill & =\varphi \left(e_n\right){\mathbf{U}}_n{\mathbf{u}}_n.\hfill \end{array}$

To obtain a recursive solution, we expand Equation (9) as follow:

(13)$\begin{array}{cc}\hfill {\mathbf{w}}_n& ={\mathbf{U}}_n({\mathbf{d}}_{n-1}+\varphi \left(e_n\right){\mathbf{u}}_n{d}_n)+{\mathbf{U}}_n\mathbf{C}{\mathbf{\theta }}_n\hfill \\ \hfill & =({\mathbf{U}}_{n-1}-{\mathbf{g}}_n{\mathbf{u}}_n^{\mathrm{T}}{\mathbf{U}}_{n-1}){\mathbf{d}}_{n-1}+\varphi \left(e_n\right){\mathbf{U}}_n{\mathbf{u}}_n{d}_n+{\mathbf{U}}_n\mathbf{C}{\mathbf{\theta }}_n\hfill \\ \hfill & ={\mathbf{w}}_{n-1}+\varphi \left(e_n\right){\mathbf{U}}_n{\mathbf{u}}_n{d}_n-{\mathbf{g}}_n{\mathbf{u}}_n^{\mathrm{T}}{\mathbf{w}}_{n-1}+{\mathbf{U}}_n\mathbf{C}{\mathbf{\theta }}_n\hfill \\ \hfill & ={\mathbf{w}}_{n-1}+\varphi \left(e_n\right)e_n{\mathbf{U}}_n{\mathbf{u}}_n+{\mathbf{U}}_n\mathbf{C}{\mathbf{\theta }}_n,\hfill \end{array}$

where the corrected error $e_n$ updated by the a priori weight ${\mathbf{w}}_{n-1}$ is obtained by

(14)$\begin{array}{cc}\hfill e_n& =d_n-{\mathbf{u}}_n^{\mathrm{T}}{\mathbf{w}}_{n-1}.\hfill \end{array}$

Substituting Equation (12) into Equation (13), we get

(15)$\begin{array}{cc}\hfill {\mathbf{w}}_n& ={\mathbf{w}}_{n-1}+{\mathbf{g}}_n{e}_n+{\mathbf{U}}_n\mathbf{C}{\mathbf{\theta }}_n\hfill \end{array}$

with

(16)$\begin{array}{cc}\hfill {\mathbf{\theta }}_n& ={\left({\mathbf{C}}^{\mathrm{T}}{\mathbf{U}}_n\mathbf{C}\right)}^{-1}(\mathbf{f}-{\mathbf{C}}^{\mathrm{T}}({\mathbf{w}}_{n-1}+{\mathbf{g}}_n{e}_n)).\hfill \end{array}$

Therefore, combing Equations (10), (11), (14) and (15), we obtain the constrained recursive CRKRSL algorithm. The main drawback of Equation (15) is that the inverse matrix ${\left({\mathbf{C}}^{\mathrm{T}}{\mathbf{U}}_n\mathbf{C}\right)}^{-1}$ needs to be updated iteratively with complexity $O\left(q^3\right)$. To reduce the computational complexity of the CRKRSL algorithm, we consider the following linear model and give some assumptions. The linear model is described by

(17)$\begin{array}{c}\hfill d_n={\mathbf{w}}_{*}^{\mathrm{T}}{\mathbf{u}}_n+v_n,\end{array}$

where ${\mathbf{w}}_{*}$ is the model parameter and $v_n$ is the noise at instant n. The assumptions are given as follows:

• A1 $\left\{{\mathbf{u}}_n\right\}$ is independent identically distributed (i.i.d), generated from a multivariate Gaussian distribution with covariance matrix $\mathbf{R}=\mathbb{E}\left[{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}\right]$;

• A2 $\left\{v_n\right\}$ is zero mean, i.i.d, and independent with $\left\{{\mathbf{u}}_n\right\}$, satisfying ${\delta }^2=\mathbb{E}\left[v_n^2\right]$;

• A3 the error $e_n$ is uncorrelated with ${\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}$.

Inspired by the average approximation [29] and based on A1–A3, the correlation matrix ${\mathbf{U}}_n$ is approximated by

(18)$\begin{array}{cc}\hfill {\mathbf{U}}_n& =\frac{1}{n}{\left(\frac{1}{n}{\displaystyle \sum _{l=1}^n}\varphi \left(e_l\right){\mathbf{u}}_l{\mathbf{u}}_l^{\mathrm{T}}\right)}^{-1}\hfill \\ \hfill & \approx \frac{1}{n}\mathbb{E}{\left[\varphi \left(e_n\right)\right]}^{-1}\mathbb{E}{\left[{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}\right]}^{-1}={\eta }_n\mathbf{Z},\hfill \end{array}$

where $\mathbf{Z}={\mathbf{R}}^{-1}=\mathbb{E}{\left[{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}\right]}^{-1}$ and ${\eta }_n={\left(n\mathbb{E}\left[\varphi \left(v_n\right)\right]\right)}^{-1}$ is approximated by the first-order Taylor expansion around the noise.

Furthermore, Equation (13) can be simplified as

(19)$\begin{array}{cc}\hfill {\mathbf{w}}_n& ={\mathbf{w}}_{n-1}+\varphi \left(e_n\right)e_n{\mathbf{U}}_n{\mathbf{u}}_n+{\mathbf{U}}_n\mathbf{C}{\mathbf{\theta }}_n\hfill \\ \hfill & ={\mathbf{w}}_{n-1}+{\eta }_n\varphi \left(e_n\right)e_n\mathbf{Z}{\mathbf{u}}_n+\mathbf{Z}\mathbf{C}\widehat{\mathbf{\theta }},\hfill \end{array}$

where $\widehat{\mathbf{\theta }}=\mathsf{\Theta }(\mathbf{f}-{\mathbf{C}}^{\mathrm{T}}({\mathbf{w}}_{n-1}+{\eta }_n\varphi \left(e_n\right)e_n\mathbf{Z}{\mathbf{u}}_n))$ and the constrained inverse matrix is defined as $\mathsf{\Theta }={\left({\mathbf{C}}^{\mathrm{T}}\mathbf{Z}\mathbf{C}\right)}^{-1}$.

Therefore, ${\mathbf{w}}_n$ is further expressed as

(20)$\begin{array}{c}\hfill {\mathbf{w}}_n=\mathbf{Q}({\mathbf{w}}_{n-1}+{\eta }_n\varphi \left(e_n\right)e_n\mathbf{Z}{\mathbf{u}}_n)+\mathbf{p}\end{array}$

with $\mathbf{Q}=\mathbf{I}-\mathbf{Z}\mathbf{C}\mathsf{\Theta }{\mathbf{C}}^{\mathrm{T}}$ and $\mathbf{p}=\mathbf{Z}\mathbf{C}\mathsf{\Theta }\mathbf{f}$.

Based on the approximation, the recursive CRKRSL algorithm has been changed to a gradient one with variable step size ${\eta }_n$ and transformed input $\mathbf{Z}{\mathbf{u}}_n$.

Remark 1.

The term $\varphi \left(e_n\right)$ in Equation (20) has a significant impact on the stability of the CRKRSL algorithm under non-Gaussian noise since CRKRSL can suppress large outliers ($e_n\to \infty$) with a small $\varphi \left(e_n\right)$. Figure 1 shows the relation between $\varphi \left(e_n\right)$ and the error $e_n$. It is clear to see that $\varphi \left(e_n\right)$ in CRKRSL is bigger than $\kappa \left(e_n\right)$ in CRMCC, when the error is small. (Note that $\varphi \left(e_n\right)=\kappa \left(e_n\right)$ if $\gamma =0$). Moreover, when $\gamma =0$, the CRKRSL degenerates to an efficient CRMCC [26] algorithm and $\varphi \left(e_n\right)$ reaches the maximum at $e_n=0$. When $\gamma >0$, $\varphi \left(e_n\right)$ reaches the maximum at local points around $e_n=0$ with a larger increment, resulting in a faster convergence speed and higher accuracy than CRMCC.

However, Equation (20) is not stable at the initial update phase, since the variable step size ${\eta }_n$ is large with a small instant n. Especially, when the kernel width of KRSL is small, the step size ${\eta }_n$ even exceeds the convergence range, leading to the attenuation of filtering performance. To overcome this unfavorable factor, we introduce a gradient strategy with a fixed step size $\mu$ to replace the update of Equation (20) in the initial L iterations, which is described by

(21)$\begin{array}{c}\hfill {\mathbf{w}}_n=\widehat{\mathbf{Q}}({\mathbf{w}}_{n-1}+\mu \varphi \left(e_n\right)e_n{\mathbf{u}}_n)+\widehat{\mathbf{p}}\end{array}$

with $\widehat{\mathbf{Q}}=\mathbf{I}-\mathbf{C}{\left({\mathbf{C}}^{\mathrm{T}}\mathbf{C}\right)}^{-1}{\mathbf{C}}^{\mathrm{T}}$ and $\widehat{\mathbf{p}}=\mathbf{C}{\left({\mathbf{C}}^{\mathrm{T}}\mathbf{C}\right)}^{-1}\mathbf{f}$.

Finally, the CRKRSL is summarized in Algorithm 1.

Algorithm 1: The CRKRSL Algorithm.

Input:

Data pair $\{{\mathbf{u}}_n,d_n\}\in {\mathbb{R}}^m\times \mathbb{R}$, $n=1,2,\cdots$.

Initialization:

Choose step-size $\mu$; kernel width $\sigma$; risk-sensitive $\gamma$; initial iterative length L;

training size $N_{tr}$;

initial weight ${\mathbf{w}}_0=\mathbf{0}$.

$e_n=d_n-{\mathbf{w}}_{n-1}^{\mathrm{T}}{\mathbf{u}}_n$

for $n=1:L$

${\mathbf{w}}_n=\widehat{\mathbf{Q}}({\mathbf{w}}_{n-1}+\mu \varphi \left(e_n\right)e_n{\mathbf{u}}_n)+\widehat{\mathbf{p}}$

end

for $n=(L+1):N_{tr}$

${\mathbf{w}}_n=\mathbf{Q}({\mathbf{w}}_{n-1}+{\eta }_n\varphi \left(e_n\right)e_n\mathbf{Z}{\mathbf{u}}_n)+\mathbf{p}$

end

Remark 2.

In Equation (20), the constant inverse matrix $\mathsf{\Theta }={\left({\mathbf{C}}^{\mathrm{T}}\mathbf{Z}\mathbf{C}\right)}^{-1}$ needs to be calculated only once before the update. On the contrary, ${\left({\mathbf{C}}^{\mathrm{T}}{\mathbf{U}}_n\mathbf{C}\right)}^{-1}$ in Equation (15) needs to be updated iteratively. Meanwhile, the update of matrix ${\mathbf{U}}_n$ by Equation (10) is avoided by using Equation (20). Therefore, the proposed CRKRSL with Equation (20) has a lower computational complexity than one with Equation (15).

3. Stability Analysis

To obtain the stability condition of mean square weight error of CRKRSL, we define the weight error and model error as

(22)$\begin{array}{c}\hfill {\tilde{\mathbf{w}}}_n={\mathbf{w}}_n-{\mathbf{w}}_o\end{array}$

(23)$\begin{array}{c}\hfill {\mathbf{w}}_{\mathsf{\Delta }}={\mathbf{w}}_{*}-{\mathbf{w}}_o\end{array}$

with the optimal weight

(24)$\begin{array}{cc}\hfill {\mathbf{w}}_o& =\underset{n\to \infty }{lim}({\mathbf{U}}_n{\mathbf{d}}_n+{\mathbf{U}}_n\mathbf{C}{\mathbf{\theta }}_n)\hfill \\ \hfill & =\underset{n\to \infty }{lim}{\mathbf{U}}_n{\mathbf{d}}_n+\underset{n\to \infty }{lim}{\mathbf{U}}_n\mathbf{C}{\left({\mathbf{C}}^{\mathrm{T}}{\mathbf{U}}_n\mathbf{C}\right)}^{-1}(\mathbf{f}-{\mathbf{C}}^{\mathrm{T}}{\mathbf{U}}_n{\mathbf{d}}_n)\hfill \\ \hfill & =\underset{n\to \infty }{lim}{\left[\frac{1}{n}{\displaystyle \sum _{l=1}^n}\varphi \left(e_l\right){\mathbf{u}}_l{\mathbf{u}}_l^{\mathrm{T}}\right]}^{-1}\left[\frac{1}{n}{\displaystyle \sum _{l=1}^n}\varphi \left(e_l\right){\mathbf{u}}_l{d}_l\right]\hfill \\ \hfill & +\underset{n\to \infty }{lim}{\left[\frac{1}{n}{\displaystyle \sum _{l=1}^n}\varphi \left(e_l\right){\mathbf{u}}_l{\mathbf{u}}_l^{\mathrm{T}}\right]}^{-1}\mathbf{C}{\left({\mathbf{C}}^{\mathrm{T}}{\left[\frac{1}{n}{\displaystyle \sum _{l=1}^n}\varphi \left(e_l\right){\mathbf{u}}_l{\mathbf{u}}_l^{\mathrm{T}}\right]}^{-1}\mathbf{C}\right)}^{-1}(\mathbf{f}-{\mathbf{C}}^{\mathrm{T}}{\mathbf{U}}_n{\mathbf{d}}_n)\hfill \\ \hfill & =\mathbf{U}\mathbf{d}+\mathbf{U}\mathbf{C}{\left({\mathbf{C}}^{\mathrm{T}}\mathbf{U}\mathbf{C}\right)}^{-1}(\mathbf{f}-{\mathbf{C}}^{\mathrm{T}}\mathbf{U}\mathbf{d}),\hfill \end{array}$

where the robust correlation matrix and vector are defined as $\mathbf{U}=\underset{n\to \infty }{lim}{\left[\frac{1}{n}{\sum }_{l=1}^n\varphi \left(e_l\right){\mathbf{u}}_l{\mathbf{u}}_l^{\mathrm{T}}\right]}^{-1}=\mathbb{E}{\left[\varphi \left(e_n\right){\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}\right]}^{-1}$ and $\mathbf{d}=\underset{n\to \infty }{lim}\left[\frac{1}{n}{\sum }_{l=1}^n\varphi \left(e_l\right){\mathbf{u}}_l{d}_l\right]=\mathbb{E}\left[\varphi \left(e_n\right){\mathbf{u}}_n{d}_n\right]$.

Subtracting ${\mathbf{w}}_o$ from both sides of Equation (20), we obtain

(25)$\begin{array}{cc}\hfill {\tilde{\mathbf{w}}}_n& =\mathbf{Q}({\mathbf{w}}_{n-1}+{\eta }_n\varphi \left(e_n\right)\mathbf{Z}{\mathbf{u}}_n{d}_n-{\eta }_n\varphi \left(e_n\right)\mathbf{Z}{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}{\mathbf{w}}_{n-1})+\mathbf{p}-{\mathbf{w}}_o\hfill \\ \hfill & =\mathbf{Q}({\mathbf{w}}_{n-1}+{\eta }_n\varphi \left(e_n\right)\mathbf{Z}{\mathbf{u}}_n({\mathbf{u}}_n^{\mathrm{T}}{\mathbf{w}}_{*}+v_n)-{\eta }_n\varphi \left(e_n\right)\mathbf{Z}{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}{\mathbf{w}}_{n-1})+\mathbf{p}-{\mathbf{w}}_o\hfill \\ \hfill & =\mathbf{Q}({\mathbf{w}}_{n-1}+{\eta }_n\varphi \left(e_n\right)\mathbf{Z}{\mathbf{u}}_n({\mathbf{u}}_n^{\mathrm{T}}({\mathbf{w}}_{\mathsf{\Delta }}+{\mathbf{w}}_o)+v_n)-{\eta }_n\varphi \left(e_n\right)\mathbf{Z}{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}{\mathbf{w}}_{n-1})+\mathbf{p}-{\mathbf{w}}_o\hfill \\ \hfill & =\mathbf{Q}({\mathbf{w}}_{n-1}-{\eta }_n\varphi \left(e_n\right)\mathbf{Z}{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}{\tilde{\mathbf{w}}}_{n-1})+{\eta }_n\varphi \left(e_n\right)\mathbf{Q}\mathbf{Z}{\mathbf{u}}_n({\mathbf{u}}_n^{\mathrm{T}}{\mathbf{w}}_{\mathsf{\Delta }}+v_n)+\mathbf{p}-{\mathbf{w}}_o\hfill \\ \hfill & =\mathbf{Q}(\mathbf{I}-{\eta }_n\varphi \left(e_n\right)\mathbf{Z}{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}){\tilde{\mathbf{w}}}_{n-1}+{\eta }_n\varphi \left(e_n\right)\mathbf{Q}\mathbf{Z}{\mathbf{u}}_n({\mathbf{u}}_n^{\mathrm{T}}{\mathbf{w}}_{\mathsf{\Delta }}+v_n)+\mathbf{Q}{\mathbf{w}}_o-{\mathbf{w}}_o+\mathbf{p}.\hfill \end{array}$

Since $\mathbf{Q}$ is an idempotent matrix, we have $\mathbf{Q}{\mathbf{w}}_o-{\mathbf{w}}_o+\mathbf{p}=\mathbf{0}$ and $\mathbf{Q}{\tilde{\mathbf{w}}}_n={\tilde{\mathbf{w}}}_n$. Then, we get

(26)$\begin{array}{cc}\hfill {\tilde{\mathbf{w}}}_n& =\mathbf{Q}(\mathbf{I}-{\eta }_n\varphi \left(e_n\right)\mathbf{Z}{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}){\tilde{\mathbf{w}}}_{n-1}+{\eta }_n\varphi \left(e_n\right)\mathbf{Q}\mathbf{Z}{\mathbf{u}}_n({\mathbf{u}}_n^{\mathrm{T}}{\mathbf{w}}_{\mathsf{\Delta }}+v_n)\hfill \\ \hfill & =(\mathbf{I}-{\eta }_n\varphi \left(e_n\right)\mathbf{Q}\mathbf{Z}{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}){\tilde{\mathbf{w}}}_{n-1}+{\eta }_n\varphi \left(e_n\right)\mathbf{Q}\mathbf{Z}{\mathbf{u}}_n({\mathbf{u}}_n^{\mathrm{T}}{\mathbf{w}}_{\mathsf{\Delta }}+v_n).\hfill \end{array}$

We take the expectation operator on both sides of the square norm, since the noise $v_n$ is independent with ${\mathbf{u}}_n$ and the input sequence $\left\{{\mathbf{u}}_n\right\}$ is i.i.d under assumptions A1–A2. We further get the useful results that the a priori error weight ${\tilde{\mathbf{w}}}_{n-1}$ is independent with ${\mathbf{u}}_n$ and $v_n$ based on the independence assumptions [32]. Hence, the cross terms are equal to zeros. Then, we obtain

(27)$\begin{array}{cc}\hfill \mathbb{E}[\parallel {\tilde{\mathbf{w}}}_n{\parallel }^2]& =\mathbb{E}[\parallel {\tilde{\mathbf{w}}}_{n-1}{\parallel }_{{\mathbf{F}}_n}^2]+{\eta }_n^2{\delta }^2\mathbb{E}\left[{\varphi }^2\left(e_n\right)\right]\mathbb{E}\left[{\mathbf{u}}_n^{\mathrm{T}}\mathbf{Z}\mathbf{Q}\mathbf{Z}{\mathbf{u}}_n\right]\hfill \\ \hfill & +{\eta }_n^2\mathbb{E}\left[{\varphi }^2\left(e_n\right)\right]{\mathbf{w}}_{\mathsf{\Delta }}^{\mathrm{T}}\mathbb{E}\left[{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}\mathbf{Z}\mathbf{Q}\mathbf{Z}{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}\right]{\mathbf{w}}_{\mathsf{\Delta }}\hfill \end{array}$

with

(28)$\begin{array}{cc}\hfill {\mathbf{F}}_n& =\mathbb{E}\left[{(\mathbf{I}-{\eta }_n\varphi \left(e_n\right)\mathbf{Q}\mathbf{Z}{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}})}^{\mathrm{T}}(\mathbf{I}-{\eta }_n\varphi \left(e_n\right)\mathbf{Q}\mathbf{Z}{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}})\right]\hfill \\ \hfill & =\mathbf{I}-2{\eta }_n\mathbb{E}\left[\varphi \left(e_n\right)\right]\mathbb{E}\left[\mathbf{Q}\mathbf{Z}{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}\right]+{\eta }_n^2\mathbb{E}\left[{\varphi }^2\left(e_n\right)\right]\mathbb{E}\left[{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}\mathbf{Z}\mathbf{Q}\mathbf{Z}{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}\right].\hfill \end{array}$

According to Isserlis’s theorem [33], the symmetric matrix with fourth-moment Gaussian variables can be separated by

(29)$\begin{array}{cc}\hfill \mathbf{V}& =\mathbb{E}\left[{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}\mathbf{Z}\mathbf{Q}\mathbf{Z}{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}\right]\hfill \\ \hfill & =\mathbb{E}\left[{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}\mathbf{Z}\right]\mathbb{E}\left[\mathbf{Q}\mathbf{Z}{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}\right]+\mathbb{E}\left[{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}\mathbf{Z}\mathbf{Q}\right]\mathbb{E}\left[\mathbf{Z}{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}\right]+\mathbb{E}\left[{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}\right]\mathbb{E}\left[{\mathbf{u}}_n^{\mathrm{T}}\mathbf{Z}\mathbf{Q}\mathbf{Z}{\mathbf{u}}_n\right]\hfill \\ \hfill & =2\mathbf{Q}+\mathrm{tr}\left\{\mathbf{Q}\mathbf{Z}\right\}\mathbf{R},\hfill \end{array}$

where $\mathrm{tr}\{·\}$ denotes the trace operator and $\mathbf{R}={\mathbf{Z}}^{-1}=\mathbb{E}\left[{\mathbf{u}}_n{\mathbf{u}}_n^{\mathrm{T}}\right]$ is a positive define correlation matrix.

Therefore, Equation (27) can be simplified as

(30)$\begin{array}{cc}\hfill \mathbb{E}[\parallel {\tilde{\mathbf{w}}}_n{\parallel }^2]& =\mathbb{E}[\parallel {\tilde{\mathbf{w}}}_{n-1}{\parallel }_{{\mathbf{F}}_n}^2]+{\eta }_n^2{\delta }^2\mathbb{E}\left[{\varphi }^2\left(e_n\right)\right]\mathrm{tr}\left\{\mathbf{Q}\mathbf{Z}\right\}+{\eta }_n^2\mathbb{E}\left[{\varphi }^2\left(e_n\right)\right]{\mathbf{w}}_{\mathsf{\Delta }}^{\mathrm{T}}(2\mathbf{Q}+\mathrm{tr}\left\{\mathbf{Q}\mathbf{Z}\right\}\mathbf{R}){\mathbf{w}}_{\mathsf{\Delta }}\hfill \\ \hfill & =\mathbb{E}[\parallel {\tilde{\mathbf{w}}}_{n-1}{\parallel }_{{\mathbf{F}}_n}^2]+{\eta }_n^2\mathbb{E}\left[{\varphi }^2\left(e_n\right)\right]\mathrm{tr}\left\{\mathbf{Q}\mathbf{Z}\right\}({\mathbf{w}}_{\mathsf{\Delta }}^{\mathrm{T}}\mathbf{R}{\mathbf{w}}_{\mathsf{\Delta }}+{\delta }^2)\hfill \end{array}$

with a simplified

(31)$\begin{array}{cc}\hfill {\mathbf{F}}_n& =\mathbf{I}-2{\eta }_n\mathbb{E}\left[\varphi \left(e_n\right)\right]\mathbf{Q}+{\eta }_n^2\mathbb{E}\left[{\varphi }^2\left(e_n\right)\right]\mathbf{V}\hfill \\ \hfill & =\mathbf{I}-2{\eta }_n\mathbb{E}\left[\varphi \left(e_n\right)\right]\mathbf{Q}+{\eta }_n^2\mathbb{E}\left[{\varphi }^2\left(e_n\right)\right](2\mathbf{Q}+\mathrm{tr}\left\{\mathbf{Q}\mathbf{Z}\right\}\mathbf{R}).\hfill \end{array}$

Define $q_k$ and $r_k$, $k\in \{1,2,\cdots ,m\}$, being the kth eigenvalue of matrix $\mathbf{Q}$ and $\mathbf{R}$. To ensure the square stability, the eigenvalues of ${\mathbf{F}}_n$ should satisfy the following condition:

(32)$\begin{array}{cc}\hfill & |1-2{\eta }_n\mathbb{E}\left[\varphi \left(e_n\right)\right]q_k+{\eta }_n^2\mathbb{E}\left[{\varphi }^2\left(e_n\right)\right](2q_k+\mathrm{tr}\left\{\mathbf{Q}\mathbf{Z}\right\}{r}_k)|<1,\hfill \\ \hfill & \mathrm{for}k=1,2,\cdots ,m.\hfill \end{array}$

Then, the convergence condition about the step size can be expressed as

(33)$\begin{array}{c}\hfill {\eta }_n<{\displaystyle \underset{i}{min}}\left(\frac{2\mathbb{E}\left[\varphi \left(e_n\right)\right]q_k}{\mathbb{E}\left[{\varphi }^2\left(e_n\right)\right](2q_k+\mathrm{tr}\left\{\mathbf{Q}\mathbf{Z}\right\}{r}_k)}\right).\end{array}$

Since the step size ${\eta }_n={\left(n\mathbb{E}\left[\varphi \left(v_n\right)\right]\right)}^{-1}$ is related to the iteration n, finally, we obtain the weight square error stability condition about the iteration n by

(34)$\begin{array}{c}\hfill n>{\displaystyle \underset{i}{max}}\left(\frac{\mathbb{E}\left[{\varphi }^2\left(e_n\right)\right](2q_k+\mathrm{tr}\left\{\mathbf{Q}\mathbf{Z}\right\}{r}_k)}{2\mathbb{E}\left[\varphi \left(v_n\right)\right]\mathbb{E}\left[\varphi \left(e_n\right)\right]q_k}\right),\end{array}$

where the nonlinear terms $\mathbb{E}\left[\varphi \left(e_n\right)\right]$ and $\mathbb{E}\left[{\varphi }^2\left(e_n\right)\right]$ can be approximated by the Taylor expansion.

Remark 3.

Inequality (34) implies that the iteration n should be sufficiently large to guarantee the convergence. When n is small, CRKRSL with Equation (20) cannot satisfy (34), resulting in fluctuations at the initial update stage. Therefore, it is reasonable to use Equation (21) to replace Equation (20) to improve the convergence speed and filtering accuracy.

4. Results and Discussion

In this section, we will show the advantages of the CRKRSL algorithm on the filtering accuracy and computational complexity for both low-dimensional and high-dimensional inputs. The noise model, data selection and algorithms comparison are given as follows, respectively.

Noise model: We first consider a pure Gaussian noise to test the filtering accuracy, i.e., Gaussian noise with $v_n\sim \mathrm{N}(0,{\delta }^2)$, where $\mathrm{N}(\overline{\mu },{\overline{\delta }}^2)$ denotes the Gaussian distribution with mean $\overline{\mu }$ and variance ${\overline{\delta }}^2$. Then, a mixed noise is considered to test the robustness of CRKRSL. The mixed noise model under a probability process is denoted as

(35)$\begin{array}{c}\hfill v_n=b\left(n\right)v_1\left(n\right)+(1-b\left(n\right))v_2\left(n\right),\end{array}$

where $b\left(n\right)\in \{0,1\}$ satisfies the binary distribution with probability $P\left\{b\right(n)=0\}=0.1$ and $P\left\{b\right(n)=1\}=0.9$. $v_1\left(n\right)$ with high probability generates the ordinary noise by a Gaussian distribution $\mathrm{N}(0,{\delta }_1^2)$ and $v_2\left(n\right)$ with low probability generates a few impulsive noise. Two types of $v_2\left(n\right)$ are considered: (a) Gaussian noise with large variance, i.e., $v_2\left(n\right)\sim \mathrm{N}(0,100)$; (b) $\alpha$-stable noise [34] with parameter function $\mathcal{F}(0.8,0,0.1,0)$.

Data selection: The training inputs sampled from a Gaussian distribution with zero mean and variance matrix $\mathbf{R}$ and 5000 samples are chosen for the simulation. The parameters $\mathbf{C}$, $\mathbf{f}$, $\mathbf{R}$ are configured as the same as in [7]. Note that there exist two sets of data with different input dimensions in [7]. The underlying dimension is either $m=7$ or $m=31$. The simulated mean square deviation (MSD) is define as $\mathrm{MSD}\left(\mathrm{dB}\right)=10{log}_{10}(\parallel {\tilde{\mathbf{w}}}_n{\parallel }^2)$ and the steady-state MSD is defined as the mean of last 1000 samples. The obtained results are averaged over 500 Monte Carlo trials. The results were run on MATLAB version R2020b on a Windows 10 operating system, configured with an Intel(R) Core(TM) i7-8700 CPU 3.20 GHz and RAM 16 GB.

Compared algorithms: The constrained algorithms, including the CLMS [7], CMCC [21], CLLAF [25] CRLS [9], and CRMCC [26], are chosen to be compared with the proposed CRKRSL. For fair comparison, the kernel widths $\sigma$ of CMCC, CRMCC, and CRKRSL are set by the same value; the regularization terms of CRLS and CRMCC are set as 0.001; the initial iterative length of CRKRSL is set as the same the input dimension, i.e., $L=m$.

4.1. Low-Dimensional Input

In this part, the input dimension m and constrained dimension q are set as $m=7$ and $q=3$, respectively. The noise considered here satisfies the following conditions, i.e., Gaussian noise $v\left(n\right)\sim \mathrm{N}(0,0.1)$ and mixed noise $v_1\left(n\right)\sim \mathrm{N}(0,0.1)$. To reflect the influence of risk-sensitive parameter $\gamma$ on the MSD, the relations between $\gamma$ and steady-state MSD are shown in Figure 2 and Figure 3 under different noise models. From Figure 2, one can see that CRKRSL has a stable steady-state MSD under Gaussian noise. Therefore, $\gamma$ has little influence on the performance of CRKRSL. From Figure 3, it is observed that the CRKRSL is sensitive to $\gamma$ under mixed noise with model (a) and achieves the lowest steady-state MSD around $\gamma =2.5$. Therefore, we choose $\gamma =2.5$ for the following algorithm comparison in Figure 4 and Figure 5. Note that $\rho$ denotes the shape parameter in the CLLAF algorithm. All necessary parameters are given in the figures.

From Figure 4, one can see that CRKRSL, CRMCC and CRLS coincide and have almost the same MSDs under Gaussian noise. This implies that the CRKRSL deals with the Gaussian noise well by choosing a large kernel width. From Figure 5, it is observed that CRKRSL has the best performance among all constrained algorithms since the risk-sensitive parameter $\gamma$ can avoid fluctuations caused by a small kernel width. Moreover, CRKRSL is more stable than CRMCC at the initial stage, potentially leading to a better filtering performance. In Table 1, we further compare the consumed time at each iteration and steady-state MSDs of each algorithm under mixed noise with model (a). The consumed time of CRKRSL is far less than that of CRMCC and CRLS.

To show the advantage of CRKRSL on computational complexity, Table 2 lists the compared results of all mentioned algorithms at each iteration. One can see that CRKRSL has a lower computational complexity than CRLS and CRMCC by avoiding calculating the inverse matrix. Although the inverse matrix ${\mathbf{U}}_n$ of CRMCC is not required to be calculated, the inverse matrix ${\left({\mathbf{C}}^{\mathrm{T}}{\mathbf{U}}_n\mathbf{C}\right)}^{-1}$ is still needed to be calculated at each iteration, resulting in a high computational complexity with a large q.

To test the performance of CRKRSL under mixed noise with model (b), Figure 6 gives the MSD results of all mentioned algorithms. One can see from Figure 6 that CRKRSL, CRMCC, CMCC and CLLAF show strong robustness to outliers, and CRKRSL has the lowest MSD, whereas the CLMS and CRLS are not stable due to being sensitive to $\alpha$-stable noise.

4.2. High-Dimensional Input

In this part, the input dimension m and constrained dimension q are set as $m=31$ and $q=1$, respectively. We only consider the mixed noise, since the Gaussian noise influences the performance of CRKRSL little by selecting a large kernel width. The mixed noise satisfies $v_1\left(n\right)\sim \mathrm{N}(0,1)$.

Figure 7 and Figure 8 show the MSDs of different algorithms under mixed noise with model (a) and model (b), respectively. It is clear to see that CRKRSL shows the best performance out of all the compared algorithms both under mixed noise with model (a) and model (b). The initial iterative length L influences the convergence speed, significantly. Therefore, the initial iterative length should be not smaller than the input dimension. In Table 3, we further compare the consumed time at each iteration and steady-state MSDs of each algorithm under mixed noise with model (a). One can see that the consumed time of CRKRSL is far less than that of CRMCC. Moreover, CRKRSL has the lowest steady-state MSD value.

5. Conclusions

By introducing the linear constraints to the kernel risk-sensitive loss (KRSL), a low-complexity constrained recursive KRSL (CRKRSL) algorithm is presented with the help of average approximation. Since the risk-sensitive parameter is able to control the smoothness of the performance surface, CRKRSL achieves higher accuracy than some existing constrained recursive algorithms. Due to the inaccuracy of the average approximation with a few inputs, a fixed step-size gradient method is adopted to avoid the instability of CRKRSL at the initial update stage. Moreover, mean square analysis indicates that iteration times influence the stability of CRKRSL significantly and simulation results confirm the advantages of CRKRSL. The effectiveness of CRKRSL highly relies on the average approximation method, which has limits when coping with nonstationary signals. In the future, we focus on finding a novel approximation method to process both stationary and nonstationary signals and further improve the computational efficiency and accuracy of constrained recursive algorithms.

Author Contributions

Conceptualization, S.X. and C.Z.; methodology, S.X.; software, S.X. and Z.G.; validation, S.X.; formal analysis, S.X.; investigation, S.X. and D.Y.; resources, C.Z.; data curation, S.X. and D.Y.; writing—original draft preparation, S.X.; writing—review and editing, C.Z. and Z.G.; visualization, S.X.; supervision, C.Z.; project administration, C.Z. and D.Y.; funding acquisition, C.Z. and Z.G. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by Chongqing Social Science Planning Project (2021BS038) and Natural Science Foundation of Chongqing, China (cstc2021jcyj-bshX0035, cstc2018jcyjA2453).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Figures and Tables

Figure 1. The relation between [Forumla omitted. See PDF.] and the error [Forumla omitted. See PDF.] ([Forumla omitted. See PDF.]).

Figure 2. Steady-state MSD versus risk-sensitive parameter [Forumla omitted. See PDF.] under Gaussian noise ([Forumla omitted. See PDF.]).

Figure 3. Steady-state MSD versus risk-sensitive parameter [Forumla omitted. See PDF.] under mixed noise with model (a) ([Forumla omitted. See PDF.]).

Figure 4. MSDs of compared algorithms under Gaussian noise ([Forumla omitted. See PDF.]).

Figure 5. MSDs of compared algorithms under mixed noise with model (a) ([Forumla omitted. See PDF.]).

Figure 6. MSDs of compared algorithms under mixed noise with model (b) ([Forumla omitted. See PDF.]).

Figure 7. MSDs of compared algorithms under mixed noise with model (a) ([Forumla omitted. See PDF.]).

Figure 8. MSDs of compared algorithms under mixed noise with model (b) ([Forumla omitted. See PDF.]).

Table 1

Consumed time and steady-state MSD of CLMS, CMCC, CLLAF, CRLS, CRMCC and CRKRSL under mixed noise with model (a) based on low-dimensional input.

Algorithm	Consumed Time (s)	Steady-State MSD (dB)
CLMS	$0.0024$	−$6.70$
CMCC	$0.0028$	−$10.86$
CLLAF	$0.0025$	−$18.58$
CRLS	$0.0324$	−$21.23$
CRMCC	$0.0327$	−$27.75$
CRKRSL	$0.0038$	−$34.34$

Table 2

Computational complexity of CLMS, CMCC, CLLAF, CRLS, CRMCC and CRKRSL at each iteration.

Algorithms	Computational Complexity
CLMS	$O\left(m^2\right)$
CMCC	$O\left(m^2\right)$
CLLAF	$O\left(m^2\right)$
CRLS	$O(m^3+q^3)$
CRMCC	$O(m^2+q^3)$
CRKRSL	$O\left(m^2\right)$

Table 3

Consumed time and steady-state MSD of CLMS, CMCC, CLLAF, CRLS, CRMCC and CRKRSL under mixed noise with model (a) based on high-dimensional input.

Algorithm	Consumed Time (s)	Steady-State MSD (dB)
CLMS	$0.0035$	$2.98$
CMCC	$0.0034$	$-2.21$
CLLAF	$0.0034$	$-3.13$
CRLS	$0.0112$	$-11.02$
CRMCC	$0.0114$	$-15.45$
CRKRSL	$0.0063$	$-19.72$

References

1. De Campos, M.L.R.; Werner, S.; Apolinário, J.A. Constrained adaptive filters. Adaptive Antenna Arrays: Trends and Applications; Springer: New York, NY, USA, 2004; pp. 46-64.

2. Frost, O. An algorithm for linearly constrained adaptive array processing. Proc. IEEE; 1972; 60, pp. 926-935. [DOI: https://dx.doi.org/10.1109/PROC.1972.8817]

3. Li, J.; Stoica, P. Robust Adaptive Beamforming; John Wiley & Sons: New York, NY, USA, 2005.

4. Diniz, P.S. Adaptive Filtering: Algorithms and Practical Implementation; Springer: Cham, Switzerland, 1997.

5. Wu, Q.; Li, Y.; Xue, W. A kernel recursive maximum versoria-like criterion algorithm for nonlinear channel equalization. Symmetry; 2019; 11, 1067. [DOI: https://dx.doi.org/10.3390/sym11091067]

6. Verdu, S. Multiuser Detection; Cambridge University Press: Cambridge, UK, 1998.

7. Arablouei, R.; Doğançay, K.; Werner, S. On the mean-square performance of the constrained LMS algorithm. Signal Process.; 2015; 117, pp. 192-197. [DOI: https://dx.doi.org/10.1016/j.sigpro.2015.05.011]

8. Resende, L.S.; Romano, J.M.T.; Bellanger, M.G. A fast least-squares algorithm for linearly constrained adaptive filtering. IEEE Trans. Signal Process.; 1996; 44, pp. 1168-1174. [DOI: https://dx.doi.org/10.1109/78.502329]

9. Arablouei, R.; Doğançay, K. Performance analysis of linear-equality-constrained least-squares estimation. IEEE Trans. Signal Process.; 2015; 63, pp. 3762-3769. [DOI: https://dx.doi.org/10.1109/TSP.2015.2424199]

10. Arablouei, R.; Doğançay, K. Reduced-complexity constrained recursive least-squares adaptive filtering algorithm. IEEE Trans. Signal Process.; 2012; 60, pp. 6687-6692. [DOI: https://dx.doi.org/10.1109/TSP.2012.2217339]

11. Werner, S.; Apolinário, J.A.; de Campos, M.L.R.; Diniz, P. Low-complexity constrained affine-projection algorithms. IEEE Trans. Signal Process.; 2005; 53, pp. 4545-4555. [DOI: https://dx.doi.org/10.1109/TSP.2005.859348]

12. Apolinário, J.A., Jr.; de Campos, M.L.R.; Bernal, O.C.P. The constrained conjugate gradient algorithm. IEEE Signal Process. Lett.; 2000; 12, pp. 351-354. [DOI: https://dx.doi.org/10.1109/97.883366]

13. De Campos, M.L.R.; Werner, S.; Apolinário, J.A. Constrained adaptation algorithms employing Householder transformation. IEEE Trans. Signal Process.; 2002; 9, pp. 2187-2195. [DOI: https://dx.doi.org/10.1109/TSP.2002.801893]

14. Arablouei, R.; Doğançay, K. Linearly-constrained recursive total least-squares algorithm. IEEE Signal Process. Lett.; 2012; 12, pp. 821-824. [DOI: https://dx.doi.org/10.1109/LSP.2012.2221705]

15. Haykin, S. Adaptive Filter Theory; Prentice-Hall: Upper Saddle River, NJ, USA, 2002.

16. Liu, W.; Pokharel, P.P.; Príncipe, J.C. Correntropy: Properties and applications in non-Gaussian signal processing. IEEE Trans. Signal Process.; 2007; 55, pp. 5286-5298. [DOI: https://dx.doi.org/10.1109/TSP.2007.896065]

17. Li, Y.; Wang, Y.; Sun, L. A proportionate normalized maximum correntropy criterion algorithm with correntropy induced metric constraint for identifying sparse systems. Symmetry; 2018; 10, 683. [DOI: https://dx.doi.org/10.3390/sym10120683]

18. Chen, B.; Xing, L.; Zhao, H.; Zheng, N.; Príncipe, J.C. Generalized correntropy for robust adaptive filtering. IEEE Trans. Signal Process.; 2016; 64, pp. 3376-3387. [DOI: https://dx.doi.org/10.1109/TSP.2016.2539127]

19. Erdogmus, D.; Príncipe, J.C. An error-entropy minimization algorithm for supervised training of nonlinear adaptive systems. IEEE Trans. Signal Process.; 2002; 50, pp. 1780-1786. [DOI: https://dx.doi.org/10.1109/TSP.2002.1011217]

20. Príncipe, J.C. Information Theoretic Learning: Renyi’s Entropy and Kernel Perspectives; Springer: New York, NY, USA, 2010.

21. Peng, S.; Chen, B.; Sun, L.; Ser, W.; Lin, Z. Constrained maximum correntropy adaptive filtering. Signal Process.; 2017; 140, pp. 116-126. [DOI: https://dx.doi.org/10.1016/j.sigpro.2017.05.009]

22. Bhattacharjee, S.S.; Shaikh, M.A.; Kumar, K.; George, N.V. Robust constrained generalized correntropy and maximum versoria criterion adaptive filters. IEEE Trans. Circuits Syst. II Express Briefs; 2021; 68, pp. 3002-3006. [DOI: https://dx.doi.org/10.1109/TCSII.2021.3063491]

23. Peng, S.; Ser, W.; Chen, B.; Sun, L.; Lin, Z. Robust constrained adaptive filtering under minimum error entropy criterion. IEEE Trans. Circuits Syst. II Express Briefs; 2018; 65, pp. 1119-1123. [DOI: https://dx.doi.org/10.1109/TCSII.2017.2789207]

24. Wang, Z.; Zhao, H.; Zeng, X. Constrained least mean M-estimation adaptive filtering algorithm. IEEE Trans. Circuits Syst. II Express Briefs; 2021; 68, pp. 1507-1511. [DOI: https://dx.doi.org/10.1109/TCSII.2020.3022081]

25. Liang, T.; Li, Y.; Zakharov, Y.V.; Xue, W.; Qi, J. Constrained least lncosh adaptive filtering algorithm. Signal Process.; 2021; 183, 108044. [DOI: https://dx.doi.org/10.1016/j.sigpro.2021.108044]

26. Qian, G.; Ning, X.; Wang, S. Recursive constrained maximum correntropy criterion algorithm for adaptive filtering. IEEE Trans. Circuits Syst. II Express Briefs; 2020; 67, pp. 2229-2233. [DOI: https://dx.doi.org/10.1109/TCSII.2019.2944271]

27. Chen, B.; Xing, L.; Xu, B.; Zhao, H.; Zheng, N.; Príncipe, J.C. Kernel risk-sensitive loss: Definition, properties and application to robust adaptive filtering. IEEE Trans. Signal Process.; 2017; 65, pp. 2888-2901. [DOI: https://dx.doi.org/10.1109/TSP.2017.2669903]

28. Qian, G.; Dong, F.; Wang, S. Robust constrained minimum mixture kernel risk-sensitive loss algorithm for adaptive filtering. Digit. Signal Process.; 2020; 107, 102859. [DOI: https://dx.doi.org/10.1016/j.dsp.2020.102859]

29. Qian, G.; Wang, S.; Wang, L.; Duan, S. Convergence analysis of a fixed point algorithm under maximum complex correntropy criterion. IEEE Signal Process. Lett.; 2018; 25, pp. 1830-1834. [DOI: https://dx.doi.org/10.1109/LSP.2018.2873413]

30. Liu, W.; Príncipe, J.C.; Haykin, S. Kernel Adaptive Filtering: A Comprehensive Introduction; Wiley: Hoboken, NJ, USA, 2011.

31. Zhang, X.-D. Matrix Analysis and Applications; Cambridge University Press: Cambridge, UK, 2017.

32. Sayed, A.H. Adaptive Filters; John Wiley & Sons: Hoboken, NJ, USA, 2008.

33. Isserlis, L. On a formula for the product-moment coefficient of any order of a normal frequency distribution in any number of variables. Biometrika; 1918; 12, pp. 134-139. [DOI: https://dx.doi.org/10.1093/biomet/12.1-2.134]

34. Zhang, Q.; Feng, W.; Iu, H.H.C.; Wang, S. Adaptive filters with robust augmented space linear model: A weighted k-NN method. IEEE Trans. Signal Process.; 2021; 69, pp. 6448-6461. [DOI: https://dx.doi.org/10.1109/TSP.2021.3130964]