Performance Analysis of AF Relays with Maximal Ratio Combining in Nakagami-m Fading Environments

Abstract

This paper investigates the maximal ratio combining (MRC) performance of an amplify and forward (AF) relay system in Nakagami-$m$ fading environments. The study considers a general scenario with distinct $m$ fading parameters for the following three links, source to relay link, and source to destination link and relay to destination link. We derive new closed form expressions for the statistics of important performance metrics, including the moment generating function, outage probability, higher order moments of equivalent signal to noise ratio (SNR), ergodic capacity, and average symbol error probability (SEP) of common modulation types. In particular, we focus on analytical SEP expressions in the context of an additive white generalized Gaussian noise (AWGGN). As an active area of research, generalized noise receives much attention for its flexible model. However, analytical performance of modulation scheme in generalized noise type has not been found in open literature for AF relaying with MRC despite its practical usefulness. Without the help of analytical solutions, the SEP in generalized noise can only be obtained by a large number of repeated simulation experiments. Therefore, we present the general SEP expression by using special Fox’s $H$ function. Simulation results verify the accuracy of our theoretical analysis and show that the diversity order of MRC criterion linearly depends upon Nakagami parameters of three links.

This work is licensed under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.

1. Introduction

Maximal ratio combining (MRC) has been shown to achieve the effective reception diversity in traditional direct communications. In a relaying system, the destination can receive signals from direct source to destination link and relay link by using MRC [1–3]. A maximum diversity order was shown in [4] for a fixed gain amplify and forward (AF) relay when MRC is used at the destination. The outage performance of zero forcing beamforming and MRC for an AF two-way underlay network was analyzed in [5] while the outage probability and approximate symbol error rate of a multiple input multiple output relay with MRC was presented in [6, 7].

Although MRC has been extensively studied, most works have focused on deriving approximate solutions or upper and lower bound performance. Exact performance expressions are important in evaluating MRC but still lack sufficient study. In fact, the performance analysis of MRC encounters enormous computational complexity and inaccuracy, making it difficult to evaluate the comprehensive fading characteristics.

In this paper, we consider an AF relaying system with the MRC receiver in a Nakagami-$m$ fading environment, in which it is generally considered difficult to find the closed form expressions for performance study, especially when the links have distinct $m$ fading parameters. In contrast to the aforementioned works, the precise performance statistics of AF system with MRC receiver are analyzed in a Nakagami-$m$ fading environment. The relay system assumes that an indirect relay link and a direct link between source and destination coexist simultaneously. We first obtain the probability density function (PDF) of equivalent signal to noise ratio (SNR) according to the convolution formula. Then the cumulative distribution function (CDF) is presented in accordance with the principle of Laplace inverse transform. New exact formulas are developed for the statistics of higher order moments of equivalent SNR, ergodic capacity, and average SEP with distinct fading parameters among different hops. In addition, our SEP expression also permits assessment of error probability in the context of an additive white generalized Gaussian noise (AWGGN) environment, which notably facilitates more general characterization of noise types.

2. System Model

A two-hop cooperative system with one source node S, one AF relay node R, and one destination node D is considered, as shown in Figure 1. The source node, relay node, and destination node are equipped with one antenna. Let $h_s$, $h_r$, and $h_d$ represent the channel coefficients for S$\to$R, R$\to$D, and S$\to$D links, respectively. Assume all the links undergo independent but not necessarily identically distributed Nakagami-$m$ fading. The instantaneous SNRs for direct S$\to$D link and indirect relay S$\to$R$\to$D link are, respectively, given by$$\begin{array}{c}\text{(1)}& {\gamma }_{\mathrm{1}}=\frac{p_s{\left|h_d\right|}^{\mathrm{2}}}{{\sigma }_d^{\mathrm{2}}},{\gamma }_{\mathrm{2}}=\frac{p_s{\left|h_s\right|}^{\mathrm{2}}{p}_r{\left|h_r\right|}^{\mathrm{2}}/{\sigma }_r^{\mathrm{2}}{\sigma }_d^{\mathrm{2}}}{p_s{\left|h_s\right|}^{\mathrm{2}}/{\sigma }_r^{\mathrm{2}}+p_r{\left|h_r\right|}^{\mathrm{2}}/{\sigma }_d^{\mathrm{2}}}\end{array}$$where $p_s$ and $p_r$ denote available transmit powers of the source node and the relay node, respectively. ${\sigma }_r^{\mathrm{2}}$ and ${\sigma }_d^{\mathrm{2}}$ are noise variances at the relay node and the destination node, respectively. The instantaneous SNR of the MRC combiner output is given by $\gamma ={\gamma }_{\mathrm{1}}+{\gamma }_{\mathrm{2}}$. The PDF of the gamma distributed variable ${\gamma }_{\mathrm{1}}$ is easily given by [7]$$\begin{array}{}\text{(2)}& f_{{\gamma }_{\mathrm{1}}}\left(z\right)=\frac{{\beta }_d^{m_d}{z}^{m_d-\mathrm{1}}{e}^{-{\beta }_{d}z}}{\Gamma \left(m_d\right)}\end{array}$$where $m_d$ is the fading severity parameter of the direct link, ${\beta }_d$ denotes the corresponding scale parameters, and $\Gamma \left(·\right)$ is the gamma function. When the fading parameter $m$ of both S$\to$R and R$\to$D links is a nonnegative integer plus one half, the PDF of ${\gamma }_{\mathrm{2}}$ can be found in [8] $$\begin{array}{}\text{(3)}& f_{{\gamma }_{\mathrm{2}}}\left(z\right)=\frac{\sqrt{\pi }{e}^{-{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}z}}{\Gamma \left(m_s\right)\Gamma \left(m_r\right)}{\displaystyle \sum }_{k_{\mathrm{1}}=\mathrm{0}}^{⌊m_s⌋}\hspace{0.17em}{\displaystyle \sum }_{k_{\mathrm{2}}=\mathrm{0}}^{⌊m_r⌋}\hspace{0.17em}{\displaystyle \sum }_{l=\mathrm{0}}^{⌊\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)/\mathrm{2}⌋}\frac{\left(⌊m_s⌋+k_{\mathrm{1}}\right)!\left(⌊m_r⌋+k_{\mathrm{2}}\right)!{\beta }_s^{m_s/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{1}}/\mathrm{2}}{\beta }_r^{m_r/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{2}}/\mathrm{2}}}{{\mathrm{4}}^{k_{\mathrm{1}}+k_{\mathrm{2}}}{k}_{\mathrm{1}}!k_{\mathrm{2}}!\left(⌊m_s⌋-k_{\mathrm{1}}\right)!\left(⌊m_r⌋-k_{\mathrm{2}}\right)!{\mathrm{2}}^{\mathrm{2}l}}\times \frac{\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)!z^{m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\mathrm{3}/\mathrm{2}}}{l!\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-\mathrm{2}l\right)!}\times {\left(-\mathrm{1}\right)}^l{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-\mathrm{2}l}\end{array}$$where $m_i$ is the fading severity parameter, ${\beta }_i$ denotes the corresponding scale parameters for $i\in \left\{s,r\right\}$, and $\lfloor ·\rfloor$ is the floor function. Since the PDF of the sum of two random variables is the convolution of the PDFs of these two variables, the PDF of MRC SNR $\gamma$ is thus expressed as (4) f γ z = f γ 1 z ∗ f γ 2 z = π β d m d e - β d z Γ m s Γ m r Γ m d ∑ k 1 = 0 m s ∑ k 2 = 0 m r ∑ l = 0 m s - k 1 + m r - k 2 / 2 m s + k 1 ! m r + k 2 ! 4 k 1 + k 2 k 1 ! k 2 ! m s - k 1 ! × β s m s / 2 - 1 / 4 - k 1 / 2 β r m r / 2 - 1 / 4 - k 2 / 2 m s - k 1 + m r - k 2 ! m r - k 2 ! 2 2 l l ! m s - k 1 + m r - k 2 - 2 l ! × - 1 l β s + β r m s - k 1 + m r - k 2 - 2 l × z m d + m s - k 1 + m r - k 2 - l - 3 / 2 × B m d , m s - k 1 + m r - k 2 - l - 1 2 × 1 F 1 m s - k 1 + m r - k 2 - l - 1 2 ; m s - k 1 + m r - k 2 - l - 1 2 + m d ; β d - β s + β r 2 zwhere $\ast$ represents convolution operation, $B\left(·\right)$ is beta function [9, eq.(8.380.1)], and ${\hspace{0.17em}}_p{F}_q\left({\alpha }_{\mathrm{1}},\dots ,{\alpha }_p;{\beta }_{\mathrm{1}},\dots ,{\beta }_q;z\right)$ is called the generalized hypergeometric function [9, eq.(9.14)]. Although the PDF of $\gamma$ seems a little complicated, it only includes the finite multiplication and addition of exponential function, power function, and hypergeometric function, which can be easily calculated by the prevailing mathematical software, such as Maple.

[figure omitted; refer to PDF]

Although there is a simple differential and integral relationship between PDF and CDF, here one challenge arises from the generalized hypergeometric function when integrating the above PDF. Thus we turn to moment generating function. Using [9, eq.(3.35.3)], the moment generating function of $\gamma$ is given by $$\begin{array}{}\text{(5)}& M_{\gamma }\left(s\right)=M_{{\gamma }_{\mathrm{1}}}\left(s\right)M_{{\gamma }_{\mathrm{2}}}\left(s\right)=\frac{\sqrt{\pi }{\beta }_d^{m_d}}{\Gamma \left(m_s\right)\Gamma \left(m_r\right)}{\displaystyle \sum }_{k_{\mathrm{1}}=\mathrm{0}}^{⌊m_s⌋}\hspace{0.17em}{\displaystyle \sum }_{k_{\mathrm{2}}=\mathrm{0}}^{⌊m_r⌋}\hspace{0.17em}{\displaystyle \sum }_{l=\mathrm{0}}^{⌊\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)/\mathrm{2}⌋}\frac{\left(⌊m_s⌋+k_{\mathrm{1}}\right)!\left(⌊m_r⌋+k_{\mathrm{2}}\right)!}{{\mathrm{4}}^{k_{\mathrm{1}}+k_{\mathrm{2}}}{k}_{\mathrm{1}}!k_{\mathrm{2}}!\left(⌊m_s⌋-k_{\mathrm{1}}\right)!}\times \frac{{\beta }_s^{m_s/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{1}}/\mathrm{2}}{\beta }_r^{m_r/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{2}}/\mathrm{2}}\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)!}{\left(⌊m_r⌋-k_{\mathrm{2}}\right)!{\mathrm{2}}^{\mathrm{2}l}l!\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-\mathrm{2}l\right)!}\times {\left(-\mathrm{1}\right)}^l{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-\mathrm{2}l}\times \Gamma \left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\frac{\mathrm{1}}{\mathrm{2}}\right){\left(s+{\beta }_d\right)}^{-m_d}\times {\left(s+{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}\right)}^{-\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\mathrm{1}/\mathrm{2}\right)}\end{array}$$Further, by using [10, eq.(2.1.3.1)], the CDF of $\gamma$ is given by $$\begin{array}{}\text{(6)}& F_{\gamma }\left(z\right)=L^{-\mathrm{1}}\left(\frac{M_{\gamma }\left(s\right)}{s}\right)=\frac{\sqrt{\pi }{\beta }_d^{m_d}}{\Gamma \left(m_s\right)\Gamma \left(m_r\right)}{\displaystyle \sum }_{k_{\mathrm{1}}=\mathrm{0}}^{⌊m_s⌋}\hspace{0.17em}{\displaystyle \sum }_{k_{\mathrm{2}}=\mathrm{0}}^{⌊m_r⌋}\hspace{0.17em}{\displaystyle \sum }_{l=\mathrm{0}}^{⌊\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)/\mathrm{2}⌋}\frac{\left(⌊m_s⌋+k_{\mathrm{1}}\right)!\left(⌊m_r⌋+k_{\mathrm{2}}\right)!}{{\mathrm{4}}^{k_{\mathrm{1}}+k_{\mathrm{2}}}{k}_{\mathrm{1}}!k_{\mathrm{2}}!\left(⌊m_s⌋-k_{\mathrm{1}}\right)!}\times \frac{{\beta }_s^{m_s/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{1}}/\mathrm{2}}{\beta }_r^{m_r/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{2}}/\mathrm{2}}\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)!}{\left(⌊m_r⌋-k_{\mathrm{2}}\right)!{\mathrm{2}}^{\mathrm{2}l}l!\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-\mathrm{2}l\right)!}\times \frac{{\left(-\mathrm{1}\right)}^l{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{m_{\mathrm{s}}-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-\mathrm{2}l}}{\Gamma \left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l+m_d+\mathrm{1}/\mathrm{2}\right)}\times z^{m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l+m_d-\mathrm{1}/\mathrm{2}}\times \Gamma \left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\frac{\mathrm{1}}{\mathrm{2}}\right)\times {\Phi }_{\mathrm{2}}\left[m_d,m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\frac{\mathrm{1}}{\mathrm{2}};m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l+m_d+\frac{\mathrm{1}}{\mathrm{2}};-{\beta }_{d}z,-{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}z\right]\end{array}$$where $L^{-\mathrm{1}}\left(·\right)$ is the Laplace inverse transform and ${\Phi }_{\mathrm{2}}\left(·,·;·;·,·\right)$ is the confluent hypergeometric function of two variables [11].

3. Performance Analysis

3.1. Outage Probability and SNR Moments

Outage probability is the probability that the SNR fails to meet a predetermined threshold and mathematically expressed as$$\begin{array}{}\text{(7)}& P_{out}=\mathrm{P}\mathrm{r}\left(\gamma \le {\gamma }_{th}\right)=F_{\gamma }\left({\gamma }_{th}\right)\end{array}$$where ${\gamma }_{th}$ is the threshold value.

According to the relationship between the moment generating function and the higher order moments, the $n$th order moment of SNR $\gamma$ is given by $$\begin{array}{}\text{(8)}& \mathrm{E}\left[{\gamma }^n\right]={\left(-\mathrm{1}\right)}^n{M}_{\gamma }^{\left(n\right)}\left(\mathrm{0}\right)=\frac{\sqrt{\pi }}{\Gamma \left(m_s\right)\Gamma \left(m_r\right)}{\displaystyle \sum }_{k_{\mathrm{1}}=\mathrm{0}}^{⌊m_s⌋}\hspace{0.17em}{\displaystyle \sum }_{k_{\mathrm{2}}=\mathrm{0}}^{⌊m_r⌋}{\displaystyle \sum }_{l=\mathrm{0}}^{⌊\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)/\mathrm{2}⌋}{\displaystyle \sum }_{k=\mathrm{0}}^n\frac{C_n^k\left(⌊m_s⌋+k_{\mathrm{1}}\right)!\left(⌊m_r⌋+k_{\mathrm{2}}\right)!{\beta }_s^{m_s/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{1}}/\mathrm{2}}}{{\mathrm{4}}^{k_{\mathrm{1}}+k_{\mathrm{2}}}{k}_{\mathrm{1}}!k_{\mathrm{2}}!\left(⌊m_s⌋-k_{\mathrm{1}}\right)!\left(⌊m_r⌋-k_{\mathrm{2}}\right)!{\mathrm{2}}^{\mathrm{2}l}}\times \frac{{\beta }_r^{m_r/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{2}}/\mathrm{2}}{\beta }_d^{-k}\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)!{\left(-\mathrm{1}\right)}^l}{l!\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-\mathrm{2}l\right)!}\times \Gamma \left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\frac{\mathrm{1}}{\mathrm{2}}\right)\times {\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}k-m_s+k_{\mathrm{1}}-m_r+k_{\mathrm{2}}+\mathrm{1}-\mathrm{2}n}\times {\left(m_d\right)}_k{\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\frac{\mathrm{1}}{\mathrm{2}}\right)}_{n-k}\end{array}$$where $\mathrm{E}\left[·\right]$ is expectation operation, $M_{\gamma }^{\left(n\right)}\left(s\right)$ is the $n$th differential with respect to $M_{\gamma }\left(s\right)$, and ${\left(a\right)}_n$ is the Pochhammer symbol [12].

3.2. Ergodic Capacity

Using PDF directly to calculate ergodic capacity imposes paramount complexity, which is a kind of integral type as follows:$$\begin{array}{}\text{(9)}& J_1={{\displaystyle \int }}_{\mathrm{0}}^{\infty }\ln \left(\mathrm{1}+z\right)e^{-\beta z}{z}^{\alpha }{\hspace{0.17em}}_{\mathrm{1}}{F}_{\mathrm{1}}\left[a;b;\left(\beta -c\right)z\right]dz\end{array}$$Unfortunately, looking at the complexity of $J_{\mathrm{1}}$, it is very difficult to write the result in a straightforward manner. To get around this problem and render mathematical feasibility, invoking the result in [13, eq.$(15)$], the ergodic capacity can be expressed in the moment generating function as$$\begin{array}{}\text{(10)}& \mathrm{E}\left[\frac{\mathrm{1}}{\mathrm{2}}{\mathrm{l}\mathrm{o}\mathrm{g}}_{\mathrm{2}}\left(\mathrm{1}+\gamma \right)\right]=\frac{\mathrm{1}}{\mathrm{2}\ln \mathrm{2}}{{\displaystyle \int }}_{\mathrm{0}}^{\infty }Ei\left(-s\right)\frac{\partial M_{\gamma }\left(s\right)}{\partial s}ds\end{array}$$where $Ei\left(-s\right)$ is the exponential integral function [9, eq.(8.211.1)]. Differentiating $M_{\gamma }\left(s\right)$ into (10) produces an integral form given by$$\begin{array}{}\text{(11)}& J_{\mathrm{2}}=-{{\displaystyle \int }}_{\mathrm{0}}^{\infty }Ei\left(-s\right){\left(s+{\beta }_d\right)}^{-\mathrm{1}-m_d}\times {\left(s+{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}\right)}^{l-m_s+k_{\mathrm{1}}-m_r+k_{\mathrm{2}}-\mathrm{1}/\mathrm{2}}ds\end{array}$$Since $J_{\mathrm{2}}$ can not be solved by traditional integral table, we have to find more sophisticated calculation method. Fortunately, the Fox’s $H$ function has been proven to be effective and widely used. Applying [14, eq.(2.9.5)], we express power function as Fox’s $H$ function given by$$\begin{array}{}\text{(12)}& {\left(s+{\beta }_d\right)}^{-\mathrm{1}-m_d}=\frac{{\beta }_d^{-\mathrm{1}-m_d}}{\Gamma \left(\mathrm{1}+m_d\right)}{H}_{\mathrm{1,1}}^{\mathrm{1,1}}\left[\frac{s}{{\beta }_d}\left|\begin{array}{c}\left(-m_d,\mathrm{1}\right)\\ \left(\mathrm{0,1}\right)\end{array}\right.\right]\end{array}$$Then by [9, eq.(8.359.1)], the exponential integral is expressed as$$\begin{array}{}\text{(13)}& -Ei\left(-s\right)=\Gamma \left(\mathrm{0},s\right)=s^{-\mathrm{1}/\mathrm{2}}{e}^{-s/\mathrm{2}}{W}_{-\mathrm{1}/\mathrm{2},\mathrm{0}}\left(s\right)\end{array}$$where $\Gamma \left(·,·\right)$ and $W_{·,·}\left(·\right)$ are the upper incomplete gamma function [9, eq.(8.350.2)] and the Whittaker function [9, eq.(9.222)]. According to the result in [15, eq.(2.6.1)], $J_{\mathrm{2}}$ can be given in a closed form in terms of bivariable Fox’s $H$ function. Finally, the ergodic capacity is written in $$\begin{array}{}\text{(14)}& \mathrm{E}\left[\frac{\mathrm{1}}{\mathrm{2}}{\mathrm{l}\mathrm{o}\mathrm{g}}_{\mathrm{2}}\left(\mathrm{1}+\gamma \right)\right]=\frac{\sqrt{\pi }}{\mathrm{2}\ln \mathrm{2}\Gamma \left(m_s\right)\Gamma \left(m_r\right)}{\displaystyle \sum }_{k_{\mathrm{1}}=\mathrm{0}}^{⌊m_s⌋}\hspace{0.17em}{\displaystyle \sum }_{k_{\mathrm{2}}=\mathrm{0}}^{⌊m_r⌋}{\displaystyle \sum }_{l=\mathrm{0}}^{⌊\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)/\mathrm{2}⌋}\frac{\left(⌊m_s⌋+k_{\mathrm{1}}\right)!\left(⌊m_r⌋+k_{\mathrm{2}}\right)!{\beta }_s^{m_s/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{1}}/\mathrm{2}}}{{\mathrm{4}}^{k_{\mathrm{1}}+k_{\mathrm{2}}}{k}_{\mathrm{1}}!k_{\mathrm{2}}!\left(⌊m_s⌋-k_{\mathrm{1}}\right)!\left(⌊m_r⌋-k_{\mathrm{2}}\right)!{\mathrm{2}}^{\mathrm{2}l}}\times \frac{{\beta }_r^{m_r/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{2}}/\mathrm{2}}\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)!{\left(-\mathrm{1}\right)}^l}{l!\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-\mathrm{2}l\right)!}{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{-m_s+k_{\mathrm{1}}-m_r+k_{\mathrm{2}}-\mathrm{1}}\times \frac{B\left(\mathrm{1},m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\mathrm{1}/\mathrm{2}\right)}{\Gamma \left(\mathrm{1}+m_d\right){\beta }_d}\times \left\{\left(m_d{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}+{\beta }_d\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\frac{\mathrm{1}}{\mathrm{2}}\right)\right)\times H_{\mathrm{2},\left(\mathrm{1}:\mathrm{1}\right),\mathrm{1},\left(\mathrm{1}:\mathrm{1}\right)}^{\mathrm{2,1},\mathrm{1,1},\mathrm{1}}\left[\begin{array}{c}{\beta }_d^{-\mathrm{1}}\\ {\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{-\mathrm{2}}\end{array}\left|\begin{array}{c}\left(\mathrm{1,1}\right),\left(\mathrm{1,1}\right)\\ \left(-m_d,\mathrm{1}\right);\left(\frac{\mathrm{1}}{\mathrm{2}}-m_s+k_{\mathrm{1}}-m_r+k_{\mathrm{2}}+l,\mathrm{1}\right)\\ \left(\mathrm{2,1}\right)\\ \left(\mathrm{0,1}\right);\left(\mathrm{0,1}\right)\end{array}\right.\right]+\left(m_d+m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\frac{\mathrm{1}}{\mathrm{2}}\right)\times H_{\mathrm{2},\left(\mathrm{1}:\mathrm{1}\right),\mathrm{1},\left(\mathrm{1}:\mathrm{1}\right)}^{\mathrm{2,1},\mathrm{1,1},\mathrm{1}}\left[\begin{array}{c}{\beta }_d^{-\mathrm{1}}\\ {\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{-\mathrm{2}}\end{array}\left|\begin{array}{c}\left(\mathrm{2,1}\right),\left(\mathrm{2,1}\right)\\ \left(-m_d,\mathrm{1}\right);\left(\frac{\mathrm{1}}{\mathrm{2}}-m_s+k_{\mathrm{1}}-m_r+k_{\mathrm{2}}+l,\mathrm{1}\right)\\ \left(\mathrm{3,1}\right)\\ \left(\mathrm{0,1}\right);\left(\mathrm{0,1}\right)\end{array}\right.\right]\right\}\end{array}$$where $H_{·,\left(·:·\right),·,\left(·:·\right)}^{·,·,·,·,·}\left(·\right)$ is the bivariable Fox’s $H$ function [15, eq.(2.2.1)].

3.3. Average SEP

For some simple modulation constellations, there is only one parameter carrying information such as phase, frequency, and amplitude. The error probability of these modulations is usually related to only one Gaussian $Q$ function, such as binary phase shift keying and pulse amplitude modulation. However, for some complex modulations, there are two or more parameters carrying signal information, so the error probability of data transmission involves the square of the Gaussian $Q$ function, such as quadrature amplitude modulation (QAM). The average SEP is expressed in a unified form as follows:$$\begin{array}{}\text{(15)}& P_e=\mathrm{E}\left[aQ\left(\sqrt{b\gamma }\right)\right]-\mathrm{E}\left[c{Q}^{\mathrm{2}}\left(\sqrt{b\gamma }\right)\right]\end{array}$$where $a$, $b$, and $c$ are constants, whose values are dependent on specific modulation constellation. For a simple modulation, only a single Gaussian $Q$ function is included in error probability and $c=\mathrm{0}$ is set in (15). Similar to the case of ergodic capacity, it is a little troublesome to calculate the error probability directly. Because of the complexity of the Gaussian $Q$ function, we start with the first part in (15). Capitalizing [16, eq.(33)], the average SEP can be rewritten in an alternative form as $$\begin{array}{}\text{(16)}& P_{e\mathrm{1}}=\mathrm{E}\left[aQ\left(\sqrt{b\gamma }\right)\right]=\frac{a}{\mathrm{2}}+\frac{a}{\mathrm{2}\sqrt{\pi }}{{\displaystyle \int }}_{\mathrm{0}}^{\infty }{G}_{\mathrm{2,2}}^{\mathrm{1,1}}\left[\frac{b}{\mathrm{2}s}\begin{array}{c}\left|\begin{array}{c}\mathrm{1,1}\\ \frac{\mathrm{1}}{\mathrm{2}},\mathrm{0}\end{array}\right.\end{array}\right]\times \frac{\partial M_{\gamma }\left(s\right)}{\partial s}ds\end{array}$$where $G\left(·\right)$ is Meijer’s G function. Utilizing standard integration by parts, after some mathematical rearrangements, we will encounter a type of integral as follows:$$\begin{array}{}\text{(17)}& J_{\mathrm{3}}={{\displaystyle \int }}_{\mathrm{0}}^{\infty }\frac{M_{\gamma }\left(s\right)}{s}{G}_{\mathrm{1,1}}^{\mathrm{1,0}}\left[\frac{b}{\mathrm{2}s}\left|\begin{array}{c}\mathrm{1}\\ \frac{\mathrm{1}}{\mathrm{2}}\end{array}\right.\right]ds\end{array}$$Substituting $M_{\gamma }\left(s\right)$ into (17) yields an integral given by $$\begin{array}{}\text{(18)}& J_{\mathrm{4}}={{\displaystyle \int }}_{\mathrm{0}}^{\infty }\frac{{\left(s+{\beta }_d\right)}^{-m_d}}{{\left(s+{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}\right)}^{m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\mathrm{1}/\mathrm{2}}}\times \frac{G_{\mathrm{1,1}}^{\mathrm{1,0}}\left[b/\mathrm{2}s\left|\begin{array}{c}\mathrm{1}\\ \mathrm{1}/\mathrm{2}\end{array}\right.\right]}{s}ds\end{array}$$Making a change of a variable $u=\mathrm{1}/s$ leads to $$\begin{array}{}\text{(19)}& J_{\mathrm{4}}={{\displaystyle \int }}_{\mathrm{0}}^{\infty }\frac{u^{m_d+m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\mathrm{3}/\mathrm{2}}{\left(\mathrm{1}+{\beta }_{d}u\right)}^{-m_d}}{{\left(\mathrm{1}+{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}u\right)}^{m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\mathrm{1}/\mathrm{2}}}\times G_{\mathrm{1,1}}^{\mathrm{1,0}}\left[\frac{b}{\mathrm{2}s}\left|\begin{array}{c}\mathrm{1}\\ \begin{array}{c}\frac{\mathrm{1}}{\mathrm{2}}\end{array}\end{array}\right.\right]ds\end{array}$$Similarly, in ergodic capacity case, by Fox’s $H$ functions, the power function is represented as$$\begin{array}{}\text{(20)}& {\left(\mathrm{1}+{\beta }_{d}u\right)}^{-m_d}=\frac{H_{\mathrm{1,1}}^{\mathrm{1,1}}\left[{\beta }_{d}u\left|\begin{array}{c}\left(\mathrm{1}-m_d,\mathrm{1}\right)\\ \left(\mathrm{0,1}\right)\end{array}\right.\right]}{\Gamma \left(m_d\right)}\end{array}$$Capitalizing the integral table [15, eq.(2.5.1)], $J_{\mathrm{4}}$ can be expressed in bivariable Fox’s function. Finally, substituting the integral result into (15) and solving the remaining algebraic operation yields average SEP containing only one single Gaussian $Q$ function given by $$\begin{array}{}\text{(21)}& P_{e\mathrm{1}}=\mathrm{E}\left[aQ\left(\sqrt{b\gamma }\right)\right]=\frac{a}{\mathrm{2}}+\frac{a}{\mathrm{2}\sqrt{\pi }}{{\displaystyle \int }}_{\mathrm{0}}^{\infty }{G}_{\mathrm{2,2}}^{\mathrm{1,1}}\left[\frac{b}{\mathrm{2}s}\left|\begin{array}{c}\mathrm{1,1}\\ \frac{\mathrm{1}}{\mathrm{2}},\mathrm{0}\end{array}\right.\right]\frac{\partial M_{\gamma }\left(s\right)}{\partial s}ds=\frac{a}{\mathrm{2}}+\frac{a}{\mathrm{2}\Gamma \left(m_s\right)\Gamma \left(m_r\right)}{\displaystyle \sum }_{k_{\mathrm{1}}=\mathrm{0}}^{⌊m_s⌋}\hspace{0.17em}{\displaystyle \sum }_{k_{\mathrm{2}}=\mathrm{0}}^{⌊m_r⌋}\hspace{0.17em}{\displaystyle \sum }_{l=\mathrm{0}}^{⌊\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)/\mathrm{2}⌋}\frac{\left(⌊m_s⌋+k_{\mathrm{1}}\right)!\left(⌊m_r⌋+k_{\mathrm{2}}\right)!{\beta }_s^{m_s/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{1}}/\mathrm{2}}}{{\mathrm{4}}^{k_{\mathrm{1}}+k_{\mathrm{2}}}{k}_{\mathrm{1}}!k_{\mathrm{2}}!\left(⌊m_s⌋-k_{\mathrm{1}}\right)!\left(⌊m_r⌋-k_{\mathrm{2}}\right)!{\mathrm{2}}^{\mathrm{2}l}}\times \frac{{\beta }_r^{m_r/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{2}}/\mathrm{2}}\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)!{\left(-\mathrm{1}\right)}^l}{l!\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-\mathrm{2}l\right)!}\Gamma \left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\frac{\mathrm{1}}{\mathrm{2}}\right){\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{1}-m_s+k_{\mathrm{1}}-m_r+k_{\mathrm{2}}}\times \left\{{{\displaystyle \int }}_{\mathrm{0}}^{\infty }{s}^{-\mathrm{1}}{\left(s+{\beta }_d\right)}^{-m_d}{\left(s+{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}\right)}^{-\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\mathrm{1}/\mathrm{2}\right)}{G}_{\mathrm{1,1}}^{\mathrm{1,0}}\left[\frac{b}{\mathrm{2}s}\left|\begin{array}{c}\mathrm{1}\\ \frac{\mathrm{1}}{\mathrm{2}}\end{array}\right.\right]ds-\sqrt{\pi }\right\}=\frac{a}{\mathrm{2}}+\frac{a}{\mathrm{2}\Gamma \left(m_s\right)\Gamma \left(m_r\right)\Gamma \left(m_d\right)}{\displaystyle \sum }_{k_{\mathrm{1}}=\mathrm{0}}^{⌊m_s⌋}\hspace{0.17em}{\displaystyle \sum }_{k_{\mathrm{2}}=\mathrm{0}}^{⌊m_r⌋}\hspace{0.17em}{\displaystyle \sum }_{l=\mathrm{0}}^{⌊\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)/\mathrm{2}⌋}\frac{\left(⌊m_s⌋+k_{\mathrm{1}}\right)!\left(⌊m_r⌋+k_{\mathrm{2}}\right)!{\beta }_s^{m_s/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{1}}/\mathrm{2}}}{{\mathrm{4}}^{k_{\mathrm{1}}+k_{\mathrm{2}}}{k}_{\mathrm{1}}!k_{\mathrm{2}}!\left(⌊m_s⌋-k_{\mathrm{1}}\right)!\left(⌊m_r⌋-k_{\mathrm{2}}\right)!{\mathrm{2}}^{\mathrm{2}l}}\times \frac{{\beta }_r^{m_r/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{2}}/\mathrm{2}}\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)!{\left(-\mathrm{1}\right)}^l}{l!\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-\mathrm{2}l\right)!}{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{1}-m_s+k_{\mathrm{1}}-m_r+k_{\mathrm{2}}}\times \left\{{\beta }_d^{m_d}{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\mathrm{1}/\mathrm{2}\right)}{\left(\frac{\mathrm{2}}{b}\right)}^{m_d+m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\mathrm{1}/\mathrm{2}}\times H_{\mathrm{1},\left(\mathrm{1}:\mathrm{1}\right),\mathrm{1},\left(\mathrm{1}:\mathrm{1}\right)}^{\mathrm{1,1},\mathrm{1,1},\mathrm{1}}\left[\begin{array}{c}\frac{\mathrm{2}{\beta }_d}{b}\\ \frac{\mathrm{2}{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}}{b}\end{array}\left|\begin{array}{c}\left(m_d+m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l,\mathrm{1}\right)\\ \left(\mathrm{1}-m_d,\mathrm{1}\right);\left(\frac{\mathrm{3}}{\mathrm{2}}-m_s+k_{\mathrm{1}}-m_r+k_{\mathrm{2}}+l,\mathrm{1}\right)\\ \left(m_d+m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l+\frac{\mathrm{1}}{\mathrm{2}},\mathrm{1}\right)\\ \left(\mathrm{0,1}\right);\left(\mathrm{0,1}\right)\end{array}\right.\right]-\sqrt{\pi }\Gamma \left(m_d\right)\Gamma \left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\frac{\mathrm{1}}{\mathrm{2}}\right)\right\}\end{array}$$

Next, we focus our attention on the second part in (15) involving the square of Gaussian $Q$ function. According to moment generating function approach, we have$$\begin{array}{}\text{(22)}& P_{e\mathrm{2}}=\mathrm{E}\left[c{Q}^{\mathrm{2}}\left(\sqrt{b\gamma }\right)\right]=\frac{c}{\pi }{{\displaystyle \int }}_{\mathrm{0}}^{\pi /\mathrm{4}}{M}_{\gamma }\left(\frac{b}{\mathrm{2}\hspace{0.17em}{\mathrm{s}\mathrm{i}\mathrm{n}}^{\mathrm{2}}\theta }\right)d\theta \end{array}$$Utilizing [17, eq.$(21)$] and after some mathematical manipulations, $P_{e\mathrm{2}}$ is written in $$\begin{array}{}\text{(23)}& P_{e\mathrm{2}}=\mathrm{E}\left[c{Q}^{\mathrm{2}}\left(\sqrt{b\gamma }\right)\right]=\frac{c{\beta }_d^{m_d}}{\mathrm{4}\sqrt{\pi }\Gamma \left(m_s\right)\Gamma \left(m_r\right)}{\displaystyle \sum }_{k_{\mathrm{1}}=\mathrm{0}}^{⌊m_s⌋}\hspace{0.17em}{\displaystyle \sum }_{k_{\mathrm{2}}=\mathrm{0}}^{⌊m_r⌋}{\displaystyle \sum }_{l=\mathrm{0}}^{⌊\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)/\mathrm{2}⌋}\frac{\left(⌊m_s⌋+k_{\mathrm{1}}\right)!\left(⌊m_r⌋+k_{\mathrm{2}}\right)!{\beta }_s^{m_s/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{1}}/\mathrm{2}}}{{\mathrm{4}}^{k_{\mathrm{1}}+k_{\mathrm{2}}}{k}_{\mathrm{1}}!k_{\mathrm{2}}!\left(⌊m_s⌋-k_{\mathrm{1}}\right)!\left(⌊m_r⌋-k_{\mathrm{2}}\right)!{\mathrm{2}}^{\mathrm{2}l}}\times \frac{{\beta }_r^{m_r/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{2}}/\mathrm{2}}\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)!{\left(-\mathrm{1}\right)}^l{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-\mathrm{2}l}}{l!\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-\mathrm{2}l\right)!\left(m_d+m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l\right)}\times \Gamma \left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\frac{\mathrm{1}}{\mathrm{2}}\right){\left(b+{\beta }_d\right)}^{-m_d}{\left(b+{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}\right)}^{-\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\mathrm{1}/\mathrm{2}\right)}\times F_D^{\left(\mathrm{3}\right)}\left[\mathrm{1},m_d,m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\frac{\mathrm{1}}{\mathrm{2}},\mathrm{1};m_d+m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l+\mathrm{1};\frac{b+\mathrm{2}{\beta }_d}{\mathrm{2}b+\mathrm{2}{\beta }_d},\frac{b+\mathrm{2}{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}}{\mathrm{2}b+\mathrm{2}{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}},\frac{\mathrm{1}}{\mathrm{2}}\right]\end{array}$$where $F_D^{\left(n\right)}$ is Lauricella’s function of the fourth kind [18, eq.$(15)$]. Finally, the average SEP is fully expressed as $P_e=P_{e\mathrm{1}}-P_{e\mathrm{2}}$.

In (21) and (23), we derive the SEP for the AF relay system, but it is possible to find the SEP of a little bit more generalized case. Recently, some researchers pointed out that white Gaussian noise is not a suitable noise model in some special scenarios, such as wireless sensor networks and underwater communication [19]. The generalized Gaussian white noise model is the best choice. However, few works have considered SEP in the context of generalized noise environment in a relay system. Under the AWGGN environment, the average SEP is given by$$\begin{array}{}\text{(24)}& P_e=\mathrm{E}\left[a{Q}_{\alpha }\left(\sqrt{b\gamma }\right)\right]\end{array}$$where $Q_{\alpha }\left(·\right)$ is the generalized $Q$ function defined as$$\begin{array}{}\text{(25)}& Q_{\alpha }\left(x\right)=\frac{\Gamma \left(\mathrm{1}/\alpha ,{\left|{\Lambda }_{\mathrm{0}}x\right|}^{\alpha }\right)}{\mathrm{2}\Gamma \left(\mathrm{1}/\alpha \right)}\end{array}$$and ${\Lambda }_{\mathrm{0}}=\sqrt{\Gamma \left(\mathrm{3}/\alpha \right)/\Gamma \left(\mathrm{1}/\alpha \right)}$. Different $\alpha$ parameters correspond to different noise environments. The corresponding relationship between $\alpha$ parameter and noise environment is summarized in [19]. The generalized SEP analysis is a complicated problem due to the involvement of generalized $Q$ function. To the best of our knowledge, this is the very first effort for this considered setup. The major challenge for this extension arises from the generalized $Q$ function.

Substituting $Q_{\alpha }\left(x\right)$ into the average SEP yields a type of integral:$$\begin{array}{}\text{(26)}& J_{\mathrm{5}}={{\displaystyle \int }}_{\mathrm{0}}^{\infty }\Gamma \left(\frac{\mathrm{1}}{\alpha },{\left|{\Lambda }_{\mathrm{0}}\sqrt{bz}\right|}^{\alpha }\right)e^{-{\beta }_{d}z}{z}^n\times {\hspace{0.17em}}_{\mathrm{1}}{F}_{\mathrm{1}}\left[v;\nu \left({\beta }_d-{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}\right)z\right]dz\end{array}$$By Fox’s $H$ function, the incomplete gamma function and hypergeometric function are, respectively, expressed as$$\begin{array}{}\text{(27)}& \Gamma \left(\frac{\mathrm{1}}{\alpha },{\left|{\Lambda }_{\mathrm{0}}\sqrt{bz}\right|}^{\alpha }\right)=G_{\mathrm{1,2}}^{\mathrm{2,0}}\left[{\left|\sqrt{bz}{\Lambda }_{\mathrm{0}}\right|}^{\alpha }\begin{array}{c}\left|\begin{array}{c}\mathrm{1}\\ \frac{\mathrm{1}}{\alpha },\mathrm{0}\end{array}\right.\end{array}\right]=\frac{\mathrm{2}}{\alpha }{H}_{\mathrm{1,2}}^{\mathrm{2,0}}\left[bz{\Lambda }_{\mathrm{0}}^{\mathrm{2}}\left|\begin{array}{c}\left(\mathrm{1},\frac{\mathrm{2}}{\alpha }\right)\\ \left(\frac{\mathrm{1}}{\alpha },\frac{\mathrm{2}}{\alpha }\right),\left(\mathrm{0},\frac{\mathrm{2}}{\alpha }\right)\end{array}\right.\right]\text{(28)}& {}_{\mathrm{1}}{F}_{\mathrm{1}}\left[v;\nu \left({\beta }_d-{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}\right)z\right]=\frac{\Gamma \left(\nu \right)}{\Gamma \left(v\right)}\times H_{\mathrm{1,2}}^{\mathrm{1,1}}\left[{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}-{\beta }_d\left|\begin{array}{c}\left(\mathrm{1}-v,\mathrm{1}\right)\\ \left(\mathrm{0,1}\right),\left(\mathrm{1}-\nu ,\mathrm{1}\right)\end{array}\right.\right]\end{array}$$Utilizing [15, eq.(2.6.2)], $J_{\mathrm{5}}$ can be solved in a closed form by bivariable Fox’s function. Thus the average SEP is given by $$\begin{array}{}\text{(29)}& P_e=\frac{a\sqrt{\pi }}{\alpha \Gamma \left(\mathrm{1}/\alpha \right)\Gamma \left(m_s\right)\Gamma \left(m_r\right)}{\displaystyle \sum }_{k_{\mathrm{1}}=\mathrm{0}}^{⌊m_{\mathrm{s}}⌋}\hspace{0.17em}{\displaystyle \sum }_{k_{\mathrm{2}}=\mathrm{0}}^{⌊m_r⌋}{\displaystyle \sum }_{l=\mathrm{0}}^{⌊\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)/\mathrm{2}⌋}\frac{\left(⌊m_s⌋+k_{\mathrm{1}}\right)!\left(⌊m_r⌋+k_{\mathrm{2}}\right)!{\beta }_s^{m_s/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{1}}/\mathrm{2}}}{{\mathrm{4}}^{k_{\mathrm{1}}+k_{\mathrm{2}}}{k}_{\mathrm{1}}!k_{\mathrm{2}}!\left(⌊m_s⌋-k_{\mathrm{1}}\right)!\left(⌊m_r⌋-k_{\mathrm{2}}\right)!{\mathrm{2}}^{\mathrm{2}l}}\times \frac{{\beta }_r^{m_r/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{2}}/\mathrm{2}}\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)!{\left(-\mathrm{1}\right)}^l}{l!\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-\mathrm{2}l\right)!}{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-\mathrm{2}l}{\beta }_d^{\mathrm{1}/\mathrm{2}-m_s+k_{\mathrm{1}}-m_r+k_{\mathrm{2}}+l}\times H_{\mathrm{1},\left(\mathrm{1}:\mathrm{1}\right),\mathrm{0},\left(\mathrm{2}:\mathrm{2}\right)}^{\mathrm{1,0},\mathrm{1,2},\mathrm{1}}\left[\begin{array}{c}\frac{b{\Lambda }_{\mathrm{0}}^{\mathrm{2}}}{{\beta }_d}\\ \frac{{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}}{{\beta }_d}-\mathrm{1}\end{array}\left|\begin{array}{c}\left(m_d+m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\frac{\mathrm{1}}{\mathrm{2}},\mathrm{1}\right)\\ \left(\mathrm{1},\frac{\mathrm{2}}{\alpha }\right);\left(\frac{\mathrm{3}}{\mathrm{2}}-m_s+k_{\mathrm{1}}-m_r+k_{\mathrm{2}}+l,\mathrm{1}\right)\\ -\\ \left(\frac{\mathrm{1}}{\alpha },\frac{\mathrm{2}}{\alpha }\right),\left(\mathrm{0},\frac{\mathrm{2}}{\alpha }\right);\left(\mathrm{0,1}\right),\left(\frac{\mathrm{3}}{\mathrm{2}}-m_s+k_{\mathrm{1}}-m_r+k_{\mathrm{2}}+l-m_d,\mathrm{1}\right)\end{array}\right.\right]\end{array}$$Note (29) is reasonable only when ${\beta }_d<{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}$ due to the convergence property of bivariable Fox’s function.

Similarly, when ${\beta }_d>{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}$, the average SEP is given by $$\begin{array}{}\text{(30)}& P_e=\frac{a\sqrt{\pi }{\beta }_d^{m_d}}{\alpha \Gamma \left(\mathrm{1}/\alpha \right)\Gamma \left(m_s\right)\Gamma \left(m_r\right)\Gamma \left(m_d\right)}{\displaystyle \sum }_{k_{\mathrm{1}}=\mathrm{0}}^{⌊m_s⌋}\hspace{0.17em}{\displaystyle \sum }_{k_{\mathrm{2}}=\mathrm{0}}^{⌊m_r⌋}{\displaystyle \sum }_{l=\mathrm{0}}^{⌊\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)/\mathrm{2}⌋}\frac{\left(⌊m_s⌋+k_{\mathrm{1}}\right)!\left(⌊m_r⌋+k_{\mathrm{2}}\right)!{\beta }_s^{m_s/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{1}}/\mathrm{2}}}{{\mathrm{4}}^{k_{\mathrm{1}}+k_{\mathrm{2}}}{k}_{\mathrm{1}}!k_{\mathrm{2}}!\left(⌊m_s⌋-k_{\mathrm{1}}\right)!\left(⌊m_r⌋-k_{\mathrm{2}}\right)!{\mathrm{2}}^{\mathrm{2}l}}\times \frac{{\beta }_r^{m_r/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{2}}/\mathrm{2}}\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)!{\left(-\mathrm{1}\right)}^l}{l!\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-\mathrm{2}l\right)!}{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{1}-\mathrm{2}{m}_d-m_s+k_{\mathrm{1}}-m_r+k_{\mathrm{2}}}\Gamma \left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\frac{\mathrm{1}}{\mathrm{2}}\right)\times H_{\mathrm{1},\left(\mathrm{1}:\mathrm{1}\right),\mathrm{0},\left(\mathrm{2}:\mathrm{2}\right)}^{\mathrm{1,0},\mathrm{1,2},\mathrm{1}}\left[\begin{array}{c}\frac{b{\Lambda }_{\mathrm{0}}^{\mathrm{2}}}{{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}}\\ \frac{{\beta }_d}{{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}}-\mathrm{1}\end{array}\left|\begin{array}{c}\left(m_d+m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\frac{\mathrm{1}}{\mathrm{2}},\mathrm{1}\right)\\ \left(\mathrm{1},\frac{\mathrm{2}}{\alpha }\right);\left(\mathrm{1}-m_d,\mathrm{1}\right)\\ -\\ \left(\frac{\mathrm{1}}{\alpha },\frac{\mathrm{2}}{\alpha }\right),\left(\mathrm{0},\frac{\mathrm{2}}{\alpha }\right);\left(\mathrm{0,1}\right),\left(\frac{\mathrm{3}}{\mathrm{2}}-m_s+k_{\mathrm{1}}-m_r+k_{\mathrm{2}}+l-m_d,\mathrm{1}\right)\end{array}\right.\right]\end{array}$$In particular, when ${\beta }_d={\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{\mathrm{2}}$, the average SEP is simplified to $$\begin{array}{}\text{(31)}& P_{e\mathrm{1}}=\frac{\sqrt{\pi }a}{\alpha \Gamma \left(\mathrm{1}/\alpha \right)\Gamma \left(m_s\right)\Gamma \left(m_r\right)\Gamma \left(m_d\right)}{\displaystyle \sum }_{k_{\mathrm{1}}=\mathrm{0}}^{⌊m_s⌋}\hspace{0.17em}{\displaystyle \sum }_{k_{\mathrm{2}}=\mathrm{0}}^{⌊m_r⌋}{\displaystyle \sum }_{l=\mathrm{0}}^{⌊\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)/\mathrm{2}⌋}\frac{\left(⌊m_s⌋+k_{\mathrm{1}}\right)!\left(⌊m_r⌋+k_{\mathrm{2}}\right)!{\beta }_s^{m_s/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{1}}/\mathrm{2}}}{{\mathrm{4}}^{k_{\mathrm{1}}+k_{\mathrm{2}}}{k}_{\mathrm{1}}!k_{\mathrm{2}}!\left(⌊m_s⌋-k_{\mathrm{1}}\right)!\left(⌊m_r⌋-k_{\mathrm{2}}\right)!{\mathrm{2}}^{\mathrm{2}l}}\times \frac{{\beta }_r^{m_r/\mathrm{2}-\mathrm{1}/\mathrm{4}-k_{\mathrm{2}}/\mathrm{2}}\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}\right)!{\left(-\mathrm{1}\right)}^l}{l!\left(m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-\mathrm{2}l\right)!}{\left(\sqrt{{\beta }_s}+\sqrt{{\beta }_r}\right)}^{m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-\mathrm{2}l}{\beta }_d^{\mathrm{1}/\mathrm{2}-m_s+k_{\mathrm{1}}-m_r+k_{\mathrm{2}}+l}\times B\left(m_d,m_s-k_{\mathrm{1}}+m_r-k_{\mathrm{2}}-l-\frac{\mathrm{1}}{\mathrm{2}}\right)H_{\mathrm{2,2}}^{\mathrm{2,1}}\left[\frac{b{\Lambda }_{\mathrm{0}}^{\mathrm{2}}}{{\beta }_d}\left|\begin{array}{c}\left(\frac{\mathrm{3}}{\mathrm{2}}-m_d-m_s+k_{\mathrm{1}}-m_r+k_{\mathrm{2}}+l,\mathrm{1}\right),\left(\mathrm{1},\frac{\mathrm{2}}{\alpha }\right)\\ \left(\frac{\mathrm{1}}{\alpha },\frac{\mathrm{2}}{\alpha }\right),\left(\mathrm{0},\frac{\mathrm{2}}{\alpha }\right)\end{array}\right.\right]\end{array}$$With knowing $P_e$ in (29), (30), and (31), we can obtain the average SEP for a variety of noise types.

Hitherto, all the performance expressions in this paper are established in the closed forms. These formulas significantly reduce the computational complexity and also greatly facilitate the general performance study in a convenient manner. As mentioned earlier, in the existing literature, the performance expressions either remain in an integral form with an integrand of moment generating function or only provide a lower bound and/or an upper bound.

4. Simulation Results

Numerical performance results are provided through both closed form analytical expressions and simulations. For simplicity, $p_s=p_r$ is assumed and $p_s/{\sigma }_r^{\mathrm{2}}$ and $p_r/{\sigma }_d^{\mathrm{2}}$ are defined as average SNR per hop. The channel gain is normalized to unity unless otherwise noted.

Figure 2 shows the tightness of the analytical curves on the MRC outage performance compared with simulation results over the whole range of SNR, where weak SD means poor quality of S$\to$D link; i.e., $\mathrm{E}\left[{\left|h_d\right|}^{\mathrm{2}}\right]/\mathrm{E}\left[{\left|h_s\right|}^{\mathrm{2}}\right]$ = -10dB. Due to the poor quality of direct channel, the outage probability of weak SD is higher than that of good quality. The diversity order can be obtained through a wide range of channel settings. For example, in case of $m_s=\mathrm{0.5}$, $m_r=\mathrm{1.5}$, and $m_d=\mathrm{0.1234}$ with weak SD, $P_e$ = 0.0301 at 28dB and $P_e$ = 0.0226 at 30dB. Thus, the diversity impact on a high SNR slope of the outage curve is quantified by $\mathrm{10}{\log }_{\mathrm{10}}\left(\mathrm{0.0301}/\mathrm{0.0226}\right)/\left(\mathrm{30}-\mathrm{28}\right)=\mathrm{0.6223}\approx \min \left(m_s,m_r\right)+m_d$. Simulation results demonstrate that MRC attains a diversity order of $m_d+\min \left(m_s,m_r\right)$.

[figure omitted; refer to PDF]

Figure 3 compares the first-order moment of the SNR with various fading parameter settings. We can see that the analytical average SNR accurately matches simulation result in all fading parameters. A constant gap exists between the weak SD channel and the good channel due to the poor quality in weak SD channel.

[figure omitted; refer to PDF]

Figure 4 shows the ergodic capacity. As expected, the theoretical curves precisely match with simulation results in all SNRs. In the high SNR regime, the capacity curves demonstrate clearly the multiplexing gain performance. For example, when $m_s=m_r=\mathrm{0.5}$ and $m_d=1.5678$, the ergodic capacities are 4.6239 at 28dB and 4.9554 at 30dB, respectively. So the multiplexing gain is quantified by $\mathrm{10}\hspace{0.17em}{\log }_{\mathrm{10}}\mathrm{2}\times \left(\mathrm{4.9554}-\mathrm{4.6239}\right)/\left(\mathrm{30}-\mathrm{28}\right)=\mathrm{0.4990}\approx \mathrm{1}/\mathrm{2}$.

[figure omitted; refer to PDF]

Figure 5 depicts the average SEP of 4QAM as a function of SNR. And, as a comparison, the SEP of the higher order constellation is shown in Figure 6, where $m_s=\mathrm{0.5}$, $m_r=\mathrm{1.5}$, and $m_d=\mathrm{1.5678}$. From both figures, an accurate diversity order agreement between analysis and simulation is found in the high SNR regime. This can be explained by the fact that outage probability and average SEP both achieve the same diversity order $m_d+\min \left(m_s,m_r\right)$. In Figure 6, all SEP curves follow the same downward trend and become approximately parallel lines in the high SNR region, indicating the same diversity order, independent of the specific constellation.

[figure omitted; refer to PDF] [figure omitted; refer to PDF]

5. Conclusion

This paper studies the performance of AF relaying with MRC in Nakagami-$m$ channels. New closed form formulas are derived for CDF, moment generating function, outage probability, higher order moments of the SNR, ergodic capacity, and average SEP and greatly facilitate the general performance study in a convenient manner in such a system. The theoretical expressions coincide with the simulation results precisely under various system settings.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (Grants nos. 61761030, 61661028, 61661032, 61861017, and 61463035) and by the China Postdoctoral Science Foundation (Grant no. 2017M622103).

References

[1] D. Lee, "SER of TAS-MRC with relay and user selection in MIMO-relay systems over non-identical nakagami fading channels," IEEE Communications Letters, vol. 21 no. 7, pp. 1645-1648, DOI: 10.1109/LCOMM.2017.2686879, 2017.

[2] Q. Wang, Y. Jing, "Performance analysis and scaling law of MRC/MRT relaying with CSI error in multi-pair massive MIMO systems," IEEE Transactions on Wireless Communications, vol. 16 no. 9, pp. 5882-5896, DOI: 10.1109/TWC.2017.2717399, 2017.

[3] M. Di Renzo, M. Iezzi, F. Graziosi, "Error performance and diversity analysis of multi-source multi-relay wireless networks with binary network coding and cooperative MRC," IEEE Transactions on Wireless Communications, vol. 12 no. 6, pp. 2883-2903, DOI: 10.1109/TWC.2013.042413.121194, 2013.

[4] N. H. Vien, H. H. Nguyen, "Performance analysis of fixed-gain amplify-and-forward relaying with MRC," IEEE Transactions on Vehicular Technology, vol. 59 no. 3, pp. 1544-1552, DOI: 10.1109/TVT.2010.2040755, 2010.

[5] Y. Cao, C. Tellambura, "Outage analysis of ZFB-MRT/MRC underlay two-way relay systems," IEEE Communications Letters, vol. 19 no. 6, pp. 1049-1052, DOI: 10.1109/LCOMM.2015.2416175, 2015.

[6] L. Yang, Q. T. Zhang, "Outage performance of MIMO relay channels with maximal ratio transmission," IEEE Electronics Letters, vol. 45 no. 5, pp. 273-275, DOI: 10.1049/el:20093231, 2009.

[7] A. M. K., M. R. Bhatnagar, "Maximal ratio transmission in af mimo relay systems over nakagami-fading channels," IEEE Transactions on Vehicular Technology, vol. 64 no. 5, pp. 1895-1903, DOI: 10.1109/TVT.2014.2334631, 2015.

[8] J. Yang, P. Fan, T. Q. Duong, X. Lei, "Exact performance of two-way af relaying in nakagami-m fading environment," IEEE Transactions on Wireless Communications, vol. 10 no. 3, pp. 980-987, DOI: 10.1109/TWC.2011.011111.101141, 2011.

[9] I. S. Gradshteyn, I. M. Ryzhik, Table of Integrals, Series and Products, 2007.

[10] A. P. Prudnikov, Y. A. Brychkov, O. I. Marichev, Integrals and Series, vol. 5, 1992.

[11] J. Si, Z. Li, H. Huang, J. Chen, R. Gao, "Capacity analysis of cognitive relay networks with the PU's interference," IEEE Communications Letters, vol. 16 no. 12, pp. 2020-2023, DOI: 10.1109/LCOMM.2012.111412.121978, 2012.

[12] A. Abdaoui, S. S. Ikki, M. H. Ahmed, E. Chtelet, "On the performance analysis of a MIMO-relaying scheme with space–time block codes," IEEE Transactions on Vehicular Technology, vol. 59 no. 7, pp. 3604-3609, DOI: 10.1109/TVT.2010.2050791, 2010.

[13] F. Yilmaz, M.-S. Alouini, "A unified MGF-based capacity analysis of diversity combiners over generalized fading channels," IEEE Transactions on Communications, vol. 60 no. 3, pp. 862-875, DOI: 10.1109/TCOMM.2012.020912.100766, 2012.

[14] A. A. Kilbas, M. Saigo, H-Transforms: Theory and Applications,DOI: 10.1201/9780203487372, 2004.

[15] A. M. Mathai, R. K. Saxena, The H-Function with Applications in Statistics and other Disciplines, 1978.

[16] F. Yilmaz, M.-S. Alouini, "A novel unified expression for the capacity and bit error probability of wireless communication systems over generalized fading channels," IEEE Transactions on Communications, vol. 60 no. 7, pp. 1862-1876, DOI: 10.1109/TCOMM.2012.062512.110846, 2012.

[17] S. Sreng, B. Escrig, M.-L. Boucheret, "Exact symbol error probability of hybrid/integrated satellite-terrestrial cooperative network," IEEE Transactions on Wireless Communications, vol. 12 no. 3, pp. 1310-1319, DOI: 10.1109/TWC.2013.013013.120899, 2013.

[18] V. A. Thomas, S. Kumar, S. Kalyani, M. El-Hajjar, "Error vector magnitude analysis of fading SIMO channels relying on MRC reception," IEEE Transactions on Communications, vol. 64 no. 4, pp. 1786-1797, DOI: 10.1109/tcomm.2016.2530797, 2016.

[19] H. Soury, F. Yilmaz, M.-S. Alouini, "Error rates of M-PAM and M-QAM in generalized fading and generalized gaussian noise environments," IEEE Communications Letters, vol. 17 no. 10, pp. 1932-1935, DOI: 10.1109/LCOMM.2013.081913.131409, 2013.