Two-Stage Method Based on Local Polynomial Fitting for a Linear Heteroscedastic Regression Model and Its Application in Economics

Abstract

We introduce the extension of local polynomial fitting to the linear heteroscedastic regression model. Firstly, the local polynomial fitting is applied to estimate heteroscedastic function, then the coefficients of regression model are obtained by using generalized least squares method. One noteworthy feature of our approach is that we avoid the testing for heteroscedasticity by improving the traditional two-stage method. Due to nonparametric technique of local polynomial estimation, we do not need to know the heteroscedastic function. Therefore, we can improve the estimation precision, when the heteroscedastic function is unknown. Furthermore, we focus on comparison of parameters and reach an optimal fitting. Besides, we verify the asymptotic normality of parameters based on numerical simulations. Finally, this approach is applied to a case of economics, and it indicates that our method is surely effective in finite-sample situations.

(ProQuest: ... denotes non-US-ASCII text omitted.)

Recommended by M. De la Sen

School of Mathematics and Statistics, Chongqing University of Technology, Chongqing 400054, China

Received 31 October 2011; Accepted 2 January 2012

1. Introduction

The heteroscedasticity in classical linear regression model is defined by the variances of random items which are not the same for different explanatory variables and observations. Heteroscedasticity often occurs in data sets in which there is a wide disparity between the largest and smallest observed values. The larger the disparity between the size of observations in a sample, the larger the likelihood that the error term observations associated with them will have different variances and therefore be heteroscedastic. That is, we would expect that the error term distribution for very large observations might have a large variance, but the error term distribution for small observations might have a small variance. Besides, researchers have observed that heteroscedasticity is usually found in cross-sectional data rather than in time series data [1, 2]. In cross-sectional data we generally deal with members of a population at a given point in time, such as individual consumers or their families; firms; industries; or geographical subdivisions, such as small, medium, or large firms, or low, medium, or high income. Besides, this study is also more common in economics, such as the relationship of sale and research and development, and profit in a certain year.

When there is a heteroscedasticity in a linear regression model [3-6], we can apply parametric methods, also the heteroscedastic approaches can be applied, such as Park test, White test and so on. Another heteroscedastic method is shown in [7]. When parametric approaches are applied, estimations of parameters we obtained by ordinary least squares estimation (OLS) are still linear and unbiased. However, the efficiency is bad [8-10]. This could lead to a uncorrect statistical diagnosis for the parameters' significance test. Similarly, it unecessarily enlarges the confidence interval when we estimate the parameter interval. Besides, the accuracy of predictive value may lower when we estimate with the regression model that we obtained. In order to solve the problem above, we can use generalized least squares estimation (GLS) when the covariance matrix of the random items is known. If it is unknown, we usually use two-stage least squares estimate, that is, we first estimate variances of the residual error, and then the generalized least squares estimator is used to obtain the coefficients of the model by using the estimate of variances of the random items [11, 12]. However, the traditional estimation method is that we suppose the residual error variances as a certain parametric model. In this paper, we try to applying local polynomial fitting to random item variances as the first step, and then GLS is used to estimate the coefficients of the model. A problem cannot be neglected is that the existence of inverse of regressor matrix transpose multiplied by regressor matrix is necessary for the existence of the unique estimator when applying the least squares method or their generalized versions. In the generalized versions, because kernel function is a symmetric probability density function with bounded support, the weighted matrix based on kernel function is positive definite, generally being also symmetric as done in usual theoretical and practical selections. In other words, the regressor matrix has to be full column rank and weighted matrix has to be nonsingular. The issues have been discussed in many papers used in theoretical issues and many applications, see [13-17]. On the one hand, because of local polynomial fitting's various nice statistical properties, the estimations obtained with this technology also possess the same good statistical properties [12, 18, 19]. On the other hand, we exploit a heteroscedastic regression model rather than the artificial structure of heteroscedasticity. Then, we can directly get the heteroscedastic function based on the nonparametric technique, which shows the relationship between variance function of random items and explanatory variables from regression results. Thus, it is unnecessary to test heteroscedasticity of the model. Particularly, the estimated value by local polynomial fitting is more accurate than that by the traditional method. Besides, we study variance function fitting when parameters change and reach the optimal parameters. Finally, a case of economics is cited in order to show that our method is indeed effective in finite-sample situations.

The rest of this paper is organized as follows. In Section 2 we construct local polynomial regression: in Section 2.1 we talk about local polynomial fitting, in Section 2.2 we study estimation with Parameters selections. Section 3 contains two-stage method with local polynomial fitting. In Section 4, we do the simulations on a given model and study the fittings under different parameters. In Section 5, we collect some real data and use the local polynomial estimating the coefficients. We draw the results in Section 6.

2. Local Polynomial Regression

Local polynomial regression [20-23] is a widely used nonparametric technique. Local polynomial fitting is an attractive method both from theoretical and practical point of view. Local polynomial method has a small mean squared error compared with the Nadaraya-Watson estimator which leads to an undesirable form of the bias and the Gasser-Muller estimator which has to pay a price in variance when dealing with a random design model. Local polynomial fitting also has other advantages. The method adapts to various types of designs such as random and fixed designs, highly clustered and nearly uniform designs. Furthermore, there is an absence of boundary effects: the bias at the boundary stays automatically of the same order as the interior, without use of specific boundary kernels. The local polynomial approximation approach is appealing on general scientific grounds: the least squares principle to be applied opens the way to a wealth of statistical knowledge and thus easy generalizations. In this section, we briefly outline the idea of the extension of local polynomial fitting to linear regression.

2.1. Local Polynomial Fitting

Consider the bivariate data (_X1 ,_Y1 ),...,(_Xn ,_Yn ) , which form an independent and identically distributed sample from a population (X,Y) . Of interest is to estimate the regression function m(_x0 )=E(Y|"X=_x0 ) and its derivatives ^{m[variant prime]} (_x0 ),^{m[variant prime][variant prime]} (_x0 ),...,^m(p) (_x0 ) . To help us understand the estimation methodology, we can regard the data as being generated from the model [figure omitted; refer to PDF] where E([varepsilon])=0 , Var ([varepsilon])=1 , and X and [varepsilon] are independent. However, this location-scale model assumption is not necessary for our development, but is adopted to provide intuitive understanding. We always denote the conditional variance of Y given X=_x0 by _σ2 (_x0 ) and the marginal density of X , that is, the design density, by f(·) .

Suppose that the (p+1)th derivative of m(x) at the point _x0 exists. We then approximate the unknown regression function m(x) locally by a polynomial of order p . A Taylor expansion gives, for x in a neighborhood of _x0 , [figure omitted; refer to PDF] This polynomial is fitted locally by a weighted least squares regression problem: minimize [figure omitted; refer to PDF] where h is a bandwidth controlling the size of the local neighborhood, and _Kh (·)=K(·/h)/h with K a kernel function assigning weights to each datum point. Throughout this paper we assume that K is a symmetric probability density function with bounded support, although this technical assumption can be relaxed significantly [24-26].

Denote by _β...j , j=0,...,p , the solution to the least squares problem (2.3). It is clear from the Taylor expansion in (2.2) that _m...ν (_x0 )=ν!_β...ν is an estimator for ^m(ν) (_x0 ),ν=0,1,...,p . To estimate the entire function ^m(ν) (·) we solve the above weighted least squares problem for all points _x0 in the domain of interest.

It is more convenient to work with matrix notation. Denote by X the design matrix of problem (2.3): [figure omitted; refer to PDF] and let [figure omitted; refer to PDF] Further, let W be the n×n diagonal matrix of weights: [figure omitted; refer to PDF] Then the weighted least squares problem (2.3) can be written as [figure omitted; refer to PDF] with β=^{(_β0 ,...,_βp )T} . The solution vector is provided by weighted least squares theory and is given by [figure omitted; refer to PDF] where the regressor matrix X has to be full column rank and the weighted matrix W has to be nonsingular. If such an inverse in (2.8) does not exist, a method of ridge regression can be adopted to solve this problem [27]. Furthermore, we can get the estimation m...(_x0 ) , [figure omitted; refer to PDF] where _E1 is a column vector (the same size of β ) with the first element equal to 1, and the rest equal to zero, that is, _E1 =_{(1,0,...,0)1×(p+1)} .

Computing the β... will suffer from large computational cost. We can use the recursive least squared method to reduce the computation complexity, and it is very powerful especially in the local polynomial fitting problems. There are several important issues about the bandwidth, the order of local polynomial function and the kernel function which have to be discussed. The three problems will be presented in Section 2.2.

2.2. Parameters Selections

To implement the local polynomial estimator, one need to choose the order p , the kernel K and the bandwidth h . These parameters are of course related to each other.

First of all, the choice of the bandwidth parameter h is considered, which plays a rather crucial role. A too large bandwidth under-parametrizes the regression function [28, 29], causing a large modeling bias, while a too small bandwidth over-parametrizes the unknown function and results in noisy estimates. The basic idea is to find a bandwidth h that minimizes the estimated mean integrated square error (MISE): [figure omitted; refer to PDF] where X=(_X1 ,...,_Xp ) , the asymptotic bias and the asymptotic variance are denoted in Lemma A.1 of the appendix. Then we can find an asymptotically optimal constant bandwidth given by [figure omitted; refer to PDF] where, _cν,p (K) is a constant which relates to the kernel function and the order of the local polynomial, f(x) , is the density function of m(·) . However, this ideal bandwidth is not directly usable since it depends on unknown functions.

Another issue in local polynomial fitting is the choice of the order of the polynomial. Since the modeling bias is primarily controlled by the bandwidth, the issue is less crucial however. For a given bandwidth, h , a large value of p would expectedly reduce the modeling bias, but would cause a large variance and a considerable computational cost. It is shown in [30] that there is a general pattern of increasing variability: for estimate ^m(ν) (_x0 ) , there is no increase in variability when passing from an even (i.e., p-ν even) p=ν+2q order fit to an odd p=ν+2q+1 order fit, but when passing from an odd p=ν+2q+1 order fit to the consecutive even p=ν+2q+2 order fit, there is a price to be paid in terms of increased variability. Therefore, even order fits p=ν+2q are not recommended. Since the bandwidth is used to control the modeling complexity, we recommend the use of lowest odd order, that is, p=ν+1 , or occasionally p=ν+3 .

Another question concerns the choice of the kernel function K . Since the estimation is based on the local regression (2.3), no negative weight K should be used. As shown in [30], the optimal weight function is K(z)=(3/4)_{(1-^z2 )+} , the Epanechnikov kernel, which minimizes the asymptotic mean square error (MSE) of the resulting local polynomial estimators.

3. Two-Stage Method with Local Polynomial Fitting

Let the dependent variable x and the explanatory variable y fulfill the following regression model: [figure omitted; refer to PDF] where _yi are the observations and _xi , i=1,...,n are independent variables. Denote [figure omitted; refer to PDF] Therefore, (3.1) can be abbreviated as [figure omitted; refer to PDF]

Suppose that

(1) E(σ(x)[varepsilon])=^{(E(σ(x)_{[varepsilon]1} ),...,E(σ(x)_{[varepsilon]n} ))T} =^(0,...0,)T .

(2) Σ=E((σ(x)[varepsilon])^{(σ(x)[varepsilon])T} )=diag (^{_σ1 2} ,...,^{_σn 2} ) , where ^{_σi 2} =var((σ(x)_{[varepsilon]i} )),i=1,2,...,n .

(3) ^{_σ1 2} ,^{_σ2 2} ,...,^{_σn 2} are not all equal, that is, there is a heteroscedasticity in model (3.4). Therefore, GLS for β is

[figure omitted; refer to PDF] If covariance matrix Σ is known, the coefficients β can be estimated. Equation (3.4) is considered as the weighted least squares estimation (WLS) for β and it possesses nice qualities. However, how to estimate the Σ is still a problem. Therefore, the so-called two-stage method of estimation is used to solve the heteroscedasticity problem. The two-stage method based on local polynomial fitting can be depicted as follows: firstly, apply local polynomial fitting to get the estimate ∑... for ∑ , that is, ^σ...i2 for ^σi2 , and then we can obtain the estimate β... for β by using (3.5). The estimator follows that [figure omitted; refer to PDF] Because of E(σ(_xi )_{[varepsilon]i} |"_xi⁾² =^{_σi 2} , we construct the following regression model in order to estimate ^σi2 , [figure omitted; refer to PDF] where _ui is the difference between (σ(x)_{[varepsilon]i}⁾² and its expectation. Suppose that b=^{(XxT_Xx )-1XxT} y is the OLS of model (3.4). Although the ordinary least squares estimate b is ineffective, it is still consistent. Therefore, the corresponding residuals hold that [figure omitted; refer to PDF] Consequently, we can approximately get [figure omitted; refer to PDF]

It can be taken as a regression model, in which the variance function is regression function and the squared residuals ^ei2 are dependent variables. In order to estimate this model, parameter estimation method would usually be taken in some articles. In other words, they suppose ^σi2 =f(^cT ,_xi ) , where the form of f is known and c=^{(_c0 ,_c1 )T} are the parameters to be estimated. Note that what we usually discuss about are ^σ2 =^σ2 (^cT ,_xi ) , ^{_σi 2} =^σi2 ln (^cT ,_xi ) and so on [31]. However, the discussion for these models requires the analysts to have a better understanding of the background in practical problems. As an example, variance of asset return is always in direct proportion with its past return. Since the variance function must be nonnegative, a nonparametric method is proposed to fit ^σi2 . This method can be depicted as follows. Then, a p -order local polynomial estimation for the variances function ^σ2 (x) is obtained according to formula (2.2). Using the least squares method for the data around the local window, we can estimate the local intercept via minimizing [figure omitted; refer to PDF] with respect to ^{{_αi }i=0p} . Therefore, the solution vector can be written as [figure omitted; refer to PDF] where the design matrix [figure omitted; refer to PDF] the weighted matrix [figure omitted; refer to PDF] and ^e2 =^{(e12 ,e22 ,...,en2 )T} . Consequently, the estimated variance function is ^σ...2 (x)=_E1 α . Finally, we can get two-stage estimate β... for β by substituting estimate ^σ...i2 for ^σi2 into (3.5), which has some wonderful statistics qualities, seeing Lemma A.2, Lemma A.3, and Theorem A.4 in the appendix.

4. Simulation and Analysis

In order to discuss the qualities of β... under the limited sample, this section gives the following model, in which we do the comparison and study the fittings under different parameters. Considering the practical background which is applied to economics, we suppose the variance function of the following form. Besides, we study the fitting effects of variance in different parameters to obtain the best one.

Denote the linear model by [figure omitted; refer to PDF] where _xi (i=1,...,n) are independent variables, _yi (i=1,...,n) are observations. Also, E(σ(_xi )_{[varepsilon]i} )^{(σ(_xi )_{[varepsilon]i} )T} =diag (^σ12 ,...,^σn2 ) , and ^σ12 ,...,^σn2 are not all equal. Suppose that the variance function of the error term is ^σ2 (x)=^e0.6x .

Step 1.

Firstly, obtain the estimation of two coefficients in model (4.1) with the ordinary least squares estimation. Secondly, calculate the squares of the residuals ^e2 =(^e12 ,...,^en2)T . Thirdly, do the local polynomial regression based on the model (3.8). In this section, it is necessary to discuss how to choose parameters. Suppose that the range of x is [-1,5] and the kernel function is the Epanechnikov kernel K(z)=(3/4)(1-^z2₎₊ . In addition, the criteria of selecting a bandwidth h is minimizing the mean integrated square error ( MISE ), which can be given by [figure omitted; refer to PDF]

We could get values of MISE under different orders and bandwidths by 10000 replicates calculation with the above equation, where n=300 , see Table 1.

Furthermore, the scatter plot about h and MISE under different orders can be drawn, see Figure 1. From Figure 1, it can be seen that values of MISE are the minimum when h=0.5 whether p=1 , p=2 , and p=3 . Therefore, h=0.5 is the optimal bandwidth. Further, value of MISE when p=2 is the minimum among the above three, that is, p=2 is the optimal order. It can be drawn the conclusion that the optimal parameters are h=0.5 and p=2 , respectively. Figure 2 shows the fitting plot and residual plot for variance function after 10000 replicates, where h=0.5 , p=2 and n=300 .

Table 1: Values of MISE under different orders and bandwidths.

MISE	h=0.2	h=0.3	h=0.4	h=0.5	h=0.6	h=0.7	h=0.8
p=1	4.383078	3.852483	2.5617354	1.215800	3.853911	6.459762	9.307465
p=2	1.325719	8.152896	2.265193	0.289220	0.945407	1.245113	1.413955
p=3	5.303323	2.320999	2.805549	1.512751	2.322089	3.784113	3.854334

Figure 1: The scatter plot under different orders.

[figure omitted; refer to PDF]

The fitting plot and its residual plot when h=0.5 and p=2 : (a) fitting plot, (b) residual plot.

(a) [figure omitted; refer to PDF]

(b) [figure omitted; refer to PDF]

Step 2.

Now we substitute ^{_σ...i 2} which is obtained from Step 1 into model (3.5), and then we will get the estimation β... of GLS, that is, _β...0 =3.1004 , _β...1 =1.4987 . Figure 3 depicts the histograms and asymptotic distributions of _β...0 and _β...1 , by which we do 10000 replicates with GLS and choose n=300 . It is easy to see from Figure 3 that the estimated distributions of parameters are subject to normal asymptotically. Besides, the OLS for _β0 and _β1 can easily be obtained, say, _b...0 =3.1813 , _b...1 =1.4394 . The fitted and true values for GLS and ones of OLS are listed in Table 1. Here, the relative error is defined by Re=(|ξ...-ξ|/ξ)×100% , where ξ and ξ... present true and fitted values, respectively. From the comparison in Table 2, we can conclude that the estimators for parameters by GLS are much better than those by OLS. Furthermore, curves of original and regression functions estimated by GLS and OLS are plotted together in order to demonstrate accuracy by GLS, see Figure 4. It is not difficult to see that the GLS regression curve is almost coincident with original curve, while there is a parent bias between OLS regression and original curve. Consequently, estimators for parameters by GLS are what we require.

Table 2: Two-stage estimates for _β0 and _β1 of GLS and OLS.

Parameters	β0	β1
True value	3.1	1.5
GLS	Fitted value	3.1004	1.4987
Relative error	0.0129%	0.0867%
OLS	Fitted value	3.1813	1.4394
Relative error	2.6226%	4.0400%

The histograms and asymptotic distributions of _β...0 and _β...1 : (a) _β...0 , (b) _β...1 .

(a) [figure omitted; refer to PDF]

(b) [figure omitted; refer to PDF]

Figure 4: Curves of original and regression functions estimated by GLS and OLS.

[figure omitted; refer to PDF]

5. Application

As an example of pure cross-sectional data with potential for heteroscedasticity, consider the data given in Table 3, which gives data on per capita consumption ( Y ) and per capita gross domestic product (GDP)( X ) for 31 province or city in the People's Republic of China in 2008. Since the cross-sectional data presented in this table are quite heterogenous in a regression of per capita consumption ( Y ) on per capita GDP ( X ), heteroscedasticity is likely.

Table 3: Per capita consumption and per capita GDP for all provinces or cities in the People's Republic of China, 2008 (values are in yuan of RMB).

Area	Y	X	Area	Y	X
Beijing	18911	58204	Hubei	6513	16206
Tianjin	11957	46122	Hunan	6240	14492
Hebei	5674	19877	Guangdong	12663	33151
Shanxi	5525	16945	Guangxi	4987	12555
Neimenggu	7062	25393	Hainan	5552	14555
Liaoning	7965	25792	Chongqing	6545	14660
Jilin	6675	19382	Sichuan	5259	12893
Heilongjiang	5986	18478	Guizhou	4057	6915
Shanghai	24260	66367	Yunnan	4553	10540
Jiangsu	9659	33928	Xizang	3215	12109
Zhejiang	12569	37411	Shanxi	5272	14607
Anhui	5278	12045	Gansu	4274	10346
Fujian	8772	25908	Qinghai	4978	14257
Jiangxi	4702	12633	Ningxia	5816	14649
Shandong	8075	27807	Xinjiang	4890	16999
Henan	5141	16012

If we want to understand relationship between per capita consumption and per capita GDP, then the regression function is as follows: [figure omitted; refer to PDF] where _B0 and _B1 are regression coefficients and μ is a vector of random errors. The ordinary least squares (OLS) estimations for _B0 and _B1 can easily be obtained, that is, _B...0 =679.72 and _B...1 =0.31117 . As follow, according to the known sample data, the regression plot with OLS can easily be drawn, seeing Figure 5. Not surprising, there is a positive relationship between per capita consumption and per capita GDP, although it is not statistically significant at the traditional levels.

Figure 5: Regression plot of OLS.

[figure omitted; refer to PDF]

To testify if the regression (5.1) suffers from heteroscedasticity, we obtain the residuals of the model and plotted them against with per capita GDP, as shown in Figure 6, in which we can see that the residuals are increasing around horizontal axis with the increase in per capita GDP. Therefore, there is a heteroscedasticity in the regression (5.1). Heteroscedasticity can be also obtained through other tests such as Park test and White test. So it can be said that the regression (5.1) suffers from heteroscedasticity. Furthermore, the generalized least squares (GLS) estimations _B...0 =665.84 and _B...1 =0.31265 can easily be got after 10000 replicates, provided with the bandwidth h=0.5 , the order p=2 , and MISE=0.275746 . Then the regression plot can be drawn in Figure 7. Compared with Figures 5 and 7, it can be said that the distribution of points around regression line in Figure 7 is more uniform than that in Figure 5. Finally, it can be exactly said that more accurate regression can be obtained by GLS than that by OLS.

Figure 6: Residual plot.

[figure omitted; refer to PDF]

Figure 7: Regression plot of GLS.

[figure omitted; refer to PDF]

6. Conclusions

In this paper we presented a new method for estimation of linear heteroscedastic regression model based on local polynomial estimation with nonparametric technique. The proposed scheme firstly adopted the local polynomial fitting to estimate heteroscedastic function, then the coefficients of regression model are obtained based on generalized least squares method. Our approach avoided the test of heteroscedasticity for the linear model. Due to nonparametric technique of local polynomial estimation, if the heteroscedastic function is unknown, the precision of estimation was improved. Furthermore, the effect of parameters on the fitting was researched and the optimal fitting was obtained. Besides, the asymptotic normality of parameters was verified by the results of numerical simulations. Finally, the simulation results under different parameters and local polynomial estimation of real data in a case of economics really indicated that our approach was effective in finite-sample situations, which did not need to assume the form of heteroscedastic function. The presented algorithm could be easily used to heteroscedastic regression model in some practical problems.

Acknowledgments

This work was supported by Chongqing CSTC foundations of China (CSTC2010BB2310, CSTC2011jjA40033), Chongqing CMEC foundations of China (KJ080614, KJ100810, and KJ100818).

References

[1] T. Amemiya, "A note on a heteroscedastic model," Journal of Econometrics , vol. 6, no. 3, pp. 365-370, 1977.

[2] P. Chen, D. Suter, "A bilinear approach to the parameter estimation of a general heteroscedastic linear system, with application to conic fitting," Journal of Mathematical Imaging and Vision , vol. 28, no. 3, pp. 191-208, 2007.

[3] J. Ma, L. Liu, "Multivariate nonlinear analysis and prediction of Shanghai stock market," Discrete Dynamics in Nature and Society , vol. 2008, 2008.

[4] H. Liu, Z. Zhang, Q. Zhao, "The volatility of the index of shanghai stock market research based on ARCH and its extended forms," Discrete Dynamics in Nature and Society , vol. 2009, 2009.

[5] H. Hu, "QML estimators in linear regression models with functional coefficient autoregressive processes," Mathematical Problems in Engineering , vol. 2010, 2010.

[6] C. R. Rao, Y. Wu, Q. Shao, "An M -estimation-based procedure for determining the number of regression models in regression clustering," Journal of Applied Mathematics and Decision Sciences , vol. 2007, 2007.

[7] Q. X. He, M. Zheng, "Local polynomial regression for heteroscedaticity in the simple linear model," Systems Engineering-Theory Methodology Applications , vol. 12, no. 2, pp. 153-156, 2003.

[8] W. Härdle, M. Müller, S. Sperlich, A. Werwatz Nonparametric and Semiparametric Models , of Springer Series in Statistics, pp. xxviii+299, Springer, New York, NY, USA, 2004.

[9] L. Galtchouk, S. Pergamenshchikov, "Sharp non-asymptotic oracle inequalities for non-parametric heteroscedastic regression models," Journal of Nonparametric Statistics , vol. 21, no. 1, pp. 1-18, 2009.

[10] J. Fan, "Local linear regression smoothers and their minimax efficiencies," The Annals of Statistics , vol. 21, no. 1, pp. 196-216, 1993.

[11] J. Fan, W. Zhang, "Statistical estimation in varying coefficient models," The Annals of Statistics , vol. 27, no. 5, pp. 1491-1518, 1999.

[12] I. Miranda Cabrera, H. L. Baños Díaz, M. L. Martínez, "Dynamical system and nonlinear regression for estimate host-parasitoid relationship," Journal of Applied Mathematics , vol. 2010, 2010.

[13] A. Bilbao-Guillerna, M. De la Sen, A. Ibeas, S. Alonso-Quesada, "Robustly stable multiestimation scheme for adaptive control and identification with model reduction issues," Discrete Dynamics in Nature and Society , vol. 2005, no. 1, pp. 31-67, 2005.

[14] T. J. Koo, "Stable model reference adaptive fuzzy control of a class of nonlinear systems," IEEE Transactions on Fuzzy Systems , vol. 9, no. 4, pp. 624-636, 2001.

[15] M. De la Sen, "Identification of a class of linear time-varying continuous-time systems," Automation and Remote Control , vol. 63, no. 2, pp. 246-261, 2002.

[16] M. Jankovic, "Adaptive nonlinear output feedback tracking with a partial high-gain observer and backstepping," Transactions on Automatic Control , vol. 42, no. 1, pp. 106-113, 1997.

[17] M. Jansson, B. Wahlberg, "On consistency of subspace methods for system identification," Automatica , vol. 34, no. 12, pp. 1507-1519, 1998.

[18] J. Fan, "Design-adaptive nonparametric regression," Journal of the American Statistical Association , vol. 87, no. 420, pp. 998-1004, 1992.

[19] M. Pandya, K. Bhatt, P. Andharia, "Bayes estimation of two-phase linear regression model," International Journal of Quality, Statistics, and Reliability , vol. 2011, 2011.

[20] L.-Y. Su, "Prediction of multivariate chaotic time series with local polynomial fitting," Computers & Mathematics with Applications , vol. 59, no. 2, pp. 737-744, 2010.

[21] L. Y. Su, "Multivariate local polynomial regression with application to Shenzhen component index," Discrete Dynamics in Nature and Society , vol. 2011, 2011.

[22] L. Y. Su, Y. J. Ma, J. J. Li, "The application of local polynomial estimation in suppressing strong chaotic noise," Chinese Physics B , vol. 21, no. 2, 2012.

[23] J. Fan, Q. Yao Nonlinear Time Series: Nonparametric and Parametric Methods , of Springer Series in Statistics, pp. xx+551, Springer, New York, NY, USA, 2003.

[24] D. Ruppert, M. P. Wand, "Multivariate locally weighted least squares regression," The Annals of Statistics , vol. 22, no. 3, pp. 1346-1370, 1994.

[25] S. Farsiu, M. D. Robinson, M. Elad, P. Milanfar, "Fast and robust multiframe super resolution," IEEE Transactions on Image Processing , vol. 13, no. 10, pp. 1327-1344, 2004.

[26] H. Takeda, S. Farsiu, P. Milanfar, "Robust kernel regression for restoration and reconstruction of images from sparse noisy data," in Proceedings of the IEEE International Conference on Image Processing (ICIP '06), pp. 1257-1260, Atlanta, Ga, USA, October 2006.

[27] L. Firinguetti, "Ridge regression in the context of a system of seemingly unrelated regression equations," Journal of Statistical Computation and Simulation , vol. 56, no. 2, pp. 145-162, 1997.

[28] J. Fan, I. Gijbels, "Data-driven bandwidth selection in local polynomial fitting: variable bandwidth and spatial adaptation," Journal of the Royal Statistical Society B , vol. 57, no. 2, pp. 371-394, 1995.

[29] L. Y. Su, F. L. Li, "Deconvolution of defocused image with multivariate local polynomial regression and iterative wiener filtering in DWT domain," Mathematical Problems in Engineering , vol. 2010, 2010.

[30] J. Fan, I. Gijbels Local Polynomial Modelling and Its Applications , vol. 66, of Monographs on Statistics and Applied Probability, pp. xvi+341, Chapman & Hall, London, UK, 1996.

[31] H. C. Rutemiller, D. A. Bowers, "Estimation in a heteroscedastic regression model," Journal of the American Statistical Association , vol. 63, pp. 552-557, 1968.

[32] H. G. William Econometric Analysis , Macmillan, New York, NY, USA, 1993.

Appendix

Appendix

Suppose _X1 ,...,_XT is a stationary sequence. Let ^Fik be the σ -algebra of events generated by the random variables {_Xj ,i...4;j...4;k} , and ^L2 (^Fik ) consists of ^Fik -measurable random variables with finite second moment. Before giving the Lemma A.1, we show the condition as follows.

Condition:

(i) The kernel K(·) is bounded with a bounded support.

(ii) ^σ2 (·) and f(·) are continuous at the points x and f(x)>0 .

(iii): The sequence is ρ -mixing processes, that is,

[figure omitted; refer to PDF]

as T[arrow right]∞ . we also assume that _Σl ρ(l)<∞ , E^Y02 <∞ .

(iv) For ρ -mixing processes, there exists a sequence of positive integers satisfying _sT [arrow right]∞ and _sT =o(Th) such that T/hρ(_sT )[arrow right]0 as T[arrow right]∞ .

Then we have the following lemma.

Lemma A.1.

Under Condition 1, if h=O(^T1/(2p+3) ) and ^m(p+1) (·) is continuous at the point x , then as T[arrow right]∞ , [figure omitted; refer to PDF] where _β0 (x)=(m(x),^{m[variant prime]} (x)...,^m(p) (x)/p!^)T , S=(_{μi+j-2)(p+1)×(p+1)} , ^S* =(_{νi+j-2)(p+1)×(p+1)} , and _cp =(_{μp+2-j)(p+1)×1} , and where _μi and _νi are defined as _μi =^{∫-∞∞tiK2} (t)dt and _νi =^∫-∞∞ti K(t)dt , respectively.

The proof of Lemma A.1 is shown in [23]. An immediate consequence of Lemma A.1 is that the asymptotic bias and the asymptotic variance for the local polynomial estimator are defined as [figure omitted; refer to PDF]

Then, the mean square error (MSE) at point x is [figure omitted; refer to PDF]

Lemma A.2.

For the model (3.4), [varepsilon] is assumed to be a vector and follows normal distribution with mean zero and covariance Σ , where Σ>0 , and P _{lim n[arrow right]∞} (X[variant prime]^Σ-1 X/n)=Q , a finite positive definite matrix. Then one has

(1) β... is a consistent estimator of β

(2) β... is asymptotically normal with the mean β and covariance matrix (X[variant prime]^Σ-1 X^)-1 .

The proof is shown in [32].

Lemma A.3.

Suppose ^σ...2 (x) is a p -order local polynomial estimator (LPE) of the variance function ^σ2 (x) . Under conditions _hn [arrow right]0 , and n_hn [arrow right]∞ :

(1) if p is an odd, and the variance function ^σ2 (x) is continuous p+1 derivative and bounded, then

[figure omitted; refer to PDF] where _K1 and _K2 are constants. Therefore,

[figure omitted; refer to PDF]

(2) if p is an even, and the variance function ^σ2 (x) is continuous p+1 and p+2 derivative and bounded, then

[figure omitted; refer to PDF] where _K1 , _K2 , _K3 , and _K4 are constants. Then we can get _{max 1...4;i...4;n} MSE(^σ...i2 )=O(^hn2p+4 +1/n_hn ) .

The proof is shown in [18].

Theorem A.4.

For the model (3.4), [varepsilon] is assumed to be a vector and follows normal distribution with mean zero and covariance Σ , where Σ>0 , and p _{lim n[arrow right]∞} (^XT X/n)=_Q1 , p _{lim n[arrow right]∞} ^XT [varepsilon]/n=_Q2 , where _Q1 and _Q2 are positive definite matrixes. Besides, they satisfy the conditions of Lemma A.3, then β _LPE is asymptotically normal with mean β and covariance ^{(XTΣ-1 X)-1} .

Proof.

By Lemma A.2 and the bibliography [32], in order to prove the above results, we only need to testify [figure omitted; refer to PDF] where Σ... is an estimator of Σ , that is to testify [figure omitted; refer to PDF] By Lemma A.3, whether p is an odd or an even, we have [figure omitted; refer to PDF] Therefore, [figure omitted; refer to PDF] The conclusions follow.