国立中山大学：《计量经济学》（英文版） Chapter 9 Heteroscedasticity

Ch. 9 Heteroscedasticity Regression disturbances whose variance are not constant across observations are heteroscedastic. In the heteroscedastic model we assume that 0 0 a10 0 0 0a2 00 It will sometimes prove useful to write a2=awi. This form is an arbitrary scaling which allows us to use a normalization, tr(Q For exampl Not. This makes the classical regression with homoscedas tic disturbance a simple special cases with wi=1,i=1, 2, ..,M Ex See the residuals at Figure 11.1 1 Ordinary Least Squares Estimation We showed in Section 8.1 that in the presence of heteroscedasticity, the Ols estimator B is unbiased and consistent. However it is inefficient relative to the GLS estimator 1.1 Estimating the Appropriate Covariance Matrix for OLS EStimators If the type of heteroscedasticity is known with certainty, then the Ols estimator is undesirable; we should use the GLS instead. The precise form of the het- eroscedasticity is usually unknown, however. In that case, Gls is not usable and we may need to salvage what we can from the results of OLS estimators

Ch. 9 Heteroscedasticity Regression disturbances whose variance are not constant across observations are heteroscedastic. In the heteroscedastic model, we assume that E(εε0 ) = σ 2Ω = σ 2         ω1 0 . . . 0 0 ω2 . . . 0 . . . . . . . . . . . . . . . . . . 0 0 . . . ωN         =         σ 2 1 0 . . . 0 0 σ 2 2 . . . 0 . . . . . . . . . . . . . . . . . . 0 0 . . . σ 2 N         . It will sometimes prove useful to write σ 2 i = σ 2ωi . This form is an arbitrary scaling which allows us to use a normalization, tr(Ω) = X N i=1 ωi = N. (For example, σ 2 = PN i=1 σ 2 i N .) This makes the classical regression with homoscedas￾tic disturbance a simple special cases with ωi = 1, i = 1, 2, ..., N. Example: See the residuals at Figure 11.1. 1 Ordinary Least Squares Estimation We showed in Section 8.1 that in the presence of heteroscedasticity, the OLS estimator βˆ is unbiased and consistent. However it is inefficient relative to the GLS estimator. 1.1 Estimating the Appropriate Covariance Matrix for OLS Estimators If the type of heteroscedasticity is known with certainty, then the OLS estimator is undesirable; we should use the GLS instead. The precise form of the het￾eroscedasticity is usually unknown, however. In that case, GLS is not usable, and we may need to salvage what we can from the results of OLS estimators. 1

The conventional estimated covariance matrix for the OLS estimator o2(X'X)-1 is inappropriate; the appropriate matrix is a(X'x) QX(X'X)-.White (1980) has shown that it is still possible to obtain an appropriate covariance es- timator of the Ols estimators even the form of heteroscedasticity is unknown What is actually required is an estimate of NoX'nX I White(1980)shows that under very general conditions, the matrix where ei is the i-th least square residual, is a consistent estimator of Z. There- fore. the white estimator ()=N(XX)-s(XX)-1 can be used as an estimator of the true variance of the oLS estimator. Inference concerning B are still possible by means of OLS estimator even when the specific structure of Q2 is not specified as B is normally distributed asymptotically. more generally, White shows that tests of the general linear hypothesis RB= g, under the null hypothesis, the statistics (RB-9)R(X'X)NSo(X'X)RT(RB-)Nxm where m denote the number of restrictions imposed Reproduce the results at Table 11.1 2 Testing for Heteroscedasticity One can rarely be certain that the disturbances are heteroscedastic however and unfortunately, what form the heteroscedasticity takes if they are. As such, it is useful to be able to test for homoscedasticity and if necessary, modify our est mation procedure accordingly

The conventional estimated covariance matrix for the OLS estimator σ 2 (X0X) −1 is inappropriate; the appropriate matrix is σ 2 (X0X) −1X0ΩX(X0X) −1 . White (1980) has shown that it is still possible to obtain an appropriate covariance es￾timator of the OLS estimators even the form of heteroscedasticity is unknown. What is actually required is an estimate of Σ = 1 N σ 2X0ΩX = 1 N X N i=1 σ 2 i xix 0 i . White (1980) shows that under very general conditions, the matrix S0 = 1 N X N i=1 e 2 i xix 0 i , where ei is the i − th least square residual, is a consistent estimator of Σ. There￾fore, the White estimator, V\ar(βˆ) = N(X0X) −1S0(X0X) −1 , can be used as an estimator of the true variance of the OLS estimator. Inference concerning β are still possible by means of OLS estimator even when the specific structure of Ω is not specified as βˆ is normally distributed asymptotically. More generally, White shows that tests of the general linear hypothesis Rβ = q, under the null hypothesis, the statistics (Rβˆ − q) 0 [R(X0X) −1NS0(X0X) −1R 0 ] −1 (Rβˆ − q) ∼ χ 2 m, where m denote the number of restrictions imposed. Exercise: Reproduce the results at Table 11.1. 2 Testing for Heteroscedasticity One can rarely be certain that the disturbances are heteroscedastic however, and unfortunately, what form the heteroscedasticity takes if they are. As such, it is useful to be able to test for homoscedasticity and if necessary, modify our esti￾mation procedure accordingly. 2

Most of the test for heteroscedasticity are based on the following strategy. OLs estimator is a consistent estimator of B even in the presence of heteroscedastic- ity. As such, the OLS residuals will mimic, albeit imperfectly because of sam ling variability, the heteroscedasticity of the true disturbance. Therefore, tests designed to detect heteroscedasticity will, in most cases, be applied to the o residuals 2.1 Nonspecific Tests for Heteroscedasticity There may be instance when the form of the heteroscedasticity is not known, but nevertheless, it is known that the disturbance variance in monotonically related to the size of a known exogenous variables z by which observation on the depen dent variable y can be ordered. One frequently used test in this instance are the Goldfeld-Quandt test Perhaps it is also believed that the broader class of heteroscedasticity is i= h(ia), where h( is a general function independent of i, is applicable (such as o?=za, 02=(a)2 and 0?= exp(z(a)). If so, the Breush-Pagan test is appropriate. If nothing is known a priori other than the heteroscedastic variance are uniformly bounded, White general test is applicable 2.1.1 The Goldfeld-Quandt Test a very popular test for determining the presence of heteroscedasticity which is monotonically related to an exogenous variables by which observations on the de- pendent variables can be ordered is the Goldfeld-Quandt(1965)test. The steps of this test are as follow 1. Order the observations by the values of the variables z 2. Choose p central observations and omit them. 3. Fit separate regression by OLS to the two groups, provides(N-p)/2> k 4. Let SSEl and SSE2 denote the sum of squared residuals based on the small variance(which you suppose they do) and the large variance group, respectively Form the statistics SSEl F

Most of the test for heteroscedasticity are based on the following strategy. OLS estimator is a consistent estimator of β even in the presence of heteroscedastic￾ity. As such, the OLS residuals will mimic, albeit imperfectly because of sam￾pling variability, the heteroscedasticity of the true disturbance. Therefore, tests designed to detect heteroscedasticity will, in most cases, be applied to the OLS residuals. 2.1 Nonspecific Tests for Heteroscedasticity There may be instance when the form of the heteroscedasticity is not known, but nevertheless, it is known that the disturbance variance in monotonically related to the size of a known exogenous variables z by which observation on the depen￾dent variable y can be ordered. One frequently used test in this instance are the Goldfeld-Quandt test. Perhaps it is also believed that the broader class of heteroscedasticity is σ 2 i = h(z 0 iα), where h(·) is a general function independent of i, is applicable (such as σ 2 i = z 0 iα, σ 2 i = (z 0 iα) 2 and σ 2 i = exp(z 0 iα)). If so, the Breush-Pagan test is appropriate. If nothing is known a priori other than the heteroscedastic variance are uniformly bounded, White general test is applicable. 2.1.1 The Goldfeld-Quandt Test A very popular test for determining the presence of heteroscedasticity which is monotonically related to an exogenous variables by which observations on the de￾pendent variables can be ordered is the Goldfeld-Quandt (1965) test. The steps of this test are as follow: 1. Order the observations by the values of the variables z. 2. Choose p central observations and omit them. 3. Fit separate regression by OLS to the two groups, provides (N − p)/2 > k. 4. Let SSE1 and SSE2 denote the sum of squared residuals based on the small variance (which you suppose they do) and the large variance group, respectively. Form the statistics F = SSE1 SSE2 = e 0 1e1/N1 − k e 0 2e2/N2 − k , 3

which will distributed as FNi-k Na-k under the null hypothesis of homoscedasticity 2N XN-k and by the null assumption that o=of=a2 2.1.2 The Breush-Pagan Test The Goldfeld-Quandt test has been found to be reasonably powerful when we are able to identify correctly the variable to use in the sample separation. This requirement does limit its generality, however. Breush-Pagan(1979)assume a border class of heteroscedasticity defined by 02=02h(ao +4 a1), where zi is a(p x 1)vector of exogenous variables. This model is homoscedastic if a1=0. Breush and Pagen consider the general estimation equation 元2 where e: represent the i-th OlS residual and a-= >ise/N. The null hy- pothesis a1=0 can be tested if the Ei are normally distributed. Let SSR denote the sum of squares obtained in an Ols estimation of Letw=,=∑1/N,amd=a+a1, Then SSR=∑1(,-m2) Breush and Pagan shows, if a1=0, then SsR 2.1.3 White's general est White address the case where nothing is known about the structure of the het- eroscedasticity other than the heteroscedastic variance a? are uniformly bounded It would be desirable to be able to test a general hypothesis of the form for all i Hi Not Ho If there is no heteroscedasticity (under Ho), then s2(X'X)will give a consistent estimator of variance 6, where if there is, then it will not(see Ch. 8 sec. 1). White

which will distributed as FN1−k,N2−k under the null hypothesis of homoscedasticity since e 0 1 e1 σ2 ∼ χ 2 N1−k and by the null assumption that σ 2 = σ 2 1 = σ 2 2 . 2.1.2 The Breush-Pagan Test The Goldfeld-Quandt test has been found to be reasonably powerful when we are able to identify correctly the variable to use in the sample separation. This requirement does limit its generality, however. Breush-Pagan (1979) assume a border class of heteroscedasticity defined by σ 2 = σ 2h(α0 + z 0 iα1), where zi is a (p × 1)vector of exogenous variables. This model is homoscedastic if α1 = 0. Breush and Pagen consider the general estimation equation eˆ 2 i σ¯ 2 = α0 + z 0 iα1 + vi , where eˆi represent the i − th OLS residual and σ¯ 2 = PN i=1 eˆ 2 i /N. The null hy￾pothesis α1 = 0 can be tested if the εi are normally distributed. Let SSR denote the sum of squares obtained in an OLS estimation of eˆi σ¯ 2 = α0 + z 0 iα1 + vi . (Let yi = eˆ 2 i σ¯ 2 , y¯ = PN i=1 yi/N, and yˆi = αˆ0 + z 0 iαˆ1. Then SSR = PN i=1(yˆi − y¯) 2 .) Breush and Pagan shows, if α1 = 0, then 1 2 SSR ∼ χ 2 p . 2.1.3 White’s General Test White address the case where nothing is known about the structure of the het￾eroscedasticity other than the heteroscedastic variance σ 2 i are uniformly bounded. It would be desirable to be able to test a general hypothesis of the form: H0 : σ 2 i = σ 2 for all i, H1 : Not H0. If there is no heteroscedasticity (under H0), then s 2 (X0X) will give a consistent estimator of variance βˆ, where if there is, then it will not (see Ch. 8 sec.1). White 4

derives a test for heteroscedasticity which consists of comparing the elements of NSo(= 2ie'xixi)and s2(X'X)(=s22i-xixi), thus indicating whether or not the usual OLS formula s(X'X) is a consistent covariance estimator. Large discrepancies between NSo and s (X'X)support the contention of heteroscedas- ticity while small discrepancies support homoscedasticity A simple operational version of this test is carried out by obtaining NR2in the regression of e? on a constant and all unique variables in x@x. This statistics is asymptotically distributed as x2, where p is the number of regressors in the egression,including the constant. Exercise Reproduce the results of Example 11.3 at p 224 3 Weighted Least Squares When Q2 is Known Having tested for and found evidence of heteroscedasticity, the logical next is to revise the estimation technique to account for it. The gls estimator is B=(X9-1x)-1x-1Y Consider the most general case, o2=02wi. Then Q2-I is a diagonal matrix whose i-th diagonal element is 1/w;. The GLS is obtained by regressing PY= vN/√oN Applying Ols to the transformed model, we obtain the weighted least squares (WLS)estimator, =∑xx∑ i=1 where Wi=1/w;. The logic of the computation is that observations with smaller variances receive a large weight in the computations of the sums and therefore

derives a test for heteroscedasticity which consists of comparing the elements of NS0(= PN i=1 e 2 i xix 0 i ) and s 2 (X0X)(= s 2 PN i=1 xix 0 i ), thus indicating whether or not the usual OLS formula s 2 (X0X) is a consistent covariance estimator. Large discrepancies between NS0 and s 2 (X0X) support the contention of heteroscedas￾ticity while small discrepancies support homoscedasticity. A simple operational version of this test is carried out by obtaining NR2 in the regression of e 2 i on a constant and all unique variables in x ⊗ x. This statistics is asymptotically distributed as χ 2 p , where p is the number of regressors in the regression, including the constant. Exercise: Reproduce the results of Example 11.3 at p.224. 3 Weighted Least Squares When Ω is Known Having tested for and found evidence of heteroscedasticity, the logical next is to revise the estimation technique to account for it. The GLS estimator is β˜ = (X0Ω −1X) −1X0Ω −1Y. Consider the most general case, σ 2 i = σ 2ωi . Then Ω −1 is a diagonal matrix whose i − th diagonal element is 1/ωi . The GLS is obtained by regressing PY =         y1/ √ ω1 y2/ √ ω2 . . . yN / √ ωN         on PX =         x1/ √ ω1 x2/ √ ω2 . . . xN / √ ωN         . Applying OLS to the transformed model, we obtain the weighted least squares (WLS) estimator, β˜ = "X N i=1 wixix 0 i #−1 "X N i=1 wixiyi # , where wi = 1/ωi . The logic of the computation is that observations with smaller variances receive a large weight in the computations of the sums and therefore 5

have greater influence in the estimate obtained A common specification is that the variance is proportional to one of the regressors or its square. If then the transformed regression model for the gls is y=B:+B1 B2 If the variance is proportional to k instead of zk, then the weight applied to each observations is 1/vEk instead of 1/k 4 Estimation when o contains unknown pa rameters The general form of the heteroscedastic regression model has too many parameters to estimate by ordinary method. Typically, the model is restricted by formulat- ing a292 as a function of a few parameters, such as a?=aag or a2=o2[]2 Write this as R(a), FGLS based on a consistent estimator of R (a)is asymptot ally equivalent to gLs. The new problem is that we must first find consistent estimators of the unknown parameters in Q(a). Two methods are typically used, two step GLs and maximum likelihood 4.1 Two-Step Estimation For the heteroscedastic model. the GLS estimator is i=1 The two step estimators are computed by first obtaining estimators of, usually using some function of the Ols residuals. then the Fgls will be i=1 a2

have greater influence in the estimate obtained. A common specification is that the variance is proportional to one of the regressors or its square. If σ 2 i = σ 2x 2 ik, then the transformed regression model for the GLS is y xk = βk + β1  x1 xk  + β2  x2 xk  + ... + ε xk . If the variance is proportional to xk instead of x 2 k , then the weight applied to each observations is 1/ √ xk instead of 1/xk. 4 Estimation When Ω Contains Unknown Pa￾rameters The general form of the heteroscedastic regression model has too many parameters to estimate by ordinary method. Typically, the model is restricted by formulat￾ing σ 2Ω as a function of a few parameters, such as σ 2 i = σ 2x α i or σ 2 i = σ 2 [x 0 iα] 2 . Write this as Ω(α), FGLS based on a consistent estimator of Ω(α) is asymptot￾ically equivalent to GLS. The new problem is that we must first find consistent estimators of the unknown parameters in Ω(α). Two methods are typically used, two step GLS and maximum likelihood. 4.1 Two-Step Estimation For the heteroscedastic model, the GLS estimator is β˜ = "X N i=1  1 σ 2 i  xix 0 i #−1 "X N i=1  1 σ 2 i  xiyi # . The two step estimators are computed by first obtaining estimators σˆ 2 i , usually using some function of the OLS residuals, then the FGLS will be βˇ = "X N i=1  1 σˆ 2 i  xix 0 i #−1 "X N i=1  1 σˆ 2 i  xiyi # . 6

where vi is just the difference between the random variable a and its expectation Since Ei is unobservable, we would use the OlS residual, for which x1(-B)=E1+u1 But in large sample, as B-6, terms in ui will become negligible, so that at least approximatel The procedure suggested is to treat the variance function as a regression and use the squares of the OLS residual as the dependent variable. For example, if 02=ao+d a1, then a consistent estimator of a will be the OLS in the model z a1+U In this model. uf is both heteroscedastic and autocorrelated. so a is consistent but inefficient. But, consistency is all that is required for asymptotically efficient estimation of B using (a) The two-step estimator may be iterated by recomputing the residuals after computing the FGLS estimate and then reentering the computation (OLS, B- e→a→B→e→ Exercise Reproduce the results at Table 11.2 on p 231 4.2 Maximum Likelihood estimation 5 Autoregressive Conditional Heteroscedastic- ity(ARCH

Let ε 2 i = σ 2 i + vi , where vi is just the difference between the random variable ε 2 i and its expectation. Since εi is unobservable, we would use the OLS residual, for which ei = εi − x 0 i (βˆ − β) = εi + ui . But in large sample, as βˆ p −→ β, terms in ui will become negligible, so that at least approximately, ei = σ 2 i + v ∗ i . The procedure suggested is to treat the variance function as a regression and use the squares of the OLS residual as the dependent variable. For example, if σ 2 i = α0 + z 0 iα1, then a consistent estimator of α will be the OLS in the model e 2 i = α0 + z 0 iα1 + v ∗ i . In this model, v ∗ i is both heteroscedastic and autocorrelated, so αˆ is consistent but inefficient. But, consistency is all that is required for asymptotically efficient estimation of β using Ω(αˆ). The two-step estimator may be iterated by recomputing the residuals after computing the FGLS estimate and then reentering the computation (OLS, βˆ → e → αˆ → βˇ → eˇ →....). Exercise: Reproduce the results at Table 11.2 on p.231. 4.2 Maximum Likelihood Estimation 5 Autoregressive Conditional Heteroscedastic￾ity (ARCH) 7