CHAPTER 4

Simple Modeling Issues and Heteroscedasticity

Today, Touro asks, “Prof. Metric, I heard that the linear regression technique only requires a model to be linear in parameters. What does that mean?” Booka then says, “I also have a question. In the previous chapters, we assumed that the data did not have any problems. What will happen if they violate one of the classic assumptions?” Prof. Metric praises them for raising good questions and tells us that several of these issues will be discussed this week. We learn that once we finish with the chapter, we will be able to:

Functional Forms

Model Transformations

Prof. Metric explains that the “linear in parameters” requirement means that the model cannot have parameters in any nonlinear form, such as . Other than that, the dependent and explanatory variables can be of any nonlinear form such as Ln(Y) Ln(X) Ln(Y), ln(X), or X ^b. The reason is that we can easily transform a model that has nonlinear variables into a linear one. Equation 4.1 presents a logarithmic model with linear parameters:

We can transform this model by letting Y* = Ln(Y) and letting X* = Ln(X); then, the model becomes a linear one:

Y* = a₁ + a₂ X*.

Similarly, a polynomial model

can be transformed by letting X* = X², and Z* = Z³; then the model also becomes a linear one in parameters:

Y = a₁ + a₂ X*+ a₃ Z*.

These transformations allow us to employ OLS estimation techniques using Excel, as usual.

Choosing a Logarithmic Model

The following log models are the most important in data analysis, and we want to learn how to interpret them.

Log-Log Model

This model is written as:

In this case, the parameter a₂ measures the elasticity of y with respect to x.

For example, the following model measures the price elasticity of demand:

ln DEMAND_i = a₁ + a₂ ln PRIC_i + e_i.

Booka raises her hand and offers her estimation for the price elasticity of demand for history books at her company. The estimation is based on her recent survey. The estimation results are:

ln DEMAND_i = 0.3 + −1 ln PRIC_i.

Invo offers “So, a 1 percent increase in the price of your history books decreases the demand for these books by 1.1 percent.” Touro adds, “Oh, the demand for your history books is slightly elastic because its price elasticity is a little greater than 1. You might want to draw several more samples to see whether the results are robust to sample changes. If this is true, your company might want to lower the prices slightly to boost sales.” Prof. Metric praises them for an insightful discussion and moves to the next section.

Log-Linear Model

This model is written as:

In this case, the slope a₂ measures the percent change (a₂ × 100) in y due to a one-unit change in x.

For example, the following model measures the impact of investing in new capital on the profit of a company:

ln PROFIT_i = a₁ + a₂ CAPITA_i + e_i.

Taila says that her company bought several new machines six months ago and found their effect on the company’s monthly profit as follows:

ln PROFIT_i = a₁ + 0.02CAPITA_i.

where CAPITAL was measured in thousands of dollars.

We are able to calculate the impact of this change in capital on the profit as ΔPROFIT = (0.02*100)% = 2%. Since the unit of capital is in thousands of dollars, this 2 percent change in profit is due to a $1,000 increase in capital.

Linear-Log Model

This model is written as:

In this case, a 1 percent change in x leads to a (a₂/100) unit change in y.

Touro offers an example model, in which spending depends on log of income:

SPEND_i = a₁ + a₂ ln(INCOM)_i + e_i.

Prof. Metric says this is an interesting case: if a₂ > 0, an increase in income will cause an increase in spending, indicating that the good is a normal good, but the log function also implies that spending will increase at a decreasing rate, because the graph of a log function is concave with a decreasing slope. Touro says this is the case with spending on travel at his company, where the result for regression is:

SPEND_i = 200 + 1400 ln(INCOM)_i + e_i,

where the unit of spending (SPEND) is in thousands of dollars.

We can calculate the impact of the change in income on the spending for travel, for Touro, as ΔSPEND = (1400/100) = 14. Since the unit of spending is in thousands of dollars, we conclude that a 1 percent change in income causes a $14,000 change in travel spending at Touro’s company.

Growth Rate

Prof. Metric reminds us that ln(1+x) ≅ x if x is small (less than 20 percent) and that we learned this approximation in algebra classes. This formula can have practical applications here. Invo offers an example on the growth of revenue from his company, where they found this equation related to revenue over time:

where r is the yearly growth rate of revenue.

We are able to go back to the first period as follows:

REVENUE_t = REVENUE₀ (1 + r)^t.

Taking the logarithms of both sides yields:

ln (REVENUE_t) = ln(REVENUE₀) + [ln(1 + r)] × t.

Let ln(REVENUE₀) = a₁ and [ln(1 + r)] = a₂, then

ln (REVENUE_t) = a₁ + a₂t.

Invo provides us with the following estimation results:

ln (REVENUE) = 0.2413 + 0.0276 × t.

We are able to write the approximation as:

[ln (1 + r)] ≃ r = (0.0276 * 100)% = 2.76%.

Hence, the growth rate of revenue at Invo’s company is approximately 2.76 percent per year.

Return to Training

Prof. Metric says that we can similarly have wages as a function of training, so:

where r is rate of return to an extra month of training.

Suppose the estimation result is:

ln(WAGE)_t = 0.4315 + 0.0124 * TRAINING_t−1,

then an additional month of training increases the wage rate by approximately 1.24 percent.

Prof. Metric reminds us that all log models can be extended to multiple regressions by adding more control variables, as discussed in Chapter 3, or by adding dummy variables, as will be discussed in the following section.

Intercept Dummy Variables

Additive Dummy Variables

An intercept dummy is called an additive dummy because this variable is added to the model. We can add dummy variables (also called indicator variables) to a model to control for the difference in characteristics among various groups. Prof. Metric tells us to go back to Booka’s example of the paperback or hardback books. The original model on their sale values can be written as follows:

Let D = 1 if the book is in paperback and D = 0 otherwise, then we have two groups in this case (numbers of groups = G = 2). Suppose the dataset has 15 observations for D = 1 and 15 observations for D = 0, then the dummy-variable method has two advantages:

Prof. Metric says that we can have more than two groups—for example, cars can come in green, red, blue, or brown, and thus G = 4.

In the aforementioned example on book sales, if we add D to equation (4.7), then

The intercept might not have any meaning, but the difference in the intercepts depicts the differences in the prices for different type of books. Suppose d = −15, and SALE is in dollars, then a paperback copy reduces the sale value by $15.

To choose the reference group, a researcher can only add (G-1) dummies in the regression. For example, for two groups, use one dummy. Otherwise, the sum of D₁ and D₂ equals 1. Since the coefficient estimate for the constant terms is a vector of all 1s, we will have a perfect-collinearity problem if we add G dummies. This problem is also called the dummy-variable trap.

Heteroscedasticity

Nature and Consequences

In the previous chapters, we assumed that the errors have a constant variance, var(e_i) = σ². Prof. Metric says that when this variance is not constant, then the classic assumption (iii) is violated, and the errors are said to be heteroskedastic. To see the problem, we all look at the original equation:

If var(e_i) = (note the subscript i), the variance changes when the identity changes, and we have a heteroscedasticity problem.

For example, if , where σ² = 100 and the variable x_i changes from $64 to $81, then we can obtain

But

Thus, the error variance, , changes from 800 to 900 instead of remaining a constant. The heteroscedasticity problem could occur with time-series data as well.

To this point, Invo says, “Oh yes, suppose we want to perform a regression of food expenditures on income, then the variance of errors might change because this variance might be correlated with income as well.”

Touro asks, “How come the variance of the errors is correlated with income?” Invo answers, “Because food expenditures also depend on the price level, which in turn might go up if income goes up. Since price is not included in the model, it must be contained in the error term. As a result, the variance of the errors is correlated with income.”

Prof. Metric praises Invo and says that there are two consequences of heteroscedasticity:

Each problem can be addressed separately, and we will learn how to overcome them in the next section.

Detecting Heteroscedasticity

A Lagrange Multiplier (LM) test is usually performed to test a variance function so that we can find out if a heteroscedasticity problem exists. The theoretical foundation behind this test is simple. Given the model in equation (4.10) with var(e_i) = , we let the error variance be a function of a variable w:

We then test for the significance of c₂. Since c₁ is a constant, if c₂ is not statistically different from zero, then var(e) = c₁ = a constant, so the hypotheses are:

H₀: c₂ = 0; H_a: c₂ ≠ 0.

We learn that there are several LM tests. In this lecture, Prof. Metric teaches us the White version, which is the most popular one and which has w = x. The test utilizes the chi-squared distribution, , where K is the number of estimated coefficients (parameters), and (K − 1) is the degrees of freedom (df). Here are the steps to perform this LM test:

First, we need to estimate the original equation: y_i = a₁ + a₂ x_i + e_i.

Next, we obtain and generate so that we can estimate the variance function:

After that, we can follow the four-step procedure for any test:

Prof. Metric then gives us an example, “Let’s say estimating = c₁ + c₂x_i + v_i yields R² = 0.25 and the sample size is N = 30; how can we perform the LM test?”

We work out the problem together by calculating:

Hence, we reject the null, meaning that c₂ is different from zero and implying that the data has a heteroskedasticity problem.

We can also type =CHIINV(0.05, 1) into an Excel cell and see that Excel reports the critical value as 3.8415 ≈ 3.84.

Correcting Heteroscedasticity

Since there are two possible consequences with heteroscedasticity, we can discuss each of them separately.

Touro exclaims, “Oh, this model no longer has a constant variable, because changes with each observation.” Prof. Metric says that is true, so we need to suppress the constant when performing the regression and that Prof. Empirie will show us how to do it in Excel.

He then says that the problem is solved, because

We also learn that the predicted values reported in Excel are for y*, so we need to multiply ŷ* by to obtain the predicted value for ŷ. The interval prediction values then can be calculated as usual.

Prof. Metric tells us that sometimes the form of heteroscedasticity is much more complex, so transforming the model by does not correct the problem. In practice, we don’t know the form of the heteroscedasticity, so we have to estimate a more general form.

We are excited and say in unison, “Let’s learn a more general case,” and Prof. Metric introduces this model:

Theoretically, ln() = ln(σ²) + γ(ln x_i), so we can find by performing a regression of this equation.

Empirically, we don’t have σ², so we need to run a regression on y_i = a₁ + a₂ x_i + e_i and obtain ê_i. After that, we need to obtain estimate values of ln(σ²) and g(ln x_i) by writing the relation as:

where α₁ = ln (σ²); α₂ = χ z_i; ln x_i.

Therefore, the estimated version of equation (4.18) is:

From equation (4.19), we can calculate the estimated variance:

Next, we can make the transformation:

The problem is solved, because

Prof. Metric says the correction can also be extended to multiple regressions. In this case, the formula for calculating the estimated variance is:

where K is the number of parameters.

Data Analyses

Prof. Empirie reminds us that the Chow test is similar to the F-tests discussed in Chapter 3, in which she already showed us how to estimate the unrestricted and restricted models. Moreover, adding dummy variables to a regression is just like adding any variable. Hence, in this section she only provides instructions for heteroscedasticity.

Detecting Heteroscedasticity

The yearly data on spending (SPEND) and salary (SAL) are available in the file Ch04.xls. Fig.4.1, and a section of the second regression with the N and R² is shown in Figure 4.1. First, we regress SPEND on SAL.

Next, we need to regress e² on SAL.

From Figure 4.1, N = 33 and R² = 0.1913, so LM_STAT = 33*0.1913 = 6.31.

Typing =CHIINV(0.05,1) into any Excel cell gives you = 3.84.

Since , we reject the null hypothesis, meaning the coefficient of the residuals is statistically different from zero, implying that the model has a heteroscedasticity problem.

Figure 4.1 R² and N for testing heteroscedasticity

Correcting Heteroscedasticity

Obtaining White’s Standard Error

Prof. Empirie has utilized the results from the first regression for Figure 4.1, and we find them in the file Ch04.xls, Fig.4.2. We learn to perform the following steps:

The results are reported in Figure 4.2 with the corrected var(a₂) = 0.0006 and se(a₂) = 0.0238.

Estimating a Transformed Model

The yearly data on SPEND and SAL are used again in this demonstration and are available in the file Ch04.xls, Fig.4.3. We learn to perform the following steps:

Figure 4.2 Obtaining white’s standard error for a₂

Next, we need to regress SPEND* on X1* and SAL*’.

The regression results for the transformed model are displayed in Figure 4.3.

Prof. Empirie reminds us that to obtain the predicted value for SPEND*, we just need to check the box Residuals when we enter the regression commands so that Excel will report them in its Summary Output. However, we still need to multiply predicted SPEND* by SAL^1/2 to obtain the predicted SPEND. The interval prediction values can then be calculated as usual.

Figure 4.3 Regression results for the transformed model

Exercises

Table 4.1 Estimation results for exercise 3

	Coefficients	Standard error	p-value
Intercept	4.0536	1.1165	0.006
Sqft	0.0671	0.0203	0.008
Volcan	−0.1056	−0.0329	0.009