2.5. Fixed Effects versus Random Effects

It should come as no surprise to learn that fixed effects methods are not the only way to estimate regression models for longitudinal data. There are several popular alternatives, many of which are readily available in SAS. To fully appreciate both the strengths and weaknesses of the fixed effects method, we need to compare it with some of these alternatives.

The closest cousin to the fixed effects model is the random effects or mixed model. We start with the same basic equation:

Now, however, instead of assuming that α_i represents a set of fixed parameters, we suppose that each α_i is a random variable with a specified probability distribution. Typically, it is assumed that α_i has a normal distribution with a mean of 0 and constant variance, and that it's independent of z_i, x_it, and ε_it.

This random effects model can be readily estimated with PROC MIXED:

PROC MIXED DATA=persyr3 COVTEST NOCLPRINT;
   CLASS id time;
   MODEL anti=pov self time / SOLUTION;
   RANDOM INTERCEPT / SUBJECT=id;
RUN;

The COVTEST option requests a test statistic for the null hypothesis that the variance of α_i = 0. NOCLPRINT suppresses printing of the values of the CLASS variables (581 values for ID). The RANDOM statement is what introduces the person-level random component into the regression model.^[] Here it says that the intercept is a random variable that differs for each value of the ID variable. Without the RANDOM statement, PROC MIXED would produce the same OLS estimates as in Output 2.11.

^[] The same results can be obtained with the following statement:

RANDOM id;

However, with this statement, the computing time increases by a factor of about 20.

Results are shown in Output 2.14. The first panel—labeled "Covariance Parameter Estimates"—gives estimates of the variances of α_i (labeled "Intercept") and ε_i (labeled "Residual"). Both variances are significantly greater than 0. The regression coefficients in the lower panel are closer to the conventional OLS estimates in Output 2.11 than they are to the fixed effects estimates in Output 2.10. Most importantly, the coefficient for POV is highly significant in the random effects model, but didn't even approach significance in the fixed effects model.

Table 2.14. Output 2.14 Random Effects Model with Time-Varying Predictors

Covariance Parameter Estimates
Cov Parm	Subject	Estimate	Standard Error	Z Value	PrZ
Intercept	id	1.3875	0.1025	13.54	<.0001
Residual		0.9971	0.04152	24.01	<.0001

Solution for Fixed Effects
Effect	time	Estimate	Standard Error	DF	tValue	Pr > |t|
Intercept		2.8832	0.2085	580	13.83	<.0001
pov		0.2947	0.07745	1158	3.81	0.0001
self		−0.05971	0.009533	1158	−6.26	<.0001
time	1	−0.2157	0.05883	1158	−3.67	0.0003
time	2	−0.1688	0.05864	1158	−2.88	0.0041
time	3	0	.	.	.	.

Why the difference between fixed and random effects estimates? The main reason is that, unlike the fixed effects model, the random effects model does not really control for between-person variation. That's because a key assumption of the method is that α_i is uncorrelated with x_it. The fixed effects model, on the other hand, imposes no restrictions on the relationship between α_i and x_it.

So what good is the random effects model? Well, it's better than conventional OLS because the standard error estimates adjust for the within-person correlation in the repeated measurements of the dependent variable. By contrast, conventional OLS standard errors are biased downward by the dependence in the multiple observations for each person. But that advantage is also shared with the fixed effects estimator.

One thing that the random effects method can do that the fixed effects method cannot is produce coefficient estimates for time-invariant variables. For example, Output 2.15 shows the results of adding seven time-invariant predictors to the random effects model we just fitted. Only two of them, GENDER and MOMWORK, are statistically significant at the .05 level. The coefficients for POV and SELF don't change much by the addition of these variables. Neither does the estimate of the variance of α_i, which would be expected to decline with the addition of strong, person-specific predictors to the model. Keep in mind that the fixed effects model also controls for these time-invariant predictors; it just doesn't produce coefficient estimates for them. Furthermore, unlike the random effects method, it controls for all time-invariant predictors, not just those that have been measured and included in the regression model.

Table 2.15. Output 2.15 Random Effects Model with Time-Varying and Time-Invariant Predictors

Covariance Parameter Estimates
Cov Parm	Subject	Estimate	Standard Error	Z Value	PrZ
Intercept	id	1.3056	0.09810	13.31	<.0001
Residual		0.9959	0.04144	24.03	<.0001

Solution for Fixed Effects
Effect	time	Estimate	Standard Error	DF	tValue	Pr > |t|
Intercept		2.7471	1.0985	573	2.50	0.0127
pov		0.2460	0.08038	1158	3.06	0.0023
self		−0.06201	0.009512	1158	−6.52	<.0001
time	1	−0.2163	0.05879	1158	−3.68	0.0002
time	2	−0.1690	0.05860	1158	−2.88	0.0040
time	3	0	.	.	.	.
black		0.2271	0.1259	1158	1.80	0.0715
hispanic		−0.2180	0.1385	1158	−1.57	0.1156
childage		0.08846	0.09125	1158	0.97	0.3325
married		−0.04933	0.1266	1158	−0.39	0.6970
gender		−0.4834	0.1067	1158	−4.53	<.0001
momage		−0.02195	0.02533	1158	−0.87	0.3864
momwork		0.2614	0.1149	1158	2.28	0.0231

Another attractive feature of the random effects model is the ability to introduce random coefficients for the time-varying predictors. For example, we can rewrite the model as

which simply puts an i subscript on the β coefficient. We then assume that β_i is a set of normally distributed random variables with a common mean and variance, both of which can be estimated. We also assume that β_i is independent of x_it, ε_itand z_i (but could possibly covary with α_i).

For the NLSY data, let's consider a model that allows the effect of POV to vary randomly across children. Here is the SAS code:^[]

^[] The same results can be obtained with the following statement

RANDOM id pov*id;

But this results in an enormous increase in computing time. Also, some might prefer to use the TYPE=UN option on the RANDOM statement. This allows for a covariance between the random intercept and the random slope.

PROC MIXED DATA=persyr3 COVTEST NOCLPRINT;
   CLASS id time;
   MODEL anti=pov self time / SOLUTION;
   RANDOM INTERCEPT pov / SUBJECT=id;
RUN;

Results are shown in Output 2.16. As shown in the first panel, the estimate of the variance of the POV coefficient is .2317. The z-test for the null hypothesis that this variance is 0 has a p-value of .0509. So there is only marginal evidence for non-zero variation in the coefficient across persons.^[] In the "Solution for Fixed Effects" panel, the estimate for POV of .3053 can be regarded as an estimate of the average effect of this variable.

^[] A more accurate test is computed by taking differences in −2 × log-likelihood for this model and the earlier model that forces the variance to be 0. However, correct calculation of this statistic also requires that the option METHOD=ML be used in the PROC statement for both models. (The default method is REML, i.e., restricted maximum likelihood). When I did this calculation, the resulting chisquare statistic was 3.0 with 1 d.f., giving a p-value of .08.

Table 2.16. Output 2.16 Random Effects Model with Random Coefficient

Covariance Parameter Estimates
Cov Parm	Subject	Estimate	Standard Error	Z Value	PrZ
Intercept	id	1.3261	0.1064	12.47	<.0001
pov	id	0.2317	0.1416	1.64	0.0509
Residual		0.9767	0.04236	23.05	<.0001

Solution for Fixed Effects
Effect	time	Estimate	Standard Error	DF	tValue	Pr > |t|
Intercept		2.8698	0.2083	580	13.77	<.0001
pov		0.3053	0.08204	170	3.72	0.0003
self		−0.05939	0.009536	988	−6.23	<.0001
time	1	−0.2154	0.05892	988	−3.66	0.0003
time	2	−0.1660	0.05859	988	−2.83	0.0047
time	3	0	.	.	.	.

There are many other possible variations on random effects models. For example, besides having a random effect for persons, one could also have random effects for higher levels of aggregation such as families or schools (assuming that the data contain multiple persons for each family or school). PROC MIXED also allows for autoregressive and other covariance structures on the ε_it component.

None of these elaborations allows the random effects model to control for all possible time-invariant predictors, as the fixed effects model does. On the other hand, if there is really no correlation between α_i and x_it—that is, between the person-specific effect and the time-varying predictors—then random effects estimates might have far less sampling variability than the fixed effects estimates. That translates into more powerful hypothesis tests and narrower confidence intervals. The reason is that the random effects method uses variation both within persons and between persons, whereas the fixed effects method uses only variation within persons.

Mundlak (1978) has argued that the random effects method should be seen as a special case of the fixed effects method. In a nutshell, the argument goes like this. We start with a conventional random effects model such as equation (2.5), and then relax its restrictions by allowing for all possible correlations between the random component and the time-varying predictors. When this is done, ML estimates of the random effects model become identical to the OLS estimates for the fixed effects model. In general, whenever one has a choice between two nested models, one being a restricted version of the other, there is a tradeoff between bias and efficiency. The more parsimonious model (the random effects model in this case) will lead to more efficient estimates, but those estimates might be biased if the restrictions of the model are incorrect. The less parsimonious model (the fixed effects model) is less prone to bias, but at the expense of greater sampling variability.

It would be nice to have a statistical test of the random effects model against the fixed effects model as an alternative. That way we would have some basis for deciding whether we can tolerate the biases inherent in the random effects method, or whether we need to go with the less restrictive fixed effects model. One such test is available in PROC TSCSREG. When you ask this procedure to estimate a random effects model, it automatically reports a Hausman test (Greene 2000, p. 576), which compares the fixed effects and random effects models.^[]

^[] The Hausman test is computed as follows. Let b be the vector of fixed effects coefficients (excluding the constant) and let β be the vector of random effects coefficients. Let Σ = var(b) − var( β) where var(b) is the estimated covariance matrix for b and similarly for β. The statistic is then , which has a chi-square distribution under the null hypothesis.

The PROC TSCSREG code for the NLSY data follows immediately. Note that TSCSREG does not have a CLASS statement, so it's necessary to create dummy variables for TIME in a DATA step. TSCSREG expects the data to be sorted by person, and within person by time. The ID statement tells TSCSREG what variables index these two dimensions. In the MODEL statement, the RANONE option is what specifies a random effects model. Changing this to FIXONE would produce a fixed effects model. As noted earlier, however, the downside of this PROC is that the coefficients for the dummy variables in the fixed effects model are explicitly estimated and reported, requiring lots of computer time and excessive output. Another disadvantage is that TSCSREG requires that the data be balanced—each person must have the same number of observations. PROC GLM and PROC MIXED, on the other hand, can handle unbalanced data sets without difficulty.

Here's the code:

DATA tscset;
   SET persyr3;
   time1=(time=1);
   time2=(time=2);
PROC TSCSREG DATA=tscset;
   ID id time;
   MODEL anti=self pov time1 time2 / RANONE;
RUN;

Results in Output 2.17 are virtually identical to what we got with PROC MIXED in Output 2.14. But we also get the "Hausman Test for Random Effects," which tests the null hypothesis of the random effects model against the alternative fixed effects model. In this case, the low p-value indicates that the random effects model should be rejected. While the Hausman test can be very useful, it is also somewhat ad hoc and can break down entirely in certain circumstances (the Σ matrix in note 5 may not be positive definite). In chapter 6 we shall see how to construct a likelihood ratio statistic for the same null hypothesis.

Table 2.17. Output 2.17 TSCSREG Output for Random Effects Model with Hausman Test

Variance Component Estimates
Variance Component for Cross Sections	1.380509
Variance Component for Error	0.994156

Hausman Test for Random Effects
DF	m Value	Pr > m
2	12.82	0.0016

Parameter Estimates
Variable	DF	Estimate	Standard Error	tValue	Pr>|t|
Intercept	1	2.883232	0.2085	13.83	<.0001
self	1	−0.05971	0.00953	−6.26	<.0001
pov	1	0.294928	0.0775	3.81	0.0001
time1	1	−0.21575	0.0588	−3.67	0.0003
time2	1	−0.16877	0.0587	−2.88	0.0041

Another approach that is closely related to the random effects method is generalized estimating equations (GEE), which can be implemented with PROC GENMOD. In the case of linear models, GEE is equivalent to generalized least squares, which also happens to be the default method used in PROC TSCSREG. GEE makes no explicit assumptions about person-specific random components in the regression model. It simply allows for correlations in the dependent variable across observations (over time in this case).

For the NLSY data, the GEE method can be used to estimate the random effects model by specifying the following in PROC GENMOD:

PROC GENMOD DATA=persyr3;
   CLASS id time;
   MODEL anti= self pov time;
   REPEATED SUBJECT=id / TYPE=EXCH MODELSE;
RUN;

The REPEATED statement invokes GEE estimation. The TYPE=EXCH option specifies that correlations between measurements of ANTI in different years are all equal. This is the correlation structure that is implied by a simple random effects model such as equation (2.5). MODELSE specifies that standard errors are calculated based on the assumed model rather than using the default method of robust standard errors.

In Output 2.18, we see that the "Exchangeable Working Correlation" is .58. This is an estimate of the correlation between error terms in different years; because we specified the EXCH option, these are identical for all pairs of years. Parameter estimates and associated statistics in Output 2.18 are virtually identical to those in Outputs 2.17 and 2.14. It's worth noting that GENMOD is by far the most computationally efficient procedure for getting these random effects parameter estimates. On my PC, PROC MIXED took .23 seconds, PROC TSCSREG took .75 seconds, and PROC GENMOD took 0.15 seconds. (In fairness to MIXED, if the model is specified using the REPEATED statement rather than the RANDOM statement, the time can be reduced to 0.09 seconds).

Table 2.18. Output 2.18 GEE Estimates of Random Effects Model

Exchangeable Working Correlation
Correlation	0.5819870399

Analysis Of GEE Parameter Estimates
Model-Based Standard Error Estimates
Parameter		Estimate	Standard Error	95% Confidence	Limits	Z	Pr>|Z|
Intercept		2.8832	0.2085	2.4745	3.2918	13.83	<.0001
self		−0.0597	0.0095	−0.0784	−0.0410	−6.26	<.0001
pov		0.2947	0.0775	0.1429	0.4465	3.80	0.0001
time	1	−0.2157	0.0588	−0.3310	−0.1004	−3.67	0.0002
time	2	−0.1688	0.0586	−0.2837	−0.0538	−2.88	0.0040
time	3	0.0000	0.0000	0.0000	0.0000	.	.
Scale		1.5444	.	.	.	.	.

Somewhat different results are obtained in Output 2.19 when the REPEATED statement is altered to read as follows:

REPEATED SUBJECT=id / TYPE=UN CORRW;

TYPE=UN specifies an unstructured correlation matrix, which is reflected in the different correlations seen in the working correlation matrix (requested with the CORRW option). I've also omitted the MODELSE option, thereby asking GENMOD to compute robust standard errors using White's (1980) method. Both of these changes should make the results somewhat less sensitive to misspecification of the error structure. The parameter estimates and associated statistics are all a bit different from those estimated under the random effects model, but still quite similar.

Table 2.19. Output 2.19 GEE Estimates for a Less Restricted Model

Working Correlation Matrix
	Col1	Col2	Col3
Row1	1.0000	0.5785	0.5359
Row2	0.5785	1.0000	0.6396
Row3	0.5359	0.6396	1.0000

Analysis Of GEE Parameter Estimates
Empirical Standard Error Estimates
Parameter		Estimate	Standard Error	95% Confidence	Limits	Z	Pr>|Z|
Intercept		2.8935	0.2322	2.4384	3.3485	12.46	<.0001
self		−0.0605	0.0102	−0.0805	−0.0405	−5.93	<.0001
pov		0.3140	0.0812	0.1549	0.4730	3.87	0.0001
time	1	−0.2164	0.0636	−0.3410	−0.0918	−3.40	0.0007
time	2	−0.1691	0.0594	−0.2855	−0.0527	−2.85	0.0044
time	3	0.0000	0.0000	0.0000	0.0000	.	.