12.2.1 Pearson’s Linear Correlation and the Concept of Factor

Let’s imagine a dataset that has n observations and, for each observation i (i = 1, …, n), values corresponding to each one of the k metric variables X, as shown in Table 12.1.

From the dataset, and given our intention of extracting factors from k variables X, we must define correlation matrix ρ that displays the values of Pearson’s linear correlation between each pair of variables, as shown in Expression (12.1).

$\mathbf{\rho }=\left(\begin{array}{cccc}1& {\rho }_{12}& \cdots & {\rho }_{1k}\\ {\rho }_{21}& 1& \cdots & {\rho }_{2k}\\ ⋮& ⋮& \ddots & ⋮\\ {\rho }_{k1}& {\rho }_{k2}& \cdots & 1\end{array}\right)$

Correlation matrix ρ is symmetrical in relation to the main diagonal that, obviously, shows values equal to 1. For example, for variables X₁ and X₂, Pearson’s correlation ρ₁₂ can be calculated by using Expression (12.2).

${\rho }_{12}=\frac{{\sum }_{i=1}^n\left(X_{1i}-{\overline{X}}_1\right)\cdot \left(X_{2i}-{\overline{X}}_2\right)}{\sqrt{{\sum }_{i=1}^n{\left(X_{1i}-{\overline{X}}_1\right)}^2}\cdot \sqrt{{\sum }_{i=1}^n{\left(X_{2i}-{\overline{X}}_2\right)}^2}}$

where ${\overline{X}}_1$ and ${\overline{X}}_2$ represent the means of variables X₁ and X₂, respectively, and this expression is analogous to Expression (4.11), defined in Chapter 4.

Thus, since Pearson’s correlation is a measure of the level of linear relationship between two metric variables, which may vary between − 1 and 1, a value closer to one of these extreme values indicates the existence of a linear relationship between the two variables under analysis, which, therefore, may significantly contribute to the extraction of a single factor. On the other hand, a Pearson correlation that is very close to 0 indicates that the linear relationship between the two variables is practically nonexistent. Therefore, different factors can be extracted.

Let’s imagine a hypothetical situation in which a certain dataset only has three variables (k = 3). A three-dimensional scatter plot can be constructed from the values of each variable for each observation. The plot can be seen in Fig. 12.1.

Only based on the visual analysis of the chart in Fig. 12.1, it is difficult to assess the behavior of the linear relationships between each pair of variables. Thus, Fig. 12.2 shows the projection of the points that correspond to each observation in each one of the planes formed by the pairs of variables, highlighting, in the dotted line, the adjustment that represents the linear relationship between the respective variables.

While Fig. 12.2A shows that there is a significant linear relationship between variables X₁ and X₂ (a very high Pearson correlation), Fig. 12.2B and C make it very clear that there is no linear relationship between X₃ and these variables. Fig. 12.3 displays these projections in a three-dimensional plot, with the respective linear adjustments in each plane (the dotted lines).

Thus, in this hypothetical example, while variables X₁ and X₂ may be represented by a single factor in a very significant way, which we will call F₁, variable X₃ may be represented by another factor, F₂, orthogonal to F₁. Fig. 12.4 illustrates the extraction of these new factors in a three-dimensional way.

So, factors can be understood as representations of latent dimensions that explain the behavior of the original variables.

Having presented these initial concepts, it is important to emphasize that in many cases researchers may choose to not extract a factor represented in a considerable way by only one variable (in this case, factor F₂), and what will define the extraction of each one of the factors is the calculation of the eigenvalues from correlation matrix ρ, as we will study in Section 12.2.3. Nevertheless, before that, it will be necessary to check the overall adequacy of the factor analysis, which will be discussed in the following section.

12.2.2 Overall Adequacy of the Factor Analysis: Kaiser-Meyer-Olkin Statistic and Bartlett’s Test of Sphericity

An adequate extraction of factors from the original variables requires correlation matrix ρ to have relatively high and statistically significant values. As discussed by Hair et al. (2009), even though visually analyzing correlation matrix ρ does not reveal if the factor extraction will in fact be adequate, a significant number of values less than 0.30 represent a preliminary indication that the factor analysis may not be adequate.

In order to verify the overall adequacy of the factor extraction itself, we must use the Kaiser-Meyer-Olkin statistic (KMO) and Bartlett’s test of sphericity.

The KMO statistic gives us the proportion of variance considered common to all the variables in the sample under analysis, that is, which can be attributed to the existence of a common factor. This statistic varies from 0 to 1 and, while values closer to 1 indicate that the variables share a very high proportion of variance (high Pearson correlations), values closer to 0 are a result of low Pearson correlations between the variables, which may indicate that the factor analysis will not be adequate. The KMO statistic, presented initially by Kaiser (1970), can be calculated through Expression (12.3).

$\mathrm{KMO}=\frac{{\sum }_{l=1}^k{\sum }_{c=1}^k{\rho }_{\mathit{lc}}^2}{{\sum }_{l=1}^k{\sum }_{c=1}^k{\rho }_{\mathit{lc}}^2+{\sum }_{l=1}^k{\sum }_{c=1}^k{\phi }_{\mathit{lc}}^2},l\ne c$

where l and c represent the rows and columns of correlation matrix ρ, respectively, and the terms φ represent the partial correlation coefficients between two variables. While Pearson’s correlation coefficients ρ are also called zero-order correlation coefficients, partial correlation coefficients φ are also known as higher-order correlation coefficients. For three variables, they are also called first-order correlation coefficients, for four variables, second-order correlation coefficients, and so on.

Let’s imagine a hypothetical situation in which a certain dataset shows three variables once again (k = 3). Is it possible that, in fact, ρ₁₂ reflects the level of linear relationship between X₁ and X₂ if variable X₃ is related to the other two? In this situation, ρ₁₂ may not represent the true level of linear relationship between X₁ and X₂ when X₃ is present, which may provide a false impression regarding the nature of the relationship between the first two. Thus, partial correlation coefficients may contribute with the analysis, since, according to Gujarati and Porter (2008), they are used when researchers wish to find out the correlation between two variables, either by controlling or ignoring the effects of other variables present in the dataset. For our hypothetical situation, it is the correlation coefficient regardless of X₃’s influence over X₁ and X₂, if any.

Hence, for three variables X₁, X₂, and X₃, we can define the first-order correlation coefficients the following way:

${\phi }_{12,3}=\frac{{\rho }_{12}-{\rho }_{13}\cdot {\rho }_{23}}{\sqrt{\left(1-{\rho }_{13}^2\right)\cdot \left(1-{\rho }_{23}^2\right)}}$

where φ_12,3 represents the correlation between X₁ and X₂, maintaining X₃ constant,

${\phi }_{13,2}=\frac{{\rho }_{13}-{\rho }_{12}\cdot {\rho }_{23}}{\sqrt{\left(1-{\rho }_{12}^2\right)\cdot \left(1-{\rho }_{23}^2\right)}}$

where φ_13,2 represents the correlation between X₁ and X₃, maintaining X₂ constant, and

${\phi }_{23,1}=\frac{{\rho }_{23}-{\rho }_{12}\cdot {\rho }_{13}}{\sqrt{\left(1-{\rho }_{12}^2\right)\cdot \left(1-{\rho }_{13}^2\right)}}$

where φ_23,1 represents the correlation between X₂ and X₃, maintaining X₁ constant.

In general, a first-order correlation coefficient can be obtained through the following expression:

${\phi }_{\mathit{ab},c}=\frac{{\rho }_{\mathit{ab}}-{\rho }_{\mathit{ac}}\cdot {\rho }_{\mathit{bc}}}{\sqrt{\left(1-{\rho }_{\mathit{ac}}^2\right)\cdot \left(1-{\rho }_{\mathit{bc}}^2\right)}}$

where a, b, and c can assume values 1, 2, or 3, corresponding to the three variables under analysis.

Conversely, for a case in which there are four variables in the analysis, the general expression of a certain partial correlation coefficient (second-order correlation coefficient) is given by:

${\phi }_{\mathit{ab},\mathit{cd}}=\frac{{\phi }_{\mathit{ab},c}-{\phi }_{\mathit{ad},c}\cdot {\phi }_{\mathit{bd},c}}{\sqrt{\left(1-{\phi }_{\mathit{ad},c}^2\right)\cdot \left(1-{\phi }_{\mathit{bd},c}^2\right)}}$

where φ_ab,cd represents the correlation between X_a and X_b, maintaining X_c and X_d constant, bearing in mind that a, b, c, and d may take on values 1, 2, 3, or 4, which correspond to the four variables under analysis.

Obtaining higher-order correlation coefficients, in which five or more variables are considered in the analysis, should always be done based on the determination of lower-order partial correlation coefficients. In Section 12.2.6, we will propose a practical example by using four variables, in which the algebraic solution of the KMO statistic will be obtained through Expression (12.8).

It is important to highlight that, even if Pearson’s correlation coefficient between two variables is 0, the partial correlation coefficient between them may not be equal to 0, depending on the values of Pearson’s correlation coefficients between each one of these variables and the others present in the dataset.

In order for a factor analysis to be considered adequate, the partial correlation coefficients between the variables must be low. This fact denotes that the variables share a high proportion of variance, and disregarding one or more of them in the analysis may hamper the quality of the factor extraction. Therefore, according to a widely accepted criterion found in the existing literature, Table 12.2 gives us an indication of the relationship between the KMO statistic and the overall adequacy of the factor analysis.

On the other hand, Bartlett’s test of sphericity (Bartlett, 1954) consists in comparing correlation matrix ρ to an identity matrix I of the same dimension. If the differences between the corresponding values outside the main diagonal of each matrix are not statistically different from 0, at a certain significance level, we may consider that the factor extraction will not be adequate. In other words, in this case, Pearson’s correlations between each pair of variables are statistically equal to 0, which makes any attempt of performing a factor extraction from the original variables unfeasible. So, we can define the null and alternative hypotheses of Bartlett’s test of sphericity the following way:

${\mathrm{H}}_0:\mathbf{\rho }=\left(\begin{array}{cccc}1& {\rho }_{12}& \cdots & {\rho }_{1k}\\ {\rho }_{21}& 1& \cdots & {\rho }_{2k}\\ ⋮& ⋮& \ddots & ⋮\\ {\rho }_{k1}& {\rho }_{k2}& \cdots & 1\end{array}\right)=\mathbf{I}=\left(\begin{array}{cccc}1& 0& \cdots & 0\\ 0& 1& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & 1\end{array}\right)$

${\mathrm{H}}_1:\mathbf{\rho }=\left(\begin{array}{cccc}1& {\rho }_{12}& \cdots & {\rho }_{1k}\\ {\rho }_{21}& 1& \cdots & {\rho }_{2k}\\ ⋮& ⋮& \ddots & ⋮\\ {\rho }_{k1}& {\rho }_{k2}& \cdots & 1\end{array}\right)\ne \mathbf{I}=\left(\begin{array}{cccc}1& 0& \cdots & 0\\ 0& 1& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & 1\end{array}\right)$

The statistic corresponding to Bartlett’s test of sphericity is an χ² statistic, which has the following expression:

${\chi }_{\text{Bartlett}}^2=-\left[\left(n-1\right)-\left(\frac{2\cdot k+5}{6}\right)\right]\cdot ln\left|D\right|$

with $\frac{k\cdot \left(k-1\right)}{2}$ degrees of freedom. We know that n is the sample size and k is the number of variables. In addition, D represents the determinant of correlation matrix ρ.

Thus, for a certain number of degrees of freedom and a certain significance level, Bartlett’s test of sphericity allows us to check if the total value of the χ_Bartlett² statistic is higher than the statistic’s critical value. If this is true, we may state that Pearson’s correlations between the pairs of variables are statistically different from 0 and that, therefore, factors can be extracted from the original variables and the factor analysis is adequate. When we develop a practical example in Section 12.2.6, we will also discuss the calculations of the χ_Bartlett² statistic and the result of Bartlett’s test of sphericity.

It is important to emphasize that we should always favor Bartlett’s test of sphericity over the KMO statistic to take a decision about the factor analysis’s overall adequacy. Given that the former is a test with a certain significance level, and the latter is only a coefficient (a statistic) calculated without any set distribution of probabilities or hypotheses that allow us to evaluate the corresponding significance level to make a decision.

In addition, it is important to mention that for only two original variables the KMO statistic will always be equal to 0.50. Conversely, the χ_Bartlett² statistic may indicate if the null hypothesis of the test of sphericity was rejected or not, depending on the magnitude of Pearson’s correlation between both variables. Thus, while the KMO statistic will be 0.50 in these situations, Bartlett’s test of sphericity will allow researchers to decide whether to extract one factor from the two original variables or not. In contrast, for three original variables, it is very common for researchers to extract two factors with the statistical significance of Bartlett’s test of sphericity, however, with the KMO statistic less than 0.50. These two situations emphasize even more the greater relevance of Bartlett’s test of sphericity in relation to the KMO statistic in the decision-making process.

Finally, we must mention that the recommendation to study Cronbach’s alpha’s magnitude, before studying the overall adequacy of the factor analysis, is commonly found in the existing literature, so that the reliability with which a factor can be extracted from original variables can be evaluated. We would like to highlight that Cronbach’s alpha only offers researchers indications of the internal consistency of the variables in the dataset so that a single factor can be extracted. Therefore, determining it is not a mandatory requisite for developing the factor analysis, since this technique allows the extraction of most factors. Nevertheless, for pedagogical purposes, we will discuss the main concepts of Cronbach’s alpha in the Appendix of this chapter, with its algebraic determination and corresponding applications in SPSS and Stata software.

Having discussed these concepts and verified the overall adequacy of the factor analysis, we can now move on to the definition of the factors.

12.2.3 Defining the Principal Component Factors: Determining the Eigenvalues and Eigenvectors of Correlation Matrix ρ and Calculating the Factor Scores

Since a factor represents the linear combination of the original variables, for k variables, we can define a maximum number of k factors (F₁, F₂, …, F_k), analogous to the maximum number of clusters that can be defined from a sample with n observations, as we discussed in the previous chapter, since a factor can also be understood as the result of the clustering of variables. Therefore, for k variables, we have:

$\begin{array}{c}{F}_{1i}=s_{11}\cdot X_{1i}+s_{21}\cdot X_{2i}+\cdots +s_{k1}\cdot X_{\mathit{ki}}\\ F_{2i}=s_{12}\cdot X_{1i}+s_{22}\cdot X_{2i}+\cdots +s_{k2}\cdot X_{\mathit{ki}}\\ ⋮\\ F_{\mathit{ki}}=s_{1k}\cdot X_{1i}+s_{2k}\cdot X_{2i}+\cdots +s_{\mathit{kk}}\cdot X_{\mathit{ki}}\end{array}$

where the terms s are known as factor scores, which represent the parameters of a linear model that relates a certain factor to the original variables. Calculating the factor scores is essential in the context of the factor analysis technique and is elaborated by determining the eigenvalues and eigenvectors of correlation matrix ρ. In Expression (12.11), we once again show correlation matrix ρ, which has already been presented in Expression (12.1).

$\mathbf{\rho }=\left(\begin{array}{cccc}1& {\rho }_{12}& \cdots & {\rho }_{1k}\\ {\rho }_{21}& 1& \cdots & {\rho }_{2k}\\ ⋮& ⋮& \ddots & ⋮\\ {\rho }_{k1}& {\rho }_{k2}& \cdots & 1\end{array}\right)$

This correlation matrix, with dimensions k × k, shows k eigenvalues λ² (λ₁² ≥ λ₂² ≥ … ≥ λ_k²), which can be obtained from solving the following equation:

$det\left({\lambda }^2\cdot \mathbf{I}-\mathbf{\rho }\right)=0$

where I is the identity matrix, also with dimensions k × k.

Since a certain factor represents the result of the clustering of variables, it is important to highlight that:

${\lambda }_1^2+{\lambda }_2^2+\cdots +{\lambda }_k^2=k$

Expression (12.12) can be rewritten as follows:

$\left|\begin{array}{cccc}{\lambda }^2-1& -{\rho }_{12}& \cdots & -{\rho }_{1k}\\ -{\rho }_{21}& {\lambda }^2-1& \cdots & -{\rho }_{2k}\\ ⋮& ⋮& \ddots & ⋮\\ -{\rho }_{k1}& -{\rho }_{k2}& \cdots & {\lambda }^2-1\end{array}\right|=0$

from which we can define the eigenvalue matrix Λ² the following way:

${\mathbf{\Lambda }}^2=\left(\begin{array}{cccc}{\lambda }_1^2& 0& \cdots & 0\\ 0& {\lambda }_2^2& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & {\lambda }_k^2\end{array}\right)$

In order to define the eigenvectors of matrix ρ based on the eigenvalues, we must solve the following equation system for each eigenvalue λ² (λ₁², λ₂², …, λ_k²):

• • Determining eigenvectors v₁₁, v₂₁, …, v_k1 from the first eigenvalue (λ₁²):

$\left(\begin{array}{cccc}{\lambda }_1^2-1& -{\rho }_{12}& \cdots & -{\rho }_{1k}\\ -{\rho }_{21}& {\lambda }_1^2-1& \cdots & -{\rho }_{2k}\\ ⋮& ⋮& \ddots & ⋮\\ -{\rho }_{k1}& -{\rho }_{k2}& \cdots & {\lambda }_1^2-1\end{array}\right)\cdot \left(\begin{array}{c}{v}_{11}\\ v_{21}\\ ⋮\\ v_{k1}\end{array}\right)=\left(\begin{array}{c}0\\ 0\\ ⋮\\ 0\end{array}\right)$

from where we obtain:

$\left\{\begin{array}{c}\left({\lambda }_1^2-1\right)\cdot v_{11}-{\rho }_{12}\cdot v_{21}\dots -{\rho }_{1k}\cdot v_{k1}=0\\ -{\rho }_{21}\cdot v_{11}+\left({\lambda }_1^2-1\right)\cdot v_{21}\dots -{\rho }_{2k}\cdot v_{k1}=0\\ ⋮\\ -{\rho }_{k1}\cdot v_{11}-{\rho }_{k2}\cdot v_{21}\dots +\left({\lambda }_1^2-1\right)\cdot v_{k1}=0\end{array}\right.$

• • Determining eigenvectors v₁₂, v₂₂, …, v_k2 from the second eigenvalue (λ₂²):

$\left(\begin{array}{cccc}{\lambda }_2^2-1& -{\rho }_{12}& \cdots & -{\rho }_{1k}\\ -{\rho }_{21}& {\lambda }_2^2-1& \cdots & -{\rho }_{2k}\\ ⋮& ⋮& \ddots & ⋮\\ -{\rho }_{k1}& -{\rho }_{k2}& \cdots & {\lambda }_2^2-1\end{array}\right)\cdot \left(\begin{array}{c}{v}_{12}\\ v_{22}\\ ⋮\\ v_{k2}\end{array}\right)=\left(\begin{array}{c}0\\ 0\\ ⋮\\ 0\end{array}\right)$

from where we obtain:

$\left\{\begin{array}{c}\left({\lambda }_2^2-1\right)\cdot v_{12}-{\rho }_{12}\cdot v_{22}\dots -{\rho }_{1k}\cdot v_{k2}=0\\ -{\rho }_{21}\cdot v_{12}+\left({\lambda }_2^2-1\right)\cdot v_{22}\dots -{\rho }_{2k}\cdot v_{k2}=0\\ ⋮\\ -{\rho }_{k1}\cdot v_{12}-{\rho }_{k2}\cdot v_{22}\dots +\left({\lambda }_2^2-1\right)\cdot v_{k2}=0\end{array}\right.$

• • Determining eigenvectors v_1k, v_2k, …, v_kk from the kth eigenvalue (λ_k²):

$\left(\begin{array}{cccc}{\lambda }_k^2-1& -{\rho }_{12}& \cdots & -{\rho }_{1k}\\ -{\rho }_{21}& {\lambda }_k^2-1& \cdots & -{\rho }_{2k}\\ ⋮& ⋮& \ddots & ⋮\\ -{\rho }_{k1}& -{\rho }_{k2}& \cdots & {\lambda }_k^2-1\end{array}\right)\cdot \left(\begin{array}{c}{v}_{1k}\\ v_{2k}\\ ⋮\\ v_{\mathit{kk}}\end{array}\right)=\left(\begin{array}{c}0\\ 0\\ ⋮\\ 0\end{array}\right)$

from where we obtain:

$\left\{\begin{array}{c}\left({\lambda }_k^2-1\right)\cdot v_{1k}-{\rho }_{12}\cdot v_{2k}\dots -{\rho }_{1k}\cdot v_{\mathit{kk}}=0\\ -{\rho }_{21}\cdot v_{1k}+\left({\lambda }_k^2-1\right)\cdot v_{2k}\dots -{\rho }_{2k}\cdot v_{\mathit{kk}}=0\\ ⋮\\ -{\rho }_{k1}\cdot v_{1k}-{\rho }_{k2}\cdot v_{2k}\dots +\left({\lambda }_k^2-1\right)\cdot v_{\mathit{kk}}=0\end{array}\right.$

Thus, we can calculate the factor scores of each factor by determining the eigenvalues and eigenvectors of correlation matrix ρ. The factor scores vectors can be defined as follows:

• • Factor scores of the first factor:

${\mathbf{S}}_1=\left(\begin{array}{c}{s}_{11}\\ s_{21}\\ ⋮\\ s_{k1}\end{array}\right)=\left(\begin{array}{c}\frac{v_{11}}{\sqrt{{\lambda }_1^2}}\\ \frac{v_{21}}{\sqrt{{\lambda }_1^2}}\\ ⋮\\ \frac{v_{k1}}{\sqrt{{\lambda }_1^2}}\end{array}\right)$

• • Factor scores of the second factor:

${\mathbf{S}}_2=\left(\begin{array}{c}{s}_{12}\\ s_{22}\\ ⋮\\ s_{k2}\end{array}\right)=\left(\begin{array}{c}\frac{v_{12}}{\sqrt{{\lambda }_2^2}}\\ \frac{v_{22}}{\sqrt{{\lambda }_2^2}}\\ ⋮\\ \frac{v_{k2}}{\sqrt{{\lambda }_2^2}}\end{array}\right)$

• • Factor scores of the kth factor:

${\mathbf{S}}_k=\left(\begin{array}{c}{s}_{1k}\\ s_{2k}\\ ⋮\\ s_{\mathit{kk}}\end{array}\right)=\left(\begin{array}{c}\frac{v_{1k}}{\sqrt{{\lambda }_k^2}}\\ \frac{v_{2k}}{\sqrt{{\lambda }_k^2}}\\ ⋮\\ \frac{v_{\mathit{kk}}}{\sqrt{{\lambda }_k^2}}\end{array}\right)$

Since the factor scores of each factor are standardized by the respective eigenvalues, the factors of the set of equations presented in Expression (12.10) must be obtained by multiplying each factor score by the corresponding original variable, standardized by using the Z-scores procedure. Thus, we can obtain each one of the factors based on the following equations:

$\begin{array}{c}{F}_{1i}=\frac{v_{11}}{\sqrt{{\lambda }_1^2}}\cdot {\mathit{ZX}}_{1i}+\frac{v_{21}}{\sqrt{{\lambda }_1^2}}\cdot {\mathit{ZX}}_{2i}+\cdots +\frac{v_{k1}}{\sqrt{{\lambda }_1^2}}\cdot {\mathit{ZX}}_{\mathit{ki}}\\ F_{2i}=\frac{v_{12}}{\sqrt{{\lambda }_2^2}}\cdot {\mathit{ZX}}_{1i}+\frac{v_{22}}{\sqrt{{\lambda }_2^2}}\cdot {\mathit{ZX}}_{2i}+\cdots +\frac{v_{k2}}{\sqrt{{\lambda }_2^2}}\cdot {\mathit{ZX}}_{\mathit{ki}}\\ ⋮\\ F_{\mathit{ki}}=\frac{v_{1k}}{\sqrt{{\lambda }_k^2}}\cdot {\mathit{ZX}}_{1i}+\frac{v_{2k}}{\sqrt{{\lambda }_k^2}}\cdot {\mathit{ZX}}_{2i}+\cdots +\frac{v_{\mathit{kk}}}{\sqrt{{\lambda }_k^2}}\cdot {\mathit{ZX}}_{\mathit{ki}}\end{array}$

where ZX_i represents the standardized value of each variable X for a certain observation i. It is important to emphasize that all the factors extracted show, between themselves, Pearson correlations equal to 0, that is, they are orthogonal to one another.

A more perceptive researcher will notice that the factor scores of each factor correspond exactly to the estimated parameters of a multiple linear regression model that has, as a dependent variable, the factor itself and, as explanatory variables, the standardized variables.

Mathematically, it is also possible to verify the existing relationship between the eigenvectors, correlation matrix ρ, and eigenvalue matrix Λ². Consequently, defining eigenvector matrix V as follows:

$\mathbf{V}=\left(\begin{array}{cccc}{v}_{11}& v_{12}& \cdots & v_{1k}\\ v_{21}& v_{22}& \cdots & v_{2k}\\ ⋮& ⋮& \ddots & ⋮\\ v_{k1}& v_{k2}& \cdots & v_{\mathit{kk}}\end{array}\right)$

we can prove that:

${\mathbf{V}}^{\text{'}}\cdot \mathbf{\rho }\cdot \mathbf{V}={\mathbf{\Lambda }}^2$

or:

$\left(\begin{array}{cccc}{v}_{11}& v_{21}& \cdots & v_{k1}\\ v_{12}& v_{22}& \cdots & v_{k2}\\ ⋮& ⋮& \ddots & ⋮\\ v_{1k}& v_{2k}& \cdots & v_{\mathit{kk}}\end{array}\right)\cdot \left(\begin{array}{cccc}1& {\rho }_{12}& \cdots & {\rho }_{1k}\\ {\rho }_{21}& 1& \cdots & {\rho }_{2k}\\ ⋮& ⋮& \ddots & ⋮\\ {\rho }_{k1}& {\rho }_{k2}& \cdots & 1\end{array}\right)\cdot \left(\begin{array}{cccc}{v}_{11}& v_{12}& \cdots & v_{1k}\\ v_{21}& v_{22}& \cdots & v_{2k}\\ ⋮& ⋮& \ddots & ⋮\\ v_{k1}& v_{k2}& \cdots & v_{\mathit{kk}}\end{array}\right)=\left(\begin{array}{cccc}{\lambda }_1^2& 0& \cdots & 0\\ 0& {\lambda }_2^2& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & {\lambda }_k^2\end{array}\right)$

In Section 12.2.6, we will discuss a practical example from which this relationship may be demonstrated.

While in Section 12.2.2, we discussed the factor analysis’s overall adequacy, in this section, we will discuss the procedures for carrying out the factor extraction, if the technique is considered adequate. Even knowing that the maximum number of factors is also equal to k for k variables, it is essential for researchers to define, based on a certain criterion, the adequate number of factors that, in fact, represent the original variables. In our hypothetical example in Section 12.2.1, we saw that only two factors (F₁ and F₂) would be enough to represent the three original variables (X₁, X₂, and X₃).

Although researchers are free to determine the number of factors to be extracted in the analysis, in a preliminary way, since they may wish to verify the validity of a previously established construct (procedure known as a priori criterion), for instance, it is essential to carry out an analysis based on the magnitude of the eigenvalues calculated from correlation matrix ρ.

As the eigenvalues correspond to the proportion of variance shared by the original variables to form each factor, as we will discuss in Section 12.2.4, since λ₁² ≥ λ₂² ≥ … ≥ λ_k² and bearing in mind that factors F₁, F₂, …, F_k are obtained from the respective eigenvalues, factors extracted from smaller eigenvalues are formed from smaller proportions of variance shared by the original variables. Since a factor represents a certain cluster of variables, factors extracted from eigenvalues less than 1 may possibly not be able to represent the behavior of a single original variable (of course there are exceptions to this rule, which occur in cases in which a certain eigenvalue is less than, but also very close to 1). The criterion for choosing the number of factors, in which only the factors that correspond to eigenvalues greater than 1 are considered, is often used and known as the latent root criterion or Kaiser criterion.

The factor extraction method presented in this chapter is known as principal components, and the first factor F₁, formed by the highest proportion of variance shared by the original variables, is also called principal factor. This method is often mentioned in the existing literature and is used in practical applications whenever researchers wish to elaborate a structural reduction of the data in order to create orthogonal factors, to define observation rankings by using the factors generated, and even to confirm the validity of previously established constructs. Other factor extraction methods, such as, the generalized least squares, unweighted least squares, maximum likelihood, alpha factoring, and image factoring, have different criteria and certain specificities and, even though they can also be found in the existing literature, they will not be discussed in this book.

Moreover, it is common to discuss the need to apply the factor analysis to variables that have multivariate normal distribution, in order to show consistency when determining the factor scores. Nevertheless, it is important to emphasize that multivariate normality is a very rigid assumption, only necessary for a few factor extraction methods, such as, the maximum likelihood method. Most factor extraction methods do not require the assumption of data multivariate normality and, as discussed by Gorsuch (1983), the principal component factor analysis seems to be, in practice, very robust against breaks in normality.

12.2.4 Factor Loadings and Communalities

Having established the factors, we can now define the factor loadings, which simply are Pearson correlations between the original variables and each one of the factors. Table 12.3 shows the factor loadings for each variable-factor pair.

Based on the latent root criterion (in which only factors resulting from eigenvalues greater than 1 are considered), we assume that the factor loadings between the factors that correspond to eigenvalues less than 1 and all the original variables are low, since they will have already presented higher Pearson correlations (loadings) with factors previously extracted from greater eigenvalues. In the same way, original variables that only share a small portion of variance with the other variables will have high factor loadings in only a single factor. If this occurs for all original variables, there will not be significant differences between correlation matrix ρ and identity matrix I, making the χ_Bartlett² statistic very low. This fact allows us to state that the factor analysis will not be adequate, and, in this situation, researchers may choose not to extract factors from the original variables.

As the factor loadings are Pearson’s correlations between each variable and each factor, the sum of the squares of these loadings in each row of Table 12.3 will always be equal to 1, since each variable shares part of its proportion of variance with all the k factors, and the sum of the proportions of variance (factor loadings or squared Pearson correlations) will be 100%.

Conversely, if less than k factors are extracted, due to the latent root criterion, the sum of the squared factor loadings in each row will not be equal to 1. This sum is called communality, which represents the total shared variance of each variable in all the factors extracted from eigenvalues greater than 1. So, we can say that:

$\begin{array}{c}{c}_{11}^2+c_{12}^2+\cdots ={\text{communality}}_{X_1}\\ c_{21}^2+c_{22}^2+\cdots ={\text{communality}}_{X_2}\\ ⋮\\ c_{k1}^2+c_{k2}^2+\cdots ={\text{communality}}_{X_k}\end{array}$

The main objective of the analysis of communalities is to check if any variable ends up not sharing a significant proportion of variance with the factors extracted. Even though there is no cutoff point from which a certain communality can be considered high or low, since the sample size can interfere in this assessment, the existence of considerably low communalities in relation to the others can indicate to researchers that they may need to reconsider including the respective variable into the factor analysis.

Therefore, after defining the factors based on the factor scores, we can state that the factor loadings will be exactly the same as the parameters estimated in a multiple linear regression model that shows, as a dependent variable, a certain standardized variable ZX and, as explanatory variables, the factors themselves, and the coefficient of determination R² of each model is equal to the communality of the respective original variable.

The sum of the squared factor loadings in each column of Table 12.3, on the other hand, will be equal to the respective eigenvalue, since the ratio between each eigenvalue and the total number of variables can be understood as the proportion of variance shared by all k original variables to form each factor. So, we can say that:

$\begin{array}{c}{c}_{11}^2+c_{21}^2+\cdots +c_{k1}^2={\lambda }_1^2\\ c_{12}^2+c_{22}^2+\cdots +c_{k2}^2={\lambda }_2^2\\ ⋮\\ c_{1k}^2+c_{2k}^2+\cdots +c_{\mathit{kk}}^2={\lambda }_k^2\end{array}$

After establishing the factors and the calculation of the factor loadings, it is also possible for some variables to have intermediate (neither very high nor very low) Pearson correlations (factor loadings) with all the factors extracted, although its communality is relatively not so low. In this case, although the solution of the factor analysis has already been obtained in an adequate way and considered concluded, researchers can, in the cases in which the factor loadings table shows intermediate values for one or more variables in all the factors, elaborate a rotation of these factors, so that Pearson’s correlations between the original variables and the new factors generated can be increased. In the following section, we will discuss factor rotation.

12.2.5 Factor Rotation

Once again, let’s imagine a hypothetical situation in which a certain dataset only has three variables (k = 3). After preparing the principal component factor analysis, two factors, orthogonal to one another, are extracted, with factor loadings (Pearson correlations) with each one of the three original variables, according to Table 12.4.

In order to construct a chart with the relative positions of each variable in each factor (a chart known as loading plot), we can consider the factor loadings to be coordinates (abscissas and ordinates) of the variables in a Cartesian plane formed by both orthogonal factors. The plot can be seen in Fig. 12.5.

In order to better visualize the variables better represented by a certain factor, we can think about a rotation around the origin of the originally extracted factors F₁ and F₂, so that we can bring the points corresponding to variables X₁, X₂, and X₃ closer to one of the new factors. These are called rotated factors F₁′ and F₂′. Fig. 12.6 shows this process in a simplified way.

Based on Fig. 12.6, for each variable under analysis, we can see that while the loading for one factor increases, for the other, it decreases. Table 12.5 shows the loading redistribution for our hypothetical situation.

Thus, for a generic situation, we can say that rotation is a procedure that maximizes the loadings of each variable in a certain factor, to the detriment of the others. In this regard, the final effect of rotation is the redistribution of factor loadings to factors that initially had smaller proportions of variance shared by all the original variables. The main objective is to minimize the number of variables with high loadings in a certain factor, since each one of the factors will start having more significant loadings only with some of the original variables. Consequently, rotation may simplify the interpretation of the factors.

Despite the fact that communalities and the total proportion of variance shared by all the variables in all the factors are not modified by the rotation (and neither are the KMO statistic or χ_Bartlett²), the proportion of variance shared by the original variables in each factor is redistributed and, therefore, modified. In other words, new eigenvalues are set λ′(λ₁′,  λ₂′,  …,  λ_k′) from the rotated factor loadings. Thus, we can say that:

$\begin{array}{c}{c^{\prime }}_{11}^2+{c^{\prime }}_{12}^2+\cdots ={\text{communality}}_{X_1}\\ {c^{\prime }}_{21}^2+{c^{\prime }}_{22}^2+\cdots ={\text{communality}}_{X_2}\\ ⋮\\ {c^{\prime }}_{k1}^2+{c^{\prime }}_{k2}^2+\cdots ={\text{communality}}_{X_k}\end{array}$

and that:

$\begin{array}{c}{c^{\prime }}_{11}^2+{c^{\prime }}_{21}^2+\cdots +{c^{\prime }}_{k1}^2={{\lambda }^{\prime }}_1^2\ne {\lambda }_1^2\\ {c^{\prime }}_{12}^2+{c^{\prime }}_{22}^2+\cdots +{c^{\prime }}_{k2}^2={{\lambda }^{\prime }}_2^2\ne {\lambda }_2^2\\ ⋮\\ {c^{\prime }}_{1k}^2+{c^{\prime }}_{2k}^2+\cdots +{c^{\prime }}_{\mathit{kk}}^2={{\lambda }^{\prime }}_k^2\ne {\lambda }_k^2\end{array}$

even if Expression (12.13) is respected, that is:

${\lambda }_1^2+{\lambda }_2^2+\cdots +{\lambda }_k^2={{\lambda }^{\prime }}_1^2+{{\lambda }^{\prime }}_2^2+\cdots +{{\lambda }^{\prime }}_k^2=k$

Besides, new rotated factor scores are obtained from the rotation of factors, s′, such that the final expressions of the rotated factors will be:

$\begin{array}{c}{F}_{1i}^{\prime }=s_{11}^{\prime }\cdot {\mathit{ZX}}_{1i}+s_{21}^{\prime }\cdot {\mathit{ZX}}_{2i}+\cdots +s_{k1}^{\prime }\cdot {\mathit{ZX}}_{\mathit{ki}}\\ F_{2i}^{\prime }=s_{12}^{\prime }\cdot {\mathit{ZX}}_{1i}+s_{22}^{\prime }\cdot {\mathit{ZX}}_{2i}+\cdots +s_{k2}^{\prime }\cdot {\mathit{ZX}}_{\mathit{ki}}\\ ⋮\\ F_{\mathit{ki}}^{\prime }=s_{1k}^{\prime }\cdot {\mathit{ZX}}_{1i}+s_{2k}^{\prime }\cdot {\mathit{ZX}}_{2i}+\cdots +s_{\mathit{kk}}^{\prime }\cdot {\mathit{ZX}}_{\mathit{ki}}\end{array}$

It is important to highlight that the overall adequacy of the factor analysis (KMO statistic and Bartlett’s test of sphericity) is not altered by the rotation, since correlation matrix ρ continues the same.

Even though there are several factor rotation methods, the orthogonal rotation method, also known as Varimax, whose main purpose is to minimize the number of variables that have high loadings on a certain factor through the redistribution of the factor loadings and maximization of the variance shared in factors that correspond to lower eigenvalues, is the most frequently used and will be used in this chapter to solve a practical example. That is where the name Varimax comes from. This method was proposed by Kaiser (1958).

The algorithm behind the Varimax rotation method consists in determining a rotation angle θ in which pairs of factors are equally rotated. Thus, as discussed by Harman (1976), for a certain pair of factors F₁ and F₂, for example, the rotated factor loadings c’ between the two factors and the k original variables are obtained from the original factor loadings c, through the following matrix multiplication:

$\left(\begin{array}{cc}{c}_{11}& c_{12}\\ c_{21}& c_{22}\\ ⋮& ⋮\\ c_{k1}& c_{k2}\end{array}\right)\cdot \left(\begin{array}{cc}cos\theta & -\mathrm{sen}\theta \\ \mathrm{sen}\theta & cos\theta \end{array}\right)=\left(\begin{array}{cc}{c}_{11}^{\prime }& c_{12}^{\prime }\\ c_{21}^{\prime }& c_{22}^{\prime }\\ ⋮& ⋮\\ c_{k1}^{\prime }& c_{k2}^{\prime }\end{array}\right)$

where θ, the counterclockwise rotation angle, is obtained by the following expression:

$\theta =0.25\cdot arctan\left[\frac{2\left(D\cdot k-A\cdot B\right)}{C\cdot k-\left(A^2-B^2\right)}\right]$

where:

$A=\sum _{l=1}^k\left(\frac{c_{1l}^2}{{\text{communality}}_l}-\frac{c_{2l}^2}{{\text{communality}}_l}\right)$

$B=\sum _{l=1}^k\left(2\cdot \frac{c_{1l}\cdot c_{2l}}{{\text{communality}}_l}\right)$

$C=\sum _{l=1}^k\left[{\left(\frac{c_{1l}^2}{{\text{communality}}_l}-\frac{c_{2l}^2}{{\text{communality}}_l}\right)}^2-{\left(2\cdot \frac{c_{1l}\cdot c_{2l}}{{\text{communality}}_l}\right)}^2\right]$

$D=\sum _{l=1}^k\left[\left(\frac{c_{1l}^2}{{\text{communality}}_l}-\frac{c_{2l}^2}{{\text{communality}}_l}\right)\cdot \left(2\cdot \frac{c_{1l}\cdot c_{2l}}{{\text{communality}}_l}\right)\right]$

In Section 12.2.6, we will use these Varimax rotation method expressions to determine the rotated factor loadings from the original loadings.

Besides Varimax, we can also mention other orthogonal rotation methods, such as, Quartimax and Equamax, even though they are less frequently mentioned in the existing literature and less used in practice. In addition to them, the researcher may also use oblique rotation methods, in which nonorthogonal factors are generated. Although they are not discussed in this chapter, we should also mention the Direct Oblimin and Promax methods in this category.

Since oblique rotation methods can sometimes be used when we wish to validate a certain construct, whose initial factors are not correlated, we recommend that an orthogonal rotation method be used so that factors extracted in other multivariate techniques can be used later, such as, certain confirmatory models, in which the lack of multicollinearity of the explanatory variables is a mandatory premise.

12.2.6 A Practical Example of the Principal Component Factor Analysis

Imagine that the same professor, deeply engaged in academic and pedagogical activities, is now interested in studying how his students’ grades behave so that, afterwards, he can propose the creation of a school performance ranking.

In order to do that, he collected information on the final grades, which vary from 0 to 10, of each one of his 100 students in the following subjects: Finance, Costs, Marketing, and Actuarial Science. Part of the dataset can be seen in Table 12.6.

The complete dataset can be found in the file FactorGrades.xls. Through this dataset, it is possible to construct Table 12.7, which shows Pearson’s correlation coefficients between each pair of variables, calculated by using the logic presented in Expression (12.2).

Therefore, we can write the expression of the correlation matrix ρ as follows:

$\mathbf{\rho }=\left(\begin{array}{cccc}1& {\rho }_{12}& {\rho }_{13}& {\rho }_{14}\\ {\rho }_{21}& 1& {\rho }_{23}& {\rho }_{24}\\ {\rho }_{31}& {\rho }_{32}& 1& {\rho }_{34}\\ {\rho }_{41}& {\rho }_{42}& {\rho }_{43}& 1\end{array}\right)=\left(\begin{array}{cccc}1.000& 0.756& -0.030& 0.711\\ 0.756& 1.000& 0.003& 0.809\\ -0.030& 0.003& 1.000& -0.044\\ 0.711& 0.809& -0.044& 1.000\end{array}\right)$

which has determinant D = 0.137.

By analyzing correlation matrix ρ, it is possible to verify that only the grades corresponding to the variable marketing do not have correlations with the grades in the other subjects, represented by the other variables. On the other hand, these show relatively high correlations with one another (0.756 between finance and costs, 0.711 between finance and actuarial, and 0.809 between costs and actuarial), which indicates that they may share significant variance to form one factor. Although this preliminary analysis is important, it cannot represent more than a simple diagnostic, since the overall adequacy of the factor analysis needs to be evaluated based on the KMO statistic and, mainly, by using the result of Bartlett’s test of sphericity.

As we discussed in Section 12.2.2, the KMO statistic provides the proportion of variance considered common to all the variables present in the analysis, and, in order to establish its calculation, we need to determine partial correlation coefficients φ between each pair of variables. In this case, it will be second-order correlation coefficients, since we are working with four variables simultaneously.

Consequently, based on Expression (12.7), first, we need to determine the first-order correlation coefficients used to calculate of the second-order correlation coefficients. Table 12.8 shows these coefficients.

Table 12.8

First-Order Correlation Coefficients

${\phi }_{12,3}=\frac{{\rho }_{12}-{\rho }_{13}\cdot {\rho }_{23}}{\sqrt{\left(1-{\rho }_{13}^2\right)\cdot \left(1-{\rho }_{23}^2\right)}}=0.756$	${\phi }_{13,2}=\frac{{\rho }_{13}-{\rho }_{12}\cdot {\rho }_{23}}{\sqrt{\left(1-{\rho }_{12}^2\right)\cdot \left(1-{\rho }_{23}^2\right)}}=-0.049$	${\phi }_{14,2}=\frac{{\rho }_{14}-{\rho }_{12}\cdot {\rho }_{24}}{\sqrt{\left(1-{\rho }_{12}^2\right)\cdot \left(1-{\rho }_{24}^2\right)}}=0.258$
${\phi }_{14,3}=\frac{{\rho }_{14}-{\rho }_{13}\cdot {\rho }_{34}}{\sqrt{\left(1-{\rho }_{13}^2\right)\cdot \left(1-{\rho }_{34}^2\right)}}=0.711$	${\phi }_{23,1}=\frac{{\rho }_{23}-{\rho }_{12}\cdot {\rho }_{13}}{\sqrt{\left(1-{\rho }_{12}^2\right)\cdot \left(1-{\rho }_{13}^2\right)}}=0.039$	${\phi }_{24,1}=\frac{{\rho }_{24}-{\rho }_{12}\cdot {\rho }_{14}}{\sqrt{\left(1-{\rho }_{12}^2\right)\cdot \left(1-{\rho }_{14}^2\right)}}=0.590$
${\phi }_{24,3}=\frac{{\rho }_{24}-{\rho }_{23}\cdot {\rho }_{34}}{\sqrt{\left(1-{\rho }_{23}^2\right)\cdot \left(1-{\rho }_{34}^2\right)}}=0.810$	${\phi }_{34,1}=\frac{{\rho }_{34}-{\rho }_{13}\cdot {\rho }_{14}}{\sqrt{\left(1-{\rho }_{13}^2\right)\cdot \left(1-{\rho }_{14}^2\right)}}=-0.033$	${\phi }_{34,2}=\frac{{\rho }_{34}-{\rho }_{23}\cdot {\rho }_{24}}{\sqrt{\left(1-{\rho }_{23}^2\right)\cdot \left(1-{\rho }_{24}^2\right)}}=-0.080$

Hence, from these coefficients and by using Expression (12.8), we can calculate the second-order correlation coefficients considered in the KMO statistic’s expression. Table 12.9 shows these coefficients.

Table 12.9

Second-Order Correlation Coefficients

${\phi }_{12,34}=\frac{{\phi }_{12,3}-{\phi }_{14,3}\cdot {\phi }_{24,3}}{\sqrt{\left(1-{\phi }_{14,3}^2\right)\cdot \left(1-{\phi }_{24,3}^2\right)}}=0.438$
${\phi }_{13,24}=\frac{{\phi }_{13,2}-{\phi }_{14,2}\cdot {\phi }_{34,2}}{\sqrt{\left(1-{\phi }_{14,2}^2\right)\cdot \left(1-{\phi }_{34,2}^2\right)}}=-0.029$	${\phi }_{23,14}=\frac{{\phi }_{23,1}-{\phi }_{24,1}\cdot {\phi }_{34,1}}{\sqrt{\left(1-{\phi }_{24,1}^2\right)\cdot \left(1-{\phi }_{34,1}^2\right)}}=0.072$
${\phi }_{14,23}=\frac{{\phi }_{14,2}-{\phi }_{13,2}\cdot {\phi }_{34,2}}{\sqrt{\left(1-{\phi }_{13,2}^2\right)\cdot \left(1-{\phi }_{34,2}^2\right)}}=0.255$	${\phi }_{24,13}=\frac{{\phi }_{24,1}-{\phi }_{23,1}\cdot {\phi }_{34,1}}{\sqrt{\left(1-{\phi }_{23,1}^2\right)\cdot \left(1-{\phi }_{34,1}^2\right)}}=0.592$	${\phi }_{34,12}=\frac{{\phi }_{34,1}-{\phi }_{23,1}\cdot {\phi }_{24,1}}{\sqrt{\left(1-{\phi }_{23,1}^2\right)\cdot \left(1-{\phi }_{24,1}^2\right)}}=-0.069$

So, based on Expression (12.3), we can calculate the KMO statistic. The terms of the expression are given by:

$\sum _{l=1}^k\sum _{c=1}^k{\rho }_{\mathit{lc}}^2={\left(0.756\right)}^2+{\left(-0.030\right)}^2+{\left(0.711\right)}^2+{\left(0.003\right)}^2+{\left(0.809\right)}^2+{\left(-0.044\right)}^2=1.734$

$\sum _{l=1}^k\sum _{c=1}^k{\phi }_{\mathit{lc}}^2={\left(0.438\right)}^2+{\left(-0.029\right)}^2+{\left(0.255\right)}^2+{\left(0.072\right)}^2+{\left(0.592\right)}^2+{\left(-0.069\right)}^2=0.619$

from where we obtain:

$\mathrm{KMO}=\frac{1.734}{1.734+0.619}=0.737$

Based on the criterion presented in Table 12.2, the value of the KMO statistic suggests that the overall adequacy of the factor analysis is middling. To test whether, in fact, correlation matrix ρ is statistically different from identity matrix I with the same dimension, we must use Bartlett’s test of sphericity, whose χ_Bartlett² statistic is given by Expression (12.9). For n = 100 observations, k = 4 variables, and correlation matrix ρ determinant D = 0.137, we have:

${\chi }_{\text{Bartlett}}^2=-\left[\left(100-1\right)-\left(\frac{2\cdot 4+5}{6}\right)\right]\cdot ln\left(0.137\right)=192.335$

with $\frac{4\cdot \left(4-1\right)}{2}=6$ degrees of freedom. Therefore, by using Table D in the Appendix, we have χ_c² = 12.592 (critical χ² for 6 degrees of freedom and with a significance level of 0.05). Thus, since χ_Bartlett² = 192.335 > χ_c² = 12.592, we can reject the null hypothesis that correlation matrix ρ is statistically equal to identity matrix I, at a significance level of 0.05.

Software packages like SPSS and Stata do not offer the χ_c² for the defined degrees of freedom and a certain significance level. However, they offer the significance level of χ_Bartlett² for these degrees of freedom. So, instead of analyzing if χ_Bartlett² > χ_c², we must verify if the significance level of χ_Bartlett² is less than 0.05 (5%) so that we can continue performing the factor analysis. Thus:

If P-value (either Sig. χ_Bartlett², or Prob. χ_Bartlett²) < 0.05, correlation matrix ρ is not statistically equal to identity matrix I with the same dimension.

The significance level of χ_Bartlett² can be obtained in Excel by using the command Formulas → Insert Function→ CHIDIST, which will open a dialog box, as shown in Fig. 12.7.

As we can see in Fig. 12.7, the P-value of the χ_Bartlett² statistic is considerably less than 0.05 (χ_Bartlett² P-value = 8.11 × 10^− 39), that is, Pearson’s correlations between the pairs of variables are statistically different from 0 and, therefore, factors can be extracted from the original variables, and the factor analysis very adequate.

Having verified the factor analysis’s overall adequacy, we can move on to the definition of the factors. In order to do that, we must initially determine the four eigenvalues λ² (λ₁² ≥ λ₂² ≥ λ₃² ≥ λ₄²) of correlation matrix ρ, which can be obtained from solving Expression (12.12). Therefore, we have:

$\left|\begin{array}{cccc}{\lambda }^2-1& -0.756& 0.030& -0.711\\ -0.756& {\lambda }^2-1& -0.003& -0.809\\ 0.030& -0.003& {\lambda }^2-1& 0.044\\ -0.711& -0.809& 0.044& {\lambda }^2-1\end{array}\right|=0$

from where we obtain:

$\left\{\begin{array}{c}{\lambda }_1^2=2.519\\ {\lambda }_2^2=1.000\\ {\lambda }_3^2=0.298\\ {\lambda }_4^2=0.183\end{array}\right.$

Consequently, based on Expression (12.15), eigenvalue matrix Λ² can be written as follows:

${\mathbf{\Lambda }}^2=\left(\begin{array}{cccc}2.519& 0& 0& 0\\ 0& 1.000& 0& 0\\ 0& 0& 0.298& 0\\ 0& 0& 0& 0.183\end{array}\right)$

Note that Expression (12.13) is satisfied, that is:

${\lambda }_1^2+{\lambda }_2^2+\cdots +{\lambda }_k^2=2.519+1.000+0.298+0.183=4$

Since the eigenvalues correspond to the proportion of variance shared by the original variables to form each factor, we can construct a shared variance table (Table 12.10).

Table 12.10

Variance Shared by the Original Variables to Form Each Factor

Factor	Eigenvalue λ2	Shared Variance (%)	Cumulative Shared Variance (%)
1	2.519	$\left(\frac{2.519}{4}\right)\cdot 100=62.975$	62.975
2	1.000	$\left(\frac{1.000}{4}\right)\cdot 100=25,010$	87.985
3	0.298	$\left(\frac{0.298}{4}\right)\cdot 100=7.444$	95.428
4	0.183	$\left(\frac{0.183}{4}\right)\cdot 100=4.572$	100.000

By analyzing Table 12.10, we can say that while 62.975% of the total variance are shared to form the first factor, 25.010% are shared to form the second factor. The third and fourth factors, whose eigenvalues are less than 1, are formed through smaller proportions of shared variance. Since the most common criterion used to choose the number of factors is the latent root criterion (Kaiser criterion), in which only the factors that correspond to eigenvalues greater than 1 are taken into consideration, the researcher can choose to conduct all the subsequent analysis with only the first two factors, formed by sharing 87.985% of the total variance of the original variables, that is, with a total variance loss of 12.015%. Nonetheless, for pedagogical purposes, let’s discuss how to calculate the factor scores by determining the eigenvectors that correspond to the four eigenvalues.

Consequently, in order to define the eigenvectors of matrix ρ based on the four eigenvalues calculated, we must solve the following equation systems for each eigenvalue, based on Expressions (12.16)–(12.21):

• • Determining eigenvectors v₁₁, v₂₁, v₃₁, v₄₁ from the first eigenvalue (λ₁² = 2.519):

$\left\{\begin{array}{c}\left(2.519-1.000\right)\cdot v_{11}-0.756\cdot v_{21}+0.030\cdot v_{31}-0.711\cdot v_{41}=0\\ -0.756\cdot v_{11}+\left(2.519-1.000\right)\cdot v_{21}-0.003\cdot v_{31}-0.809\cdot v_{41}=0\\ 0.030\cdot v_{11}-0.003\cdot v_{21}+\left(2.519-1.000\right)\cdot v_{31}+0.044\cdot v_{41}=0\\ -0.711\cdot v_{11}-0.809\cdot v_{21}+0.044\cdot v_{31}+\left(2.519-1.000\right)\cdot v_{41}=0\end{array}\right.$

from where we obtain:

$\left(\begin{array}{c}{v}_{11}\\ v_{21}\\ v_{31}\\ v_{41}\end{array}\right)=\left(\begin{array}{c}0.5641\\ 0.5887\\ -0.0267\\ 0.5783\end{array}\right)$

• • Determining eigenvectors v₁₂, v₂₂, v₃₂, v₄₂ from the second eigenvalue (λ₂² = 1.000):

$\left\{\begin{array}{c}\left(1.000-1.000\right)\cdot v_{12}-0.756\cdot v_{22}+0.030\cdot v_{32}-0.711\cdot v_{42}=0\\ -0.756\cdot v_{12}+\left(1.000-1.000\right)\cdot v_{22}-0.003\cdot v_{32}-0.809\cdot v_{42}=0\\ 0.030\cdot v_{12}-0.003\cdot v_{22}+\left(1.000-1.000\right)\cdot v_{32}+0.044\cdot v_{42}=0\\ -0.711\cdot v_{12}-0.809\cdot v_{22}+0.044\cdot v_{32}+\left(1.000-1.000\right)\cdot v_{42}=0\end{array}\right.$

from where we obtain:

$\left(\begin{array}{c}{v}_{12}\\ v_{22}\\ v_{32}\\ v_{42}\end{array}\right)=\left(\begin{array}{c}0.0068\\ 0.0487\\ 0.9987\\ -0.0101\end{array}\right)$

• • Determining eigenvectors v₁₃, v₂₃, v₃₃, v₄₃ from the third eigenvalue (λ₃² = 0.298):

$\left\{\begin{array}{c}\left(0.298-1.000\right)\cdot v_{13}-0.756\cdot v_{23}+0.030\cdot v_{33}-0.711\cdot v_{43}=0\\ -0.756\cdot v_{13}+\left(0.298-1.000\right)\cdot v_{23}-0.003\cdot v_{33}-0.809\cdot v_{43}=0\\ 0.030\cdot v_{13}-0.003\cdot v_{23}+\left(0.298-1.000\right)\cdot v_{33}+0.044\cdot v_{43}=0\\ -0.711\cdot v_{13}-0.809\cdot v_{23}+0.044\cdot v_{33}+\left(0.298-1.000\right)\cdot v_{43}=0\end{array}\right.$

from where we obtain:

$\left(\begin{array}{c}{v}_{13}\\ v_{23}\\ v_{33}\\ v_{43}\end{array}\right)=\left(\begin{array}{c}0.8008\\ -0.2201\\ -0.0003\\ -0.5571\end{array}\right)$

• • Determining eigenvectors v₁₄, v₂₄, v₃₄, v₄₄ from the fourth eigenvalue (λ₄² = 0.183):

$\left\{\begin{array}{c}\left(0.183-1.000\right)\cdot v_{14}-0.756\cdot v_{24}+0.030\cdot v_{34}-0.711\cdot v_{44}=0\\ -0.756\cdot v_{14}+\left(0.183-1.000\right)\cdot v_{24}-0.003\cdot v_{34}-0.809\cdot v_{44}=0\\ 0.030\cdot v_{14}-0.003\cdot v_{24}+\left(0.183-1.000\right)\cdot v_{34}+0.044\cdot v_{44}=0\\ -0.711\cdot v_{14}-0.809\cdot v_{24}+0.044\cdot v_{34}+\left(0.183-1.000\right)\cdot v_{44}=0\end{array}\right.$

from where we obtain:

$\left(\begin{array}{c}{v}_{14}\\ v_{24}\\ v_{34}\\ v_{44}\end{array}\right)=\left(\begin{array}{c}0.2012\\ -0.7763\\ 0.0425\\ 0.5959\end{array}\right)$

After having determined the eigenvectors, a more inquisitive researcher may prove the relationship presented in Expression (12.27), that is:

${\mathbf{V}}^{\prime }\cdot \mathbf{\rho }\cdot \mathbf{V}={\mathbf{\Lambda }}^2$

$\begin{array}{l}\left(\begin{array}{cccc}0.5641& 0.5887& -0.0267& 0.5783\\ 0.0068& 0.0487& 0.9987& -0.0101\\ 0.8008& -0.2201& -0.0003& -0.5571\\ 0.2012& -0.7763& 0.0425& 0.5959\end{array}\right)\cdot \left(\begin{array}{cccc}1.000& 0.756& -0.030& 0.711\\ 0.756& 1.000& 0.003& 0.809\\ -0.030& 0.003& 1.000& -0.044\\ 0.711& 0.809& -0.044& 1.000\end{array}\right)\cdot \left(\begin{array}{cccc}0.5641& 0.0068& 0.8008& 0.2012\\ 0.5887& 0.0487& -0.2201& -0.7763\\ -0.0267& 0.9987& -0.0003& 0.0425\\ 0.5783& -0.0101& -0.5571& 0.5959\end{array}\right)\\ =\left(\begin{array}{cccc}2.519& 0& 0& 0\\ 0& 1.000& 0& 0\\ 0& 0& 0.298& 0\\ 0& 0& 0& 0.183\end{array}\right)\end{array}$

Based on Expressions (12.22)–(12.24), we can calculate the factor scores that correspond to each one of the standardized variables for each one of the factors. Thus, from Expression (12.25), we are able to write the expressions for factors F₁, F₂, F₃, and F₄, as follows:

$\begin{array}{c}{F}_{1i}=\frac{0.5641}{\sqrt{2.519}}\cdot {\mathit{Zfinance}}_i+\frac{0.5887}{\sqrt{2.519}}\cdot {\mathit{Zcosts}}_i-\frac{0.267}{\sqrt{2.519}}\cdot {\mathit{Zmarketing}}_i+\frac{0.5783}{\sqrt{2.519}}\cdot {\mathit{Zactuarial}}_i\\ F_{2i}=\frac{0.0068}{\sqrt{1.000}}\cdot {\mathit{Zfinance}}_i+\frac{0.0487}{\sqrt{1.000}}\cdot {\mathit{Zcosts}}_i+\frac{0.9987}{\sqrt{1.000}}\cdot {\mathit{Zmarketing}}_i-\frac{0.0101}{\sqrt{1.000}}\cdot {\mathit{Zactuarial}}_i\\ F_{3i}=\frac{0.8008}{\sqrt{0.298}}\cdot {\mathit{Zfinance}}_i-\frac{0.2201}{\sqrt{0.298}}\cdot {\mathit{Zcosts}}_i-\frac{0.0003}{\sqrt{0.298}}\cdot {\mathit{Zmarketing}}_i-\frac{0.5571}{\sqrt{0.298}}\cdot {\mathit{Zactuarial}}_i\\ F_{4i}=\frac{0.2012}{\sqrt{0.183}}\cdot {\mathit{Zfinance}}_i-\frac{0.7763}{\sqrt{0.183}}\cdot {\mathit{Zcosts}}_i+\frac{0.0425}{\sqrt{0.183}}\cdot {\mathit{Zmarketing}}_i+\frac{0.5959}{\sqrt{0.183}}\cdot {\mathit{Zactuarial}}_i\end{array}$