LDA works as a dimensionality reduction tool, just like PCA, however instead of calculating the eigenvalues of the covariance matrix of the data as a whole, LDA calculates eigenvalues and eigenvectors of within-class and between-class scatter matrices. Performing LDA can be broken down into five steps:
-
Calculate mean vectors of each class
-
Calculate within-class and between-class scatter matrices
-
Calculate eigenvalues and eigenvectors for
-
Keep the top k eigenvectors by ordering them by descending eigenvalues
-
Use the top eigenvectors to project onto the new space
Let's look at an example.