i
i
i
i
i
i
i
i
5.4. Eigenvalues and Matrix Diagonalization 107
This form used the standard symbol σ
i
for the ith singular value. Again, for a
symmetric matrix, the eigenvalues and the singular values are the same (σ
i
= λ
i
).
We will examine the geometry of SVD further in Section 6.1.6.
Frequently Asked Questions
• Why is matrix multiplication defined the way it is rather than just element
by element?
Element by element multiplication is a perfectly good way to define matrix mul-
tiplication, and indeed it has nice properties. However, in practice it is not very
useful. Ultimately most matrices are used to transform column vectors, e.g., in
3D you might have
b = Ma,
where a and b are vectors and M is a 3×3 matrix. To allow geometric operations
such as rotation, combinations of all three elements of a must go into each element
of b. That requires us to either go row-by-row or column-by-column through M.
That choice is made based on composition of matrices having the desired property,
M
2
(M
1
a)=(M
2
M
1
)a
which allows us to use one composite matrix C = M
2
M
1
to transformour vector.
This is valuable when many vectors will be transformed by the same composite
matrix. So, in summary, the somewhat weird rule for matrix multiplication is en-
gineered to have these desired properties.
• Sometimes I hear that eigenvalues and singular values are the same
thing and sometimes that one is the square of the other. Which is right?
If a real matrix A is symmetric, and its eigenvalues are non-negative, then its
eigenvalues and singular values are the same. If A is not symmetric, the ma-
trix M = AA
T
is symmetric and has non-negative real eignenvalues. The sin-
gular values of A and A
T
are the same and are the square roots of the singu-
lar/eigenvalues of M. Thus, when the square root statement is made, it is because
two different matrices (with a very particular relationship) are being talked about:
M = AA
T
.
Notes
The discussion of determinants as volumes is based on A Vector Space Approach
to Geometry (Hausner, 1998). Hausner has an excellent discussion of vector