More Matrix Eigenvalue Theory

Equation (785) takes the form of a matrix eigenvalue equation:

$$({\bf G} - \lambda\,{\bf M})\,\bf {x} = {\bf0}.$$ (786)

Here, ${\bf G}$ and ${\bf M}$ are both real symmetric matrices, whereas $\lambda$ is termed the eigenvalue, and ${\bf x}$ the associated eigenvector. The above matrix equation is essentially a set of ${\cal F}$ homogeneous simultaneous algebraic equations for the components of ${\bf x}$. As is well-known, a necessary condition for such a set of equations to possess a non-trivial solution is that the determinant of the matrix must be zero: i.e.,

$$\vert{\bf G} -\lambda\,{\bf M}\vert = 0.$$ (787)

The above formula reduces to an ${\cal F}$th-order polynomial equation for $\lambda$. Hence, we conclude that Equation (786) is satisfied by ${\cal F}$ eigenvalues, and ${\cal F}$ associated eigenvectors.

We can easily demonstrate that the eigenvalues are all real. Suppose that $\lambda_k$ and ${\bf x}_k$ are the $k$th eigenvalue and eigenvector, respectively. Then we have

$${\bf G}\,{\bf x}_k = \lambda_k\,{\bf M}\,{\bf x}_k.$$ (788)

Taking the transpose and complex conjugate of the above equation, and right multiplying by ${\bf x}_k$, we obtain

$${\bf x}_k^{\ast\,T}\,{\bf G}^{\ast\,T}\,{\bf x}_k = \lambda_k^\ast\, {\bf x}_k^{\ast\,T}\,{\bf M}^{\ast\,T}\,{\bf x}_k.$$ (789)

Here, $~^T$ denotes a transpose, and $~^\ast$ a complex conjugate. However, since ${\bf G}$ and ${\bf M}$ are both real symmetric matrices, it follows that ${\bf G}^{\ast\,T} = {\bf G}$ and ${\bf M}^{\ast\,T}={\bf M}$. Hence,

$${\bf x}_k^{\ast\,T}\,{\bf G}\,{\bf x}_k = \lambda_k^\ast\, {\bf x}_k^{\ast\,T}\,{\bf M}\,{\bf x}_k.$$ (790)

Next, left multiplying Equation (788) by ${\bf x}_k^{\ast\,T}$, we obtain

$${\bf x}_k^{\ast\,T}\,{\bf G}\,{\bf x}_k = \lambda_k\, {\bf x}_k^{\ast\,T}\,{\bf M}\,{\bf x}_k.$$ (791)

Taking the difference between the above two expressions, we get

$$(\lambda_k^\ast-\lambda_k)\,{\bf x}_k^{\ast\,T}\,{\bf M}\,{\bf x}_k = 0.$$ (792)

Since ${\bf x}_k^{\ast\,T}\,{\bf M}\,{\bf x}_k$ is not generally zero, except in the trivial case where ${\bf x}_k$ is a null vector, we conclude that $\lambda_k^\ast = \lambda_k$ for all $k$. In other words, the eigenvalues are all real. It immediately follows that the eigenvectors can also be chosen to be all real.

Consider two distinct eigenvalues, $\lambda_k$ and $\lambda_l$, with the associated eigenvectors ${\bf x}_k$ and ${\bf x}_l$, respectively. We have

$${\bf G}\,{\bf x}_k = \lambda_k\,{\bf M}\,{\bf x}_k,$$ (793)

$${\bf G}\,{\bf x}_l = \lambda_l\,{\bf M}\,{\bf x}_l.$$ (794)

Right multiplying the transpose of Equation (793) by ${\bf x}_l$, and left multiplying Equation (794) by ${\bf x}_k^T$, we obtain

$${\bf x}_k^T\,{\bf G}\,{\bf x}_l = \lambda_k\,{\bf x}_k^T\,{\bf M}\,{\bf x}_l,$$ (795)

$${\bf x}_k^T\,{\bf G}\,{\bf x}_l = \lambda_l\,{\bf x}_k^T\,{\bf M}\,{\bf x}_l.$$ (796)

Taking the difference between the above two expressions, we get

$$(\lambda_k-\lambda_l)\,{\bf x}_k^T\,{\bf M}\,{\bf x}_l = 0.$$ (797)

Hence, we conclude that

$${\bf x}_k^T\,{\bf M}\,{\bf x}_l = 0,$$ (798)

provided $\lambda_k\neq \lambda_l$. In other words, two eigenvectors corresponding to two different eigenvalues are ``orthogonal'' to one another (in the sense specified in the above equation). Moreover, it is easily demonstrated that different eigenvectors corresponding to the same eigenvalue can be chosen in such a manner that they are also orthogonal to one another--see Section 8.5. Thus, we conclude that all of the eigenvectors are mutually orthogonal. Since Equation (786) only specifies the directions, and not the lengths, of the eigenvectors, we are free to normalize our eigenvectors such that

$${\bf x}_k^T\,{\bf M}\,{\bf x}_l = \delta_{kl},$$ (799)

where $\delta_{kl}=1$ when $k=l$, and $\delta_{kl}=0$ otherwise. Note, finally, that since the ${\bf x}_k$, for $k=1,{\cal F}$, are mutually orthogonal, they are independent (i.e., one eigenvector cannot be expressed as a linear combination of the others), and completely span ${\cal F}$-dimensional vector space.