Matrix eigenvalue theory

Suppose that ${\bf A}$ is a real symmetric square matrix of dimension $n$. If follows that ${\bf A}^\ast = {\bf A}$ and ${\bf A}^T = {\bf A}$, where $~^\ast$ denotes a complex conjugate, and $~^T$ denotes a transpose. Consider the matrix equation

$${\bf A} \,{\bf x} = \lambda\,{\bf x}.$$ (A.144)

Any column vector ${\bf x}$ that satisfies this equation is called an eigenvector of ${\bf A}$. Likewise, the associated number $\lambda$ is called an eigenvalue of ${\bf A}$ (Gradshteyn and Ryzhik 1980c). Let us investigate the properties of the eigenvectors and eigenvalues of a real symmetric matrix.

Equation (A.144) can be rearranged to give

$$({\bf A} - \lambda\,{\bf 1})\,{\bf x} = {\bf0},$$ (A.145)

where ${\bf 1}$ is the unit matrix. The preceding matrix equation is essentially a set of $n$ homogeneous simultaneous algebraic equations for the $n$ components of ${\bf x}$. A well-known property of such a set of equations is that it only has a nontrivial solution when the determinant of the associated matrix is set to zero (Gradshteyn and Ryzhik 1980c). Hence, a necessary condition for the preceding set of equations to have a nontrivial solution is that

$$\vert{\bf A} - \lambda\,{\bf 1}\vert = 0,$$ (A.146)

where $\vert\cdots\vert$ denotes a determinant. This formula is essentially an $n$th-order polynomial equation for $\lambda$. We know that such an equation has $n$ (possibly complex) roots. Hence, we conclude that there are $n$ eigenvalues, and $n$ associated eigenvectors, of the $n$-dimensional matrix ${\bf A}$.

Let us now demonstrate that the $n$ eigenvalues and eigenvectors of the real symmetric matrix ${\bf A}$ are all real. We have

$${\bf A}\,{\bf x}_i = \lambda_i\,{\bf x}_i,$$ (A.147)

and, taking the transpose and complex conjugate,

$${\bf x}_i^{\ast\,T}\,{\bf A} = \lambda_i^{\,\ast}\,{\bf x}_i^{\ast\,T},$$ (A.148)

where ${\bf x}_i$ and $\lambda_i$ are the $i$th eigenvector and eigenvalue of ${\bf A}$, respectively. Left multiplying Equation (A.147) by ${\bf x}_i^{\ast\,T}$, we obtain

$${\bf x}_i^{\ast\,T} {\bf A}\,{\bf x}_i = \lambda_i\,{\bf x}_i^{\ast\,T}{\bf x}_i.$$ (A.149)

Likewise, right multiplying Equation (A.148) by ${\bf x}_i$, we get

$${\bf x}_i^{\ast\,T}\,{\bf A}\,{\bf x}_i = \lambda_i^{\,\ast}\,{\bf x}_i^{\ast\,T}{\bf x}_i.$$ (A.150)

The difference of the previous two equations yields

$$(\lambda_i - \lambda_i^{\,\ast})\,{\bf x}_i^{\ast\,T} {\bf x}_i = 0.$$ (A.151)

It follows that $\lambda_i=\lambda_i^{\,\ast}$, because ${\bf x}_i^{\ast\,T}{\bf x}_i$ (which is ${\bf x}_i^{\,\ast}\cdot{\bf x}_i$ in vector notation) is real and positive definite. Hence, $\lambda_i$ is real. It immediately follows that ${\bf x}_i$ is real.

Next, let us show that two eigenvectors corresponding to two different eigenvalues are mutually orthogonal. Let

$${\bf A}\,{\bf x}_i = \lambda_i\,{\bf x}_i,$$ (A.152)

$${\bf A}\,{\bf x}_j = \lambda_j\,{\bf x}_j,$$ (A.153)

where $\lambda_i\neq \lambda_j$. Taking the transpose of the first equation and right multiplying by ${\bf x}_j$, and left multiplying the second equation by ${\bf x}_i^T$, we obtain

$${\bf x}_i^T\,{\bf A}\,{\bf x}_j = \lambda_i\,{\bf x}_i^T{\bf x}_j,$$ (A.154)

$${\bf x}_i^T\,{\bf A}\,{\bf x}_j = \lambda_j\,{\bf x}_i^T{\bf x}_j.$$ (A.155)

Taking the difference of the preceding two equations, we get

$$(\lambda_i-\lambda_j)\,{\bf x}_i^T{\bf x}_j = 0.$$ (A.156)

Because, by hypothesis, $\lambda_i\neq \lambda_j$, it follows that ${\bf x}_i^T{\bf x}_j = 0$. In vector notation, this is the same as ${\bf x}_i \cdot{\bf x}_j=0$. Hence, the eigenvectors ${\bf x}_i$ and ${\bf x}_j$ are mutually orthogonal.

Suppose that $\lambda_i=\lambda_j=\lambda$. In this case, we cannot conclude that ${\bf x}_i^T{\bf x}_j = 0$ by the preceding argument. However, it is easily seen that any linear combination of ${\bf x}_i$ and ${\bf x}_j$ is an eigenvector of ${\bf A}$ with eigenvalue $\lambda$. Hence, it is possible to define two new eigenvectors of ${\bf A}$, with the eigenvalue $\lambda$, that are mutually orthogonal. For instance,

$${\bf x}_i' = {\bf x}_i,$$ (A.157)

$${\bf x}_j' = {\bf x}_j - \left(\frac{{\bf x}_i^T{\bf x}_j}{{\bf x}_i^T{\bf x}_i}\right) {\bf x}_i.$$ (A.158)

It should be clear that this argument can be generalized to deal with any number of eigenvalues that take the same value.

In conclusion, a real symmetric $n$-dimensional matrix possesses $n$ real eigenvalues, with $n$ associated real eigenvectors, that are, or can be chosen to be, mutually orthogonal.