THE SPECTRAL DECOMPOSITION OF SYMMETRIC MATRICES

The picture above is an application of singular value decomposition in image processing. This post will discuss a special case of SVD, where the matrix to be decomposed is a symmetric matrix. The study of symmetric matrices is important in order to understand multivariate statistics. The variance-covariance matrix, which is often used in multivariate statistics, is an example of a symmetric matrix. The first result that can be obtained if a matrix is symmetric is shown in the following theorem.

Theorem 1

Let A be a k × k symmetric matrix. Then A has k pairs of eigenvalues and eigenvectors namely $({\lambda}_1, \vec{e}_1}), ({\lambda}_2, \vec{e}_2}), \ldots , ({\lambda}_k, \vec{e}_k})$ . The eigenvectors can be chosen such that their norms are 1 and be mutually perpendicular. The eigenvectors are unique unless two or more eigenvalues are equal.

The spectral decomposition of symmetric matrices is performed by applying the following theorem.

Theorem 2 [Spectral Decomposition of Symmetric Matrices]

Let A be a k × k symmetric matrix, $\lambda_1, \lambda_2, \ldots \lambda_k$ are the eigenvalues of A, and $\vec{e}_1, \vec{e}_2, \ldots , \vec{e}_k$ are the associated normalized eigenvectors of A which are mutually perpendicular. Then the matrix A can be expressed as:

Are the eigenvectors obtained from symmetric matrices perpendicular to each other? The theorem below answers this question.

Theorem 3

If A is a symmetric matrix, then eigenvectors from different eigenspaces are orthogonal.

The theorem implies that the eigenvectors referred to in Theorem 2 are not necessarily perpendicular to each other. However, if the eigenvectors come from different eigenspaces, then they are perpendicular to each other.

Are the eigenvectors obtained from symmetric matrices independent of each other? Let the next theorem address it.

Theorem 4

If $\vec{v}_1, \vec{v}_2, \ldots , \vec{v}_n$ are eigenvectors of A obtained corresponding to distinct eigenvalues $\lambda_1, \lambda_2, \ldots , \lambda_n$ then $\vec{v}_1, \vec{v}_2, \ldots , \vec{v}_n$ are independent.

Referring to Theorem 4. the eigenvectors in Theorem 2 can be dependent. However, if the eigenvectors are obtained from distinct eigenvalues, then the eigenvectors are independent of each other.

Example

Consider the matrix A as follows.

What is the spectral decomposition of A? Is the decomposition unique?

Answer
It can be shown that the characteristic equation of A is $(\lambda - 12)(\lambda - 6)^{2} = 0$ . For $\lambda = \lambda_1 = 12$ we get the eigenspace:

The basis of E₁ with norm 1 is:

For $\lambda = \lambda_2 =\lambda_3 = 6$ we get the eigenspace:

In order to determine the spectral decomposition of A, we need the eigenvectors that are perpendicular to each other and have norms of 1. In this case, Theorem 3 ensures that the eigenvectors in E₁ are perpendicular to the eigenvectors in E₂. Now we have to find the eigenvectors in E₂ that are perpendicular to each other. E₂ is spanned by $\vec{u} = \begin{pmatrix}2 \\ 0 \\ 1 \end{pmatrix}$ and $\vec{v} = \begin{pmatrix}1 \\ 1 \\ 0 \end{pmatrix}$ as the basis vectors. To obtain an orthonormal basis for E₂, we can apply the Gram-Schmidt process. Normalization of $\vec{v}$ produces one of its orthonormal basis vectors, namely $\vec{e}_2 = \begin{pmatrix}1/{\sqrt{2}} \\ 1/{ \sqrt{2}} \\ 0 \end{pmatrix}$ . The projection of $\vec{u}$ on the vector space spanned by $\vec{e}_2$ is:

The component of $\vec{u}$ perpendicular to the projection is:

Normalizing this vector results in $\vec{e}_3 = \begin{pmatrix}1/{\sqrt{3}} \\ -1/{\sqrt{3}} \\ 1/{\sqrt{3} } \end{pmatrix}$ . The vectors $\vec{e}_3$ and $\vec{e}_2$ span the eigenspace E₂, so $\{ \vec{e}_2, \vec{e}_3 \}$ is an orthonormal basis for E₂. So, the spectral decomposition of A is:

The decomposition is not unique. There are infinitely possible bases for E₂. For instance, we can take two independent vectors in E₂such as $\vec{p} = \begin{pmatrix}0 \\ 2 \\ -1 \end{pmatrix}$ and $\vec{q} = \begin{pmatrix}3 \\ 1 \\ 1 \end{pmatrix}$ . Note that $\{ \vec{p}, \vec{q} \}$ is also a basis of E₂. With the Gram-Schmidt process, we can have another orthonormal basis. To demonstrate this, normalize $\vec{p}$ to produce a basis vector with norm 1, i.e. $\vec{f}_2 = \begin{pmatrix}0 \\ 2/{\sqrt{ 5}} \\ -1/{\sqrt{5}} \end{pmatrix}$ . The projection of the vector $\vec{q}$ on the vector space spanned by $\vec{f}_2$ is:

The component of $\vec{q}$ perpendicular to the projection is:

Normalizing this vector results in $\vec{f}_3 = \begin{pmatrix} 5/{\sqrt{30}} \\ 1/{\sqrt{30}} \\ 2/{\sqrt{30}} \end{pmatrix}$ . This vector and $\vec{f}_2$ span the eigenspace E₂, so $\{ \vec{f}_2, \vec{f}_3 \}$ is another orthonormal basis of E₂. So, the spectral decomposition for A is:

On 24/02/2022

Leave a Reply Cancel reply

Related Topics

THE GRAM-SCHMIDT PROCESS

DIAGONALIZATION OF SYMMETRIC MATRICES

SIMILAR MATRICES