THE GRAM-SCHMIDT PROCESS - edsmathscholar.com

The Gram-Schmidt Process produces an orthogonal basis for a nonzero subspace of $\mathbb{R}^n$ . It is based on the following theorem.

Theorem

Let $\begin{Bmatrix}\vec{b}_1, \: \vec{b}_2, \: \cdots , \vec{b}_m \end{Bmatrix}$ be a basis for a nonzero subspace W of $\mathbb{R}^n$ and:

$\\ \vec{c}_1 = \vec{b}_1 \\ \vec{c}_2 = \vec{b}_2 - \frac{\langle \vec{b}_2 , \vec{c}_1 \rangle}{\langle \vec{c}_1 , \vec{c}_1 \rangle} \vec{c}_1 \\ \vec{c}_3 = \vec{b}_3 - \frac{\langle \vec{b}_3 , \vec{c}_1 \rangle}{\langle \vec{c}_1 , \vec{c}_1 \rangle} \vec{c}_1 - \frac{\langle \vec{b}_3 , \vec{c}_2 \rangle}{\langle \vec{c}_2 , \vec{c}_2 \rangle} \vec{c}_2 \\ \vdots \\ \vec{c}_m = \vec{b}_m - \frac{\langle \vec{b}_m , \vec{c}_1 \rangle}{\langle \vec{c}_1 , \vec{c}_1 \rangle} \vec{c}_1 - \frac{\langle \vec{b}_m , \vec{c}_2 \rangle}{\langle \vec{c}_2 , \vec{c}_2 \rangle} \vec{c}_2 - \cdots - \frac{\langle \vec{b}_m , \vec{c}_{m-1} \rangle}{\langle \vec{c}_{m-1} , \vec{c}_{m-1} \rangle} \vec{c}_{m-1}$ .

Then $\begin{Bmatrix}\vec{c}_1, \: \vec{c}_2, \: \cdots , \vec{c}_m \end{Bmatrix}$ is an orthogonal basis for W. In addition, the subspace spanned by $\begin{Bmatrix}\vec{b}_1, \: \vec{b}_2, \: \cdots , \vec{b}_k \end{Bmatrix}$ is the same as that spanned by $\begin{Bmatrix}\vec{c}_1, \: \vec{c}_2, \: \cdots , \vec{c}_k \end{Bmatrix}$ for 1 ≤ k ≤ m.

Note:

In the theorem above, $\langle \vec{u} , \vec{v} \rangle$ denotes the inner product of $\vec{u}$ and $\vec{v}$ .

Example 1

Let W be a subspace of $\mathbb{R}^4$ spanned by $B = \begin{Bmatrix}\vec{b}_1, \: \vec{b}_2, \: \vec{b}_3 \end{Bmatrix}$ where $\vec{b}_1 = (2,-1,0,2), \: \vec{b}_2 = (1,1,1,-1), \: \vec{b}_3 = (0,2,-2,1)$ . Find an orthogonal basis for W by applying the Gram-Schmidt Process. (Use the Euclidean inner product $\langle \vec{v}_1 , \vec{v}_2\rangle = \vec{v}_1 \cdot \vec{v}_2$ .)

Answer

$\\ \vec{c}_1 = \vec{b}_1 = (2,-1,0,2) \\ \\ \langle \vec{b}_2, \vec{c}_1 \rangle = (1,1,1,-1) \cdot (2,-1,0,2) = 1 \cdot 2 + 1 \cdot (-1) + 1 \cdot 0 + (-1) \cdot 2 = -1 \\ \\ \langle \vec{c}_1, \vec{c}_1 \rangle = (2,-1,0,2) \cdot (2,-1,0,2) = 2 \cdot 2 + (-1) \cdot (-1) + 0 \cdot 0 + 2 \cdot 2 = 9 \\ \\ \vec{c}_2 = \vec{b}_2 - \frac{\langle \vec{b}_2 , \vec{c}_1 \rangle}{\langle \vec{c}_1 , \vec{c}_1 \rangle} \vec{c}_1 = (1,1,1,-1) - \frac{-1}{9} (2,-1,0,2) = (\frac{11}{9}, \frac{8}{9}, 1, - \frac{7}{9}) \\ \\ \langle \vec{b}_3, \vec{c}_1 \rangle = (0,2,-2,1) \cdot (2,-1,0,2) = 0 \cdot 2 + 2 \cdot (-1) + (-2) \cdot 0 + 1 \cdot 2 = 0 \\ \\ \langle \vec{b}_3, \vec{c}_2 \rangle = (0,2,-2,1) \cdot (\frac{11}{9}, \frac{8}{9}, 1, - \frac{7}{9}) = 0 \cdot \frac{11}{9} + 2 \cdot \frac{8}{9} + (-2) \cdot 1 + 1 \cdot (- \frac{7}{9}) = -1 \\ \\ \langle \vec{c}_2, \vec{c}_2 \rangle = (\frac{11}{9}, \frac{8}{9}, 1, - \frac{7}{9}) \cdot (\frac{11}{9}, \frac{8}{9}, 1, - \frac{7}{9}) = \frac{11}{9} \cdot \frac{11}{9} + \frac{8}{9} \cdot \frac{8}{9} + 1 \cdot 1 + (- \frac{7}{9}) \cdot (- \frac{7}{9}) = \frac{35}{9} \\ \\ \vec{c}_3 = \vec{b}_3 - \frac{\langle \vec{b}_3 , \vec{c}_1 \rangle}{\langle \vec{c}_1 , \vec{c}_1 \rangle} \vec{c}_1 - \frac{\langle \vec{b}_3 , \vec{c}_2 \rangle}{\langle \vec{c}_2 , \vec{c}_2 \rangle} \vec{c}_2 = (0,2,-2,1) - \frac{0}{9} (2,-1,0,2) - \frac{-1}{35/9} (\frac{11}{9}, \frac{8}{9}, 1, - \frac{7}{9}) \\ \\ \vec{c}_3 = (\frac{11}{35}, \frac{78}{35}, - \frac{61}{35}, \frac{4}{5})$

The theorem above implies that $C = \begin{Bmatrix}\vec{c}_1, \: \vec{c}_2, \: \vec{c}_3 \end{Bmatrix}$ is an orthogonal basis for W.

Example 2

Continuing the previous example, find an orthonormal basis for W.

Answer

Since $C = \begin{Bmatrix}\vec{c}_1, \: \vec{c}_2, \: \vec{c}_3 \end{Bmatrix}$ is an orthogonal basis, by normalizing each of its elements the orthonormal basis will be obtained. To normalize them, the first step is to calculate the norms of the basis vectors.

$\\ \Vert \vec{c}_1 \Vert = \sqrt{2^2 + (-1)^2 + 0^2 + 2^2} = 3 \\ \\ \Vert \vec{c}_2 \Vert = \sqrt{(\frac{11}{9})^2 + (\frac{8}{9})^2 + 1^2 + (- \frac{7}{9})^2} = \frac{\sqrt{35}}{3} \\ \\ \Vert \vec{c}_3 \Vert = \sqrt{(\frac{11}{35})^2 + (\frac{78}{35})^2 + (- \frac{61}{35})^2 + (\frac{4}{5})^2} = \frac{3}{35} \sqrt{1190}$

Then, the orthonormal basis for W is $U = \begin{Bmatrix}\vec{u}_1, \: \vec{u}_2, \: \vec{u}_3 \end{Bmatrix}$ where

$\\ \vec{u}_1 = \frac{1}{\Vert \vec{c}_1 \Vert} \vec{c}_1 = (\frac{2}{3}, - \frac{1}{3}, 0, \frac{2}{3}) \\ \\ \vec{u}_2 = \frac{1}{\Vert \vec{c}_2 \Vert} \vec{c}_2 = (\frac{11}{3 \sqrt{35}}, \frac{8}{3 \sqrt{35}}, \frac{3}{\sqrt{35}}, - \frac{7}{3 \sqrt{35}}) \\ \\ \vec{u}_3 = \frac{1}{\Vert \vec{c}_3 \Vert} \vec{c}_3 = (\frac{11}{3 \sqrt{1190}}, \frac{78}{3 \sqrt{1190}}, \frac{-61}{3 \sqrt{1190}}, \frac{28}{3 \sqrt{1190}})$

Example 3

Let p(x) = a₀x² + a₁x + a₂ and q(x) = b₀x²+ b₁x + b₂ be vectors in P₂ with the inner product <p,q> = a₀b₀ + a₁b₁ + a₂b₂. Determine whether B = {x²-1, x-1} is an orthonormal set. If not, use the Gram-Schmidt Process to form an orthonormal set.

Answer

Let $\vec{b}_1 = x^2 -1$ and $\vec{b}_2 = x - 1$ . These two vectors are not orthogonal to each other since $\langle \vec{b}_1,\vec{b}_2 \rangle = 1 \cdot 0 + 0 \cdot 1 + (-1) \cdot (-1) = 1 \neq 0$ . Thus, B cannot be an orthonormal set. To form the desired orthonormal set, we will apply the Gram-Schmidt Process to produce an orthogonal set $C = \begin{Bmatrix}\vec{c}_1, \: \vec{c}_2 \end{Bmatrix}$ and then normalize every vector in C.

$\\ \vec{c}_1 = \vec{b}_1 = x^2 - 1 \\ \\ \langle \vec{b}_2, \vec{c}_1 \rangle = \langle (x - 1),(x^2 - 1) \rangle = 0 \cdot 1 + 1 \cdot 0 + (-1)(-1) = 1 \\ \\ \langle \vec{c}_1, \vec{c}_1 \rangle = \langle (x^2-1),(x^2-1) \rangle =1 \cdot 1 + 0 \cdot 0 + (-1) \cdot (-1) = 2 \\ \\ \vec{c}_2 = \vec{b}_2 - \frac{\langle \vec{b}_2 , \vec{c}_1 \rangle}{\langle \vec{c}_1 , \vec{c}_1 \rangle} \vec{c}_1 = (x-1) - \frac{1}{2} (x^1 - 1) = - \frac{1}{2} x^2 + x - \frac{1}{2}$

The theorem above implies that $C = \begin{Bmatrix}\vec{c}_1, \: \vec{c}_2 \end{Bmatrix}$ is an orthogonal basis of a subspace of P₂ and hence an orthogonal set. What is left is to normalize all the vectors in C. This proceeds as follows.

$\\ \langle \vec{c}_1, \vec{c}_1 \rangle = \langle (x^2-1),(x^2-1) \rangle =1 \cdot 1 + 0 \cdot 0 + (-1) \cdot (-1) = 2 \\ \\ \langle \vec{c}_2, \vec{c}_2 \rangle = \langle (- \frac{1}{2} x^2 + x - \frac{1}{2}),(- \frac{1}{2} x^2 + x - \frac{1}{2}) \rangle = (- \frac{1}{2}) \cdot (- \frac{1}{2}) + 1 \cdot 1 + (- \frac{1}{2}) \cdot (- \frac{1}{2}) = \frac{3}{2} \\ \\ \Vert \vec{c}_1 \Vert = \sqrt{\langle \vec{c}_1 , \vec{c}_1 \rangle} = \sqrt{2} \\ \\ \Vert \vec{c}_2 \Vert = \sqrt{\langle \vec{c}_2 , \vec{c}_2 \rangle} = \sqrt{\frac{3}{2}} = \frac{1}{2} \sqrt{6} \\ \\ \frac{\vec{c}_1}{\Vert \vec{c}_1 \Vert} = \frac{x^2 - 1}{\sqrt{2}} \\ \\ \frac{\vec{c}_2}{\Vert \vec{c}_2 \Vert} = \frac{- \frac{1}{2} x^2 + x - \frac{1}{2}}{\frac{1}{2} \sqrt{6}} = \frac{-x^2+2x-1}{\sqrt{6}}$