Chapter 6

6.2 the Gram–Schmidt process

Definition (orthonormal basis).

Let V be an inner product space. A subset of V is an orthonormal basis for V if it is an ordered basis that is orthonormal.

Theorem 6.3.

Let V be an inner product space and \(S = {v_1, v_2, . . . , v_k}\) be an orthogonal subset of V consisting of nonzero vectors. If \(y ∈ span(S)\), then

\(y=\sum_{i=1}^{k} \frac{\left\langle y, v_{i}\right\rangle}{\left\|v_{i}\right\|^{2}} v_{i}\)

证明：设$y = \sum_{i=1}^ka_iv_i$，写出$\langle y, v_i\rangle$表达式即可得出。

Corollary 1.

If, in addition to the hypotheses of Theorem 6.3, S is orthonormal and y ∈ span(S), then

\(y=\sum_{i=1}^{k} \left\langle y, v_{i}\right\rangle v_{i}\)

Corollary 2.

Let V be an inner product space, and let S be an orthogonal subset of V consisting of nonzero vectors. Then S is linearly independent.

Theorem 6.4. (the Gram–Schmidt process)

Let V be an inner product space and \(S = \{w_1, w_2, \ldots, w_n\}\) be a linearly independent subset of V. Define \(S′ = \{v_1, v_2, \ldots, v_n\}\), where \(v_1 = w_1\) and \(v_{k}=w_{k}-\sum_{j=1}^{k-1} \frac{\left\langle w_{k}, v_{j}\right\rangle}{\left\|v_{j}\right\|^{2}} v_{j} \quad \text { for } 2 \leq k \leq n.\) Then S′ is an orthogonal set of nonzero vectors such that span(S′) = span(S).

Theorem 6.5.

Let V be a nonzero finite-dimensional inner product space. Then V has an orthonormal basis \(\beta\). Furthermore, if \(\beta = \{v_1,v_2,...,v_n\}\) and x ∈ V, then \(x=\sum_{i=1}^{n}\left\langle x, v_{i}\right\rangle v_{i}\)

证明：用 Gram–Schmidt 把 orthogonal basis 构造出来，再 normalize 即可。至于$x=\sum_{i=1}^\left\langle x, v_\right\rangle v_$ 实际上就是 Thereom 6.3 Corollary 1.

Corollary.

Let V be a finite-dimensional inner product space with an orthonormal basis $ = \beta = {v_1, v_2, \ldots, v_n}$. Let T be a linear operator on V, and let A = \([T]_\beta\). Then for any i and j, \(Aij = \langle T(vj),vi\rangle.\)

Definition (Fourier coefficients).

Let $\beta $ be an orthonormal subset (possibly infinite) of an inner product space V, and let \(x ∈ V\). We define the Fourier coefficients of \(x\) relative to \(\beta\) to be the scalars \(?x, y?\), where \(y ∈ β\).

Definition (orthogonal complement).

Let S be a nonempty subset of an inner product space V. We define \(S^\perp\) to be the set of all vectors in V that are orthogonal to every vector in S; that is, \(S^{\perp}=\{x \in V:\langle x, y\rangle= 0 \text { for all } y \in S\}\). The set \(s^\perp\) is called the orthogonal complement of S.

注意

• S可以是任意集合，不一定是 subspace；

• 若$0 \in S$, \(S\cap S^\perp = \{0\}\); 否则$S\cap S^\perp = \emptyset$.

Theorem 6.6.

Let \(W\) be a finite-dimensional subspace of an inner product space \(V\), and let \(y∈V\). Then there exist unique vectors \(u∈W\) and \(z\in W^\perp\) such that \(y=u+z\). Furthermore, if${v_1,v_2,\ldots,v_k}$is an orthonormal basis for \(W\), then \(u=\sum_{i=1}^{k}\left\langle y, v_{i}\right\rangle v_{i}\)

证明：直接令$u=\sum_{i=1}^{\left\langle y, v_\right\rangle v_$，令$z = y - u$，只需证$z\in W}\perp$. 对任意 \(j\), 有 \(\begin{aligned}\left\langle z, v_{j}\right\rangle &=\left\langle\left(y-\sum_{i=1}^{k}\left\langle y, v_{i}\right\rangle v_{i}\right), v_{j}\right\rangle=\left\langle y, v_{j}\right\rangle-\sum_{i=1}^{k}\left\langle y, v_{i}\right\rangle\left\langle v_{i}, v_{j}\right\rangle \\ &=\left\langle y, v_{j}\right\rangle-\left\langle y, v_{j}\right\rangle= 0 \end{aligned}\)

下证 unique. 假设$y = u + z = u‘ + z‘, u‘ \in W, z‘ \in W^\perp$, 则$u - u‘\in W, z - z‘ \in W^\perp$, \(u - u‘ = z - z‘ \in W \cap W^\perp = \{0\}.\)

Corollary (orthogonal projection).

The vector \(u = \sum_{i=1}^{k}\left\langle y, v_{i}\right\rangle v_{i}\) is the unique vector in \(W\) that is “closest” to \(y\); that is, for any \(x ∈ W\),$ ∥y ? x∥ ≥ ∥y ? u∥$, and this inequality is an equality if and only if \(x = u\). \(u\) is called the orthogonal projection of \(y\) on \(W\).

证明：

注意到$\langle u-x, z\rangle = 0$

\(\|y-x\|^2 = \|u + z - x\|^2 = \|u-x\|^2 + \|z\|^2 \ge \|z\|^2\)

Theorem 6.7.

Suppose that \(S=\left\{v_{1}, v_{2}, \ldots, v_{k}\right\}\) is an orthonormal set in an n-dimensional inner product space \(V\). Then

(a) S can be extended to an orthonormal basis \(\{v_1, v_2, \ldots, v_k, v_{k+1}, \ldots, v_n\}\) for \(V\).

(b) If \(W = span(S)\), then \(S_1 = \{v_{k+1}, v_{k+2}, \ldots, v_n\}\) is an orthonormal basis for \(W^\perp\).

(c) If \(W\) is any subspace of \(V\), then \(dim(V) = dim(W) + dim(W^\perp)\).

证明：

(a) 先 extend，然后用 Gram–Schmidt process.

(b) 显然$S_1 \subseteq W^\perp$, 只需证$span(S_1) = W^\perp$. \(\forall x = \sum_{i = 1}^{n}a_iv_i \in W^\perp, \langle x, v_i\rangle = 0\) for $1 \le i \le k$, 所以$x = \sum_{i = k + 1}^a_iv_i \in span(S_1).$

(c) 由(b)显然。

原文：https://www.cnblogs.com/RabbitHu/p/LinearAlgebraChapter6.html