Chapter 6 Study Guide

calendar_today August 9, 2026 article 607 words access_time 5 min replay Modified August 9, 2026

Table of Contents

Chapter 6 — Orthogonality and Least Squares

Covers §6.1–6.3. Full problems and recorded answers in 2160 Compendium.

Weighting

33 questions, 6 true/false (18%) — the smallest chapter, and the last material before the final. It was also the most recently studied (all three sections worked on 07/13), so it is the freshest but least reviewed.

§6.1 Inner Product, Length, and Orthogonality

Definition 1 (Inner product, norm, distance).

$$\mathbf{u}\cdot\mathbf{v}=\mathbf{u}^{T}\mathbf{v}=u_1v_1+u_2v_2+\cdots+u_nv_n$$$$\|\mathbf{v}\|=\sqrt{\mathbf{v}\cdot\mathbf{v}}=\sqrt{v_1^{2}+\cdots+v_n^{2}}, \qquad \|\mathbf{v}\|^{2}=\mathbf{v}\cdot\mathbf{v}$$$$\operatorname{dist}(\mathbf{u},\mathbf{v})=\|\mathbf{u}-\mathbf{v}\|$$

The dot product returns a scalar. It is commutative and distributive, and $(c\mathbf{u})\cdot\mathbf{v}=c(\mathbf{u}\cdot\mathbf{v})$.

Orthogonality

$$\mathbf{u}\perp\mathbf{v} \iff \mathbf{u}\cdot\mathbf{v}=0$$

That is the entire test — compute the dot product and compare to zero. The zero vector is orthogonal to everything.

Pythagorean Theorem: $\|\mathbf{u}+\mathbf{v}\|^{2}=\|\mathbf{u}\|^{2}+\|\mathbf{v}\|^{2}$ holds exactly when $\mathbf{u}\perp\mathbf{v}$.

Unit vectors — normalizing

$$\mathbf{u}=\frac{\mathbf{v}}{\|\mathbf{v}\|}$$

Divide by the length; the result has norm 1 and points the same direction. Expect an irrational $\|\mathbf{v}\|$ — leave the radical in place rather than decimalizing.

Tested as — dot products and the ratio $\frac{\mathbf{u}\cdot\mathbf{v}}{\mathbf{v}\cdot\mathbf{v}}$ 6.1.1, 6.1.2, 6.1.5 · norm 6.1.7 · unit vector 6.1.9, 6.1.10 · distance 6.1.14 · orthogonality check 6.1.15–6.1.18

Watch for $\frac{\mathbf{u}\cdot\mathbf{v}}{\mathbf{v}\cdot\mathbf{v}}$

Three §6.1 problems ask for this quotient specifically. It is not idle arithmetic — it is exactly the coefficient in the orthogonal projection formula of §6.2. Recognizing it early makes the next section trivial.

§6.2 Orthogonal Sets

Definition 2 (Orthogonal and orthonormal sets).

A set is orthogonal if every distinct pair is orthogonal: $\mathbf{u}_i\cdot\mathbf{u}_j=0$ for $i\neq j$. It is orthonormal if additionally every vector is a unit vector.

For three vectors that means checking three pairs — $(1,2)$, $(1,3)$, $(2,3)$. Missing one is the standard error.

Why orthogonal bases are worth having

If $\{\mathbf{u}_1,\dots,\mathbf{u}_p\}$ is an orthogonal basis for $W$, then any $\mathbf{y}\in W$ expands as

$$\mathbf{y}=c_1\mathbf{u}_1+\cdots+c_p\mathbf{u}_p, \qquad c_j=\frac{\mathbf{y}\cdot\mathbf{u}_j}{\mathbf{u}_j\cdot\mathbf{u}_j}$$

No system to solve — each coefficient is an independent quotient. Compare §1.3, where finding coefficients meant row reducing an augmented matrix. This is the payoff of orthogonality.

Orthogonal projection onto a line

The projection of $\mathbf{y}$ onto the line through $\mathbf{u}$ and the origin:

$$\hat{\mathbf{y}}=\operatorname{proj}_{\mathbf{u}}\mathbf{y}=\frac{\mathbf{y}\cdot\mathbf{u}}{\mathbf{u}\cdot\mathbf{u}}\,\mathbf{u}$$

The orthogonal decomposition is then

$$\mathbf{y}=\hat{\mathbf{y}}+\mathbf{z}, \qquad \mathbf{z}=\mathbf{y}-\hat{\mathbf{y}} \perp \mathbf{u}$$

$\hat{\mathbf{y}}$ lies in $\text{Span}\{\mathbf{u}\}$ and $\mathbf{z}$ is orthogonal to it. Always sanity-check that $\mathbf{z}\cdot\mathbf{u}=0$.

Orthogonal ⇒ independent

An orthogonal set of nonzero vectors is automatically linearly independent. So an orthogonal set of $n$ nonzero vectors in $\mathbb{R}^n$ is immediately a basis — no row reduction required.

Tested as — is the set orthogonal 6.2.2, 6.2.3, 6.2.5, 6.2.6 · orthogonal basis and coordinates 6.2.7, 6.2.10 · projection onto a line 6.2.11 · orthogonal decomposition 6.2.13, 6.2.14 · orthonormal / normalize 6.2.17, 6.2.19, 6.2.21

§6.3 Orthogonal Projections

Theorem 3 (The Orthogonal Decomposition Theorem).

Let $W$ be a subspace of $\mathbb{R}^n$. Every $\mathbf{y}\in\mathbb{R}^n$ can be written uniquely as

$$\mathbf{y}=\hat{\mathbf{y}}+\mathbf{z}, \qquad \hat{\mathbf{y}}\in W,\ \ \mathbf{z}\in W^{\perp}$$

If $\{\mathbf{u}_1,\dots,\mathbf{u}_p\}$ is an orthogonal basis for $W$:

$$\hat{\mathbf{y}}=\operatorname{proj}_{W}\mathbf{y}=\frac{\mathbf{y}\cdot\mathbf{u}_1}{\mathbf{u}_1\cdot\mathbf{u}_1}\mathbf{u}_1+\cdots+\frac{\mathbf{y}\cdot\mathbf{u}_p}{\mathbf{u}_p\cdot\mathbf{u}_p}\mathbf{u}_p$$

The formula requires an orthogonal basis — that is why every problem says “verify the set is orthogonal, then project.”

Best Approximation Theorem

$\hat{\mathbf{y}}$ is the closest point in $W$ to $\mathbf{y}$: for every other $\mathbf{v}\in W$,

$$\|\mathbf{y}-\hat{\mathbf{y}}\| < \|\mathbf{y}-\mathbf{v}\|$$

The distance from $\mathbf{y}$ to $W$ is $\|\mathbf{z}\|=\|\mathbf{y}-\hat{\mathbf{y}}\|$. This is the idea least-squares is built on.

Two useful special cases

If $\mathbf{y}$ is already in $W$, then $\operatorname{proj}_W\mathbf{y}=\mathbf{y}$ and $\mathbf{z}=\mathbf{0}$
If $\mathbf{y}\perp W$, then $\operatorname{proj}_W\mathbf{y}=\mathbf{0}$

Several §6.3 true/false items are exactly these degenerate cases.

Tested as — verify orthogonal then project 6.3.4, 6.3.5 · decompose $\mathbf{y}$ into $W$ and $W^{\perp}$ parts 6.3.8, 6.3.9 · true/false 6.3.21–6.3.28

The through-line

Chapter 6 is Chapter 1 with a better basis. Finding coordinates went from solve an augmented system (§1.3) to compute independent quotients (§6.2), purely because the basis is orthogonal. Everything in §6.3 follows from that one simplification.

Graph

mindmap ↗