Vector Spaces to $A^k$: Why the Procedures of Lay Chapters 4–5 Actually Work

If you have row-reduced a matrix and found pivots, you can pass an exam question. But if a grader asks "why does row reduction give a basis for the column space?" you freeze. That gap between "I can do the procedure" and "I see why the procedure works" is exactly what this tutorial closes. Every central theorem from Lay Chapters 4 and 5 is proved in full, every worked example is walked line by line, and the narrative builds a single conceptual ladder from the ten axioms of a vector space all the way to $A^k = P D^k P^{-1}$.

The reader who asked for this tutorial said: "I can do the computations but don't really get why they work." So the proof is the point. When the why clicks, the procedure stops being a checklist and starts being a chain of if-and-only-ifs you can re-derive from scratch. That is what exam fluency looks like.Lay

1. What is a vector space, and how do you tell a subspace from an arbitrary subset?

A vector space is a nonempty set $V$ together with two operations — vector addition and scalar multiplication by real numbers — that obey ten axioms. The axioms look bureaucratic until you realize they are just the rules you already use when adding and scaling column vectors in $\mathbb{R}^n$:

1. $\mathbf{u} + \mathbf{v}$ is in $V$ (closure under addition).
2. $\mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u}$ (commutativity).
3. $\mathbf{u} + \mathbf{v} + \mathbf{w} = \mathbf{u} + (\mathbf{v} + \mathbf{w})$ (associativity).
4. There is a zero vector $\mathbf{0}$ in $V$ with $\mathbf{u} + \mathbf{0} = \mathbf{u}$.
5. For each $\mathbf{u}$ there is $-\mathbf{u}$ in $V$ with $\mathbf{u} + (-\mathbf{u}) = \mathbf{0}$.
6. $c\mathbf{u}$ is in $V$ (closure under scalar multiplication).
7. $c(\mathbf{u} + \mathbf{v}) = c\mathbf{u} + c\mathbf{v}$.
8. $(c + d)\mathbf{u} = c\mathbf{u} + d\mathbf{u}$.
9. $c(d\mathbf{u}) = (cd)\mathbf{u}$.
10. $1 \cdot \mathbf{u} = \mathbf{u}$.

The set $\mathbb{P}_n$ of polynomials of degree at most $n$ is a vector space. To see why (and to see the proof machinery, not just the assertion), take two polynomials $p(t) = a_0 + a_1 t + \dots + a_n t^n$ and $q(t) = b_0 + b_1 t + \dots + b_n t^n$. Their sum

$$p(t) + q(t) = (a_0 + b_0) + (a_1 + b_1)t + \dots + (a_n + b_n)t^n$$

is again a polynomial of degree at most $n$ — every coefficient is still a real number, and the degree has not exceeded $n$. Similarly, for any scalar $\alpha$, $\alpha \cdot p(t) = \alpha a_0 + \alpha a_1 t + \dots + \alpha a_n t^n$ is in $\mathbb{P}_n$. The other eight axioms transfer from the coefficient-wise rules you already trust on $\mathbb{R}^{n+1}$. So $\mathbb{P}_n$ is a vector space.

图注：The handwritten proof writes out $p(t) + q(t)$ and $\alpha \cdot p(t)$ term by term, with underlines under the new coefficients $(a_i + b_i)$ and $\alpha a_i$. The proof shows the only thing that needs checking — that sums and scalar multiples of degree-$\leq n$ polynomials stay degree-$\leq n$ — and the rest of the ten axioms are inherited.

A subspace is a subset $H$ of $V$ that is itself a vector space. You do not need to re-check all ten axioms on $H$. If $H$ is a subset of $V$ that contains the zero vector of $V$, is closed under addition, and is closed under scalar multiplication, then $H$ is automatically a subspace: it inherits the other axioms (commutativity, associativity, etc.) from $V$ because every operation on $H$ is also an operation in $V$. This is the subspace test:

• (a) The zero vector of $V$ is in $H$.
• (b) For all $\mathbf{u}, \mathbf{v} \in H$, $\mathbf{u} + \mathbf{v} \in H$.
• (c) For all $\mathbf{u} \in H$ and all scalars $c$, $c\mathbf{u} \in H$.

The "zero in $H$" check is a one-line observation rather than a separate axiom: if $H$ is closed under scalar multiplication, take $c = 0$ and $0 \cdot \mathbf{u} = \mathbf{0}$ for any $\mathbf{u} \in H$. So in practice, scalar multiple closed $\Longrightarrow \mathbf{0} \in H$, and you can drop check (a) if you already have (c).

图注：The note writes "addition, scalar multiple $\Longrightarrow \mathbf{0} \in H$" — the reminder that the zero-vector check is not an extra axiom but a free consequence of closure under scalar multiplication.

For an $m \times n$ matrix $A$, three named subspaces are attached to $A$:

• Nul $A$ = ${ \mathbf{x} \in \mathbb{R}^n : A\mathbf{x} = \mathbf{0} }$. A subset of $\mathbb{R}^n$ because $A$ has $n$ columns.
• Col $A$ = Span of the columns of $A$. A subset of $\mathbb{R}^m$ because each column has $m$ entries.
• Row $A$ = Span of the rows of $A$ = Col $A^\top$. A subset of $\mathbb{R}^n$ because each row has $n$ entries.

Proving Nul $A$ is a subspace illustrates the test in three lines. (a) $A \cdot \mathbf{0} = \mathbf{0}$, so $\mathbf{0} \in \text{Nul } A$. (b) If $\mathbf{u}, \mathbf{v} \in \text{Nul } A$, then $A(\mathbf{u} + \mathbf{v}) = A\mathbf{u} + A\mathbf{v} = \mathbf{0} + \mathbf{0} = \mathbf{0}$, so $\mathbf{u} + \mathbf{v} \in \text{Nul } A$. (c) If $c$ is any scalar, $A(c\mathbf{u}) = c(A\mathbf{u}) = c \cdot \mathbf{0} = \mathbf{0}$, so $c\mathbf{u} \in \text{Nul } A$. The proof is the same logic the subspace test encodes: the linearity $A(\mathbf{u} + \mathbf{v}) = A\mathbf{u} + A\mathbf{v}$ and $A(c\mathbf{u}) = c(A\mathbf{u})$ is exactly the closure the test asks for. This is why "every kernel of a linear map is a subspace" is a one-line observation once you have seen the proof.Lay

The subspace test is the main device in Chapter 4 — once you have it internalized, "is this subset a subspace?" reduces to "does it contain zero and is it closed?" Most of the named objects in Chapters 4 and 5 are subspaces, and most of the proof work is the same three-line check.

Lay stops at three subspaces per matrix. Strang surfaces a fourth — the left nullspace Nul $A^\top$ in $\mathbb{R}^m$ — which pairs orthogonally with Col $A$. For exam work this section, you only need to know Nul $A$, Col $A$, and Row $A$; the left nullspace reappears in orthogonal subspaces and SVD (Chapters 6 and 7), which is outside the exam scope.

图注：Two adjacent boxes. Left box = $\mathbb{R}^n$ containing the row space (dimension $r$) and the nullspace (dimension $n - r$); right box = $\mathbb{R}^m$ containing the column space (dimension $r$) and the left nullspace (dimension $m - r$). The pairs meet orthogonally at $\mathbf{0}$. Lay covers the three left-column subspaces; the left nullspace is the one piece Chapter 4 omits.StrangEssay

A subspace is just a vector space living inside a bigger vector space, and you recognize it by zero + closure. The three canonical subspaces attached to a matrix live in different ambient spaces (Nul $A$ in $\mathbb{R}^n$, Col $A$ in $\mathbb{R}^m$, Row $A$ in $\mathbb{R}^n$) because $A$ sends $\mathbf{x}$ in $\mathbb{R}^n$ to $A\mathbf{x}$ in $\mathbb{R}^m$.

2. How big is this space — independent sets, bases, and a number called dimension

A set ${\mathbf{v}_1, \dots, \mathbf{v}_p}$ in a vector space $V$ is linearly independent if the equation

$$c_1 \mathbf{v}_1 + c_2 \mathbf{v}_2 + \dots + c_p \mathbf{v}_p = \mathbf{0}$$

forces every $c_i = 0$. The set is linearly dependent if a nontrivial combination equals zero — that is, some weights are nonzero. The key theorem characterizes dependence concretely: ${\mathbf{v}_1, \dots, \mathbf{v}_p}$ (with $\mathbf{v}_1 \neq \mathbf{0}$) is linearly dependent if and only if some $\mathbf{v}_j$ (with $j > 1$) is a linear combination of $\mathbf{v}1, \dots, \mathbf{v}{j-1}$. The proof reverses the dependence relation: from $c_1 \mathbf{v}1 + \dots + c_p \mathbf{v}p = \mathbf{0}$ with some $c_j \neq 0$, isolate $\mathbf{v}j = -\frac{1}{c_j}(c_1 \mathbf{v}1 + \dots + c{j-1} \mathbf{v}{j-1} + c{j+1} \mathbf{v}{j+1} + \dots + c_p \mathbf{v}_p)$. Dependence is redundancy — one vector in the set is built from the earlier ones.Lay

A basis for a subspace $H$ is a set that is both linearly independent and spans $H$. Equivalently, it is a minimal spanning set (removing any vector shrinks the span) and a maximal independent set (adding any vector creates a dependence). These two views are linked by the Spanning Set Theorem: if one vector in a spanning set is a linear combination of the others, removing it preserves the span; iterating this deletion produces a basis. So any spanning set can be reduced to a basis by stripping dependent vectors one at a time.

图注：The standard basis ${e_1, e_2, e_3}$ for $\mathbb{R}^3$ drawn as column vectors on the left and as light-blue arrows along the three coordinate axes on the right. The reader already knows these three vectors span and are independent — the visual anchor for the abstract definition of "basis."

The crucial question is: does every basis of a finite-dimensional vector space have the same size? Yes, and here is the proof by mutual bound. Let $B_1 = {\mathbf{b}_1, \dots, \mathbf{b}_n}$ be a basis and $B_2$ be any other basis.

• Since $B_1$ spans $V$ and $B_2$ is independent, $B_2$ has no more than $n$ vectors (any independent set inside the span of $B_1$ has size $\leq n$).
• Since $B_2$ spans $V$ and $B_1$ is independent, $B_1$ has no more than $|B_2|$ vectors (any independent set inside the span of $B_2$ has size $\leq |B_2|$).

So $|B_2| \leq n$ and $n \leq |B_2|$, hence $|B_2| = n$. The dimension $\dim V$ is well-defined; it is a property of the space, not of any particular basis.Lay

图注：Three diagrams showing subspaces of $\mathbb{R}^3$: (a) a 0-dim point at the origin and a 1-dim line through the origin; (b) a 2-dim plane through the origin; (c) the whole 3-dim volume containing a 2-dim plane. The geometric picture that drives the equal-basis-size proof: in $\mathbb{R}^3$ you cannot fit more than three independent lines because the space itself is three-dimensional.

The Rank Theorem ties dimension to row reduction. For an $m \times n$ matrix $A$, $\text{rank } A = \dim \text{Col } A = \dim \text{Row } A = $ (number of pivots), and $\text{rank } A + \text{nullity } A = n$. So the column space and row space always have the same dimension (the rank), even though they live in different ambient spaces ($\mathbb{R}^m$ and $\mathbb{R}^n$). The nullity counts the dimension of the null space.

Here is the recipe for finding bases from an echelon form of $A$:

• Basis for Nul $A$: reduce to reduced echelon form; write the general solution as a linear combination of vectors, one vector per free variable. The resulting vectors are automatically independent (each free variable appears in exactly one coordinate), so they form a basis.
• Basis for Col $A$: identify the pivot positions in the echelon form; take the corresponding columns from $A$ itself (not from the echelon form). The pivot columns of the echelon form are linearly independent, but row operations change the columns; the dependence relations among columns of $A$ are preserved, so the corresponding columns of $A$ are also independent and span Col $A$.
• Basis for Row $A$: take the nonzero rows of any echelon form of $A$. Row operations preserve the row space, so the nonzero rows of the echelon form span the same row space and are independent (each adds a pivot).

Every finite-dimensional space has a single well-defined size called $\dim V$, and the proof that this number does not depend on the basis is the proof that dimension is a property of the space.

3. Coordinates depend on the basis — and that dependence is exactly what makes vector spaces look like $\mathbb{R}^n$

With a basis $B = {\mathbf{b}_1, \dots, \mathbf{b}_n}$ for $V$ fixed, every vector $\mathbf{x} \in V$ has a unique representation

$$\mathbf{x} = c_1 \mathbf{b}_1 + c_2 \mathbf{b}_2 + \dots + c_n \mathbf{b}_n.$$

This is the Unique Representation Theorem. Existence comes from $B$ spanning $V$; uniqueness comes from $B$ being linearly independent: if $\mathbf{x} = c_1 \mathbf{b}_1 + \dots + c_n \mathbf{b}_n = d_1 \mathbf{b}_1 + \dots + d_n \mathbf{b}_n$, then $(c_1 - d_1)\mathbf{b}_1 + \dots + (c_n - d_n)\mathbf{b}_n = \mathbf{0}$, and linear independence forces every $c_i - d_i = 0$. The column vector $[\mathbf{x}]_B = [c_1, \dots, c_n]^\top$ is the coordinate vector of $\mathbf{x}$ relative to $B$.Lay

The matrix $P_B = [\mathbf{b}_1 ; \mathbf{b}_2 ; \cdots ; \mathbf{b}_n]$ satisfies $\mathbf{x} = P_B [\mathbf{x}]_B$ — left-multiplying by $P_B$ converts B-coordinates into standard coordinates. Since the columns of $P_B$ form a basis of $\mathbb{R}^n$, $P_B$ is invertible by the Invertible Matrix Theorem, and $P_B^{-1}$ is the inverse direction (from standard coordinates back to B-coordinates).

图注:The diagram shows a 2D coordinate system where the basis $B = {\mathbf{b}_1, \mathbf{b}_2}$ cuts the plane into a parallelogram grid. The point $\mathbf{x}$ lies 3 units along $\mathbf{b}_1$ and 2 units along $\mathbf{b}_2$, so $[\mathbf{x}]_B = (3, 2)$. The visual makes "coordinate = count along the basis" concrete before the formal change-of-basis machinery arrives.

The coordinate mapping $\mathbf{x} \mapsto [\mathbf{x}]_B$ is a linear, one-to-one, onto map from $V$ to $\mathbb{R}^n$ — that is, an isomorphism. To prove this:

• Linearity: $[\mathbf{u} + \mathbf{v}]_B = [\mathbf{u}]_B + [\mathbf{v}]_B$ (the B-coordinates of a sum are the sum of the B-coordinates) and $[c\mathbf{u}]_B = c[\mathbf{u}]_B$ (the B-coordinates of a scalar multiple are the scalar multiple of the B-coordinates). The proof is the same uniqueness argument: the B-coordinates of $\mathbf{u} + \mathbf{v}$ are the coefficients that reproduce $\mathbf{u} + \mathbf{v}$ as a combination of the $\mathbf{b}_i$, and the coefficients $c_i + d_i$ where $c_i$ and $d_i$ are the coordinates of $\mathbf{u}$ and $\mathbf{v}$ do the job.
• Injectivity: if $[\mathbf{x}]_B = \mathbf{0}$, then $\mathbf{x} = 0 \cdot \mathbf{b}_1 + \dots + 0 \cdot \mathbf{b}_n = \mathbf{0}$. The kernel of the coordinate mapping is ${\mathbf{0}}$, so the map is one-to-one.
• Surjectivity: every $\mathbf{c} = [c_1, \dots, c_n]^\top \in \mathbb{R}^n$ is the coordinate vector of $c_1 \mathbf{b}_1 + \dots + c_n \mathbf{b}_n \in V$.

A one-to-one linear transformation from $V$ onto $W$ is an isomorphism, and any vector-space calculation in $V$ is accurately reproduced in $W$ and vice versa. The punchline: any $n$-dimensional vector space looks like $\mathbb{R}^n$ once you fix a basis. $\mathbb{P}_3$ is not $\mathbb{R}^4$, but it is isomorphic to $\mathbb{R}^4$ via the map $a_0 + a_1 t + a_2 t^2 + a_3 t^3 \mapsto [a_0, a_1, a_2, a_3]^\top$.

For ordered bases $\alpha$ of $V$ and $\beta$ of $W$, the matrix of a linear transformation $L: V \to W$ is $L_\alpha^\beta = [[L(\mathbf{v}1)]\beta, \dots, [L(\mathbf{v}n)]\beta]$ — the $\beta$-coordinates of $L$ applied to each $\alpha$-basis vector, written as columns. The derivative $L: \mathbb{P}_3 \to \mathbb{P}_2$ with $L(p) = p'$ on the standard monomial bases $\alpha = {1, t, t^2, t^3}$ and $\beta = {1, t, t^2}$ has matrix

$$L_\alpha^\beta = \begin{bmatrix} 0 & 1 & 0 & 0 \ 0 & 0 & 2 & 0 \ 0 & 0 & 0 & 3 \end{bmatrix}.$$

The first column is $[L(1)]\beta = [0]\beta = [0, 0, 0]^\top$; the second is $[L(t)]\beta = [1]\beta = [1, 0, 0]^\top$; the third is $[L(t^2)]\beta = [2t]\beta = [0, 2, 0]^\top$; the fourth is $[L(t^3)]\beta = [3t^2]\beta = [0, 0, 3]^\top$.

Changing the basis changes the coordinates, not the vector; the matrix $P_B$ is the bridge, and the coordinate mapping is the formal reason an abstract $n$-dimensional space is really just $\mathbb{R}^n$ in disguise.

4. What happens when you change the basis — the change-of-coordinates matrix

Two bases $B = {\mathbf{b}_1, \dots, \mathbf{b}_n}$ and $C = {\mathbf{c}_1, \dots, \mathbf{c}n}$ for the same space $V$ give two coordinate languages. The change-of-coordinates matrix from $B$ to $C$ is $P{C\leftarrow B} = P_C^{-1} P_B$, and it satisfies $[\mathbf{x}]C = P{C\leftarrow B} [\mathbf{x}]_B$ for every $\mathbf{x} \in V$.Lay

The derivation is one line. The same vector $\mathbf{x}$ has two decompositions:

$$\mathbf{x} = P_B [\mathbf{x}]_B = P_C [\mathbf{x}]_C.$$

Multiplying the first equality by $P_C^{-1}$ on the left:

$$[\mathbf{x}]_C = P_C^{-1} P_B [\mathbf{x}]B = P{C\leftarrow B} [\mathbf{x}]_B.$$

The columns of $P_{C\leftarrow B}$ are the C-coordinate vectors of the B-basis vectors:

$$P_{C\leftarrow B} = [[\mathbf{b}_1]_C, [\mathbf{b}_2]_C, \dots, [\mathbf{b}_n]_C].$$

To compute $P_{C\leftarrow B}$ in practice, augment $[\mathbf{c}_1 ; \mathbf{c}_2 | \mathbf{b}_1 ; \mathbf{b}2]$ and row-reduce to $[\mathbf{I} | P{C\leftarrow B}]$. The augmented system "express each $\mathbf{b}_i$ as a combination of the $\mathbf{c}_j$" is exactly the same row reduction you already do for any linear system; the change-of-coordinates matrix is the block on the right after row reduction completes.

图注:The image shows the algebraic equation

$$\begin{bmatrix} 1/3 & 1/3 \ -1/3 & 2/3 \end{bmatrix} \begin{bmatrix} 3 \ 2 \end{bmatrix} = \begin{bmatrix} 5/3 \ 1/3 \end{bmatrix}$$

with curly-brace labels: the 2×2 matrix is "Inverse change of basis", the column $[3;2]$ is "Vector in our language", and the result $[5/3; 1/3]$ is "Vector in her language". The worked numerical example of the formula $[\mathbf{x}]_C = P_C^{-1} P_B [\mathbf{x}]_B$.Sanderson

The canonical Lay figure for this concept is the "two coordinate systems for the same vector space" diagram: the same vector $\mathbf{x}$ mapped to $[\mathbf{x}]_B$ in one copy of $\mathbb{R}^n$ and to $[\mathbf{x}]C$ in another copy, with $P{C\leftarrow B}$ as the arrow between coordinate languages.

图注:Two side-by-side 2D grids. In the left panel, basis $B = {\mathbf{b}_1, \mathbf{b}_2}$ defines one set of grid lines; in the right panel, basis $C = {\mathbf{c}_1, \mathbf{c}_2}$ defines another. The same vector $\mathbf{x}$ appears in both grids but has different coordinates — $[\mathbf{x}]_B$ on the left, $[\mathbf{x}]_C$ on the right. The figure makes "change of basis = switch coordinate language" visual.Lay

The full commutative picture of the change-of-basis map is the diagram below: $V$ sits on top, two copies of $\mathbb{R}^n$ sit on the bottom (one for $B$-coordinates, one for $C$-coordinates), the coordinate mappings descend from $V$ to each, and $P_{C\leftarrow B}$ acts horizontally between the two $\mathbb{R}^n$ copies.

图注:Three parallelograms in a triangular configuration: top = $V$ (containing $\mathbf{x}$); bottom-left = $\mathbb{R}^n$ (containing $[\mathbf{x}]_C$); bottom-right = $\mathbb{R}^n$ (containing $[\mathbf{x}]_B$). Arrows descend labelled $[;]_C$ and $[;]B$; the horizontal arrow between the two $\mathbb{R}^n$ copies is "multiplication by $P{C\leftarrow B}$". The diagram captures: going from $V$ to $C$-coordinates directly via $[;]_C$ is the same as going via $[;]B$ and then multiplying by $P{C\leftarrow B}$.

The coordinates of a vector depend on the basis, but the change between two coordinate systems is itself a single matrix multiplication. There is nothing hidden in "change of basis" beyond the change-of-coordinates formula.

5. Which vectors does $A$ leave pointing where they already point — and why those matter most

A nonzero vector $\mathbf{x}$ is an eigenvector of an $n \times n$ matrix $A$ if $A\mathbf{x} = \lambda \mathbf{x}$ for some scalar $\lambda$ (the eigenvalue). Both halves of the definition matter: $\mathbf{x}$ must be nonzero (the zero vector satisfies $A\mathbf{0} = \lambda \mathbf{0}$ trivially, so $\mathbf{0}$ is never an eigenvector), and $\lambda$ is a scalar, not a vector. The equation rewrites as $(A - \lambda I)\mathbf{x} = \mathbf{0}$, so $\lambda$ is an eigenvalue of $A$ if and only if $A - \lambda I$ is singular, if and only if $\det(A - \lambda I) = 0$. Eigenvalues are the roots of a polynomial, not arbitrary numbers.Lay

The set of all eigenvectors for a given $\lambda$, together with $\mathbf{0}$, is the eigenspace $E_\lambda = \text{Nul}(A - \lambda I)$. It is a subspace because it is a null space, and every null space is a subspace — the payoff of Chapter 4 applied here. Concretely: $A\mathbf{0} = \mathbf{0}$, so $\mathbf{0} \in E_\lambda$; if $\mathbf{u}, \mathbf{v} \in E_\lambda$, then $A(\mathbf{u} + \mathbf{v}) = \lambda \mathbf{u} + \lambda \mathbf{v} = \lambda(\mathbf{u} + \mathbf{v})$ so $\mathbf{u} + \mathbf{v} \in E_\lambda$; and $A(c\mathbf{u}) = c\lambda\mathbf{u} = \lambda(c\mathbf{u})$ so $c\mathbf{u} \in E_\lambda$.

Geometrically, an eigenvector is a vector $A$ does not rotate. The eigenvectors of a matrix are the directions $A$ acts on by pure stretching (or by stretching and direction-reversing, if $\lambda$ is negative). The eigenspace is the line (in 2D), plane (in 3D), or higher-dimensional subspace in which $A$ acts as multiplication by $\lambda$.

图注:A 2D Cartesian grid with two vectors drawn from the origin. The red vector $\mathbf{u}$ and its image $A\mathbf{u}$ lie on the same line through the origin (eigenvector, eigenvalue direction-reversing). The blue vector $\mathbf{v}$ and its image $A\mathbf{v}$ also lie on the same line, scaled by a positive factor (eigenvector, eigenvalue positive). The figure makes "eigenvector = direction $A$ preserves" concrete before the formal $A\mathbf{x} = \lambda\mathbf{x}$ arrives.Lay

Two structural theorems sit at the heart of Chapter 5.

Theorem: eigenvalues of a triangular matrix. The eigenvalues of a triangular matrix are its diagonal entries. The proof is a clean inspection. If $A$ is upper triangular, then $A - \lambda I$ is also upper triangular with diagonal entries $a_{11} - \lambda, a_{22} - \lambda, \dots, a_{nn} - \lambda$ and zeros below the diagonal. The equation $(A - \lambda I)\mathbf{x} = \mathbf{0}$ has a free variable if and only if one of the diagonal entries is zero (because each pivot position depends only on its own diagonal entry when all entries below are zero). The diagonal entry is zero if and only if $\lambda$ equals one of the diagonal entries of $A$. Three lines of algebra; no determinant gymnastics.

Theorem: eigenvectors for distinct eigenvalues are linearly independent. Let $\mathbf{v}_1, \dots, \mathbf{v}_r$ be eigenvectors with distinct eigenvalues $\lambda_1, \dots, \lambda_r$. The set is linearly independent. Proof by induction: start with the empty set (trivially independent). Suppose $\mathbf{v}_1, \dots, \mathbf{v}_p$ are linearly independent (with $p < r$) and suppose

$$c_1 \mathbf{v}_1 + \dots + c_p \mathbf{v}p + \mathbf{v}{p+1} = \mathbf{0}$$

(taking the coefficient of $\mathbf{v}_{p+1}$ to be 1 by the dependence characterization, since $\mathbf{v}_1 \neq \mathbf{0}$). Multiply by $A$ on the left and use $A\mathbf{v}_i = \lambda_i \mathbf{v}_i$:

$$c_1 \lambda_1 \mathbf{v}1 + \dots + c_p \lambda_p \mathbf{v}p + \lambda{p+1} \mathbf{v}{p+1} = \mathbf{0}.$$

Multiply the original equation by $\lambda_{p+1}$ and subtract:

$$c_1(\lambda_1 - \lambda_{p+1}) \mathbf{v}1 + \dots + c_p(\lambda_p - \lambda{p+1}) \mathbf{v}_p = \mathbf{0}.$$

Since $\mathbf{v}1, \dots, \mathbf{v}p$ are linearly independent, every $c_i(\lambda_i - \lambda{p+1}) = 0$. The eigenvalues are distinct, so $\lambda_i - \lambda{p+1} \neq 0$, giving $c_1 = \dots = c_p = 0$. But then the original equation says $\mathbf{v}{p+1} = \mathbf{0}$, contradicting that $\mathbf{v}{p+1}$ is an eigenvector. So the set is independent.

The Lay worked example $A = \begin{bmatrix} 1 & 6 \ 5 & 2 \end{bmatrix}$ illustrates both. The eigenvalues are $7$ and $-4$; the eigenspaces are Span${[1, 1]^\top}$ (the line of slope 1 through the origin) and Span${[-6, 5]^\top}$ (the line of slope $-5/6$ through the origin). On the first line, $A$ stretches vectors by $7$; on the second, $A$ flips and stretches by $4$.

图注:A 2D coordinate system with two perpendicular lines through the origin. The red line with positive slope is the eigenspace for $\lambda = 7$; a sample vector near $(1,1)$ maps to $(7,7)$. The blue line with negative slope is the eigenspace for $\lambda = -4$; a sample vector near $(1.5, -1)$ maps to the opposite direction through the origin, near $(-6, 4)$. The figure shows the geometric picture of the worked example.Lay

图注:The color version of the same picture from the lecture deck. The eigenspace for $\lambda = 7$ is drawn in blue with positive slope; the eigenspace for $\lambda = -4$ is drawn in blue with negative slope. Dashed arrows show "Multiplication by 7" (outward along the $\lambda = 7$ line) and "Multiplication by −4" (through the origin, opposite direction along the $\lambda = -4$ line). The point $(6, -5)$ is marked on the $\lambda = -4$ line.

For projection and reflection, the cleanest worked examples come from Strang's Figure 6.2.

图注:Two side-by-side 2D diagrams. Left: a projection $P$ onto a line; vectors on the line stay put ($\lambda = 1$), vectors perpendicular to the line collapse to $\mathbf{0}$ ($\lambda = 0$). Right: a reflection $E$ across a line; vectors on the line stay put ($\lambda = 1$), vectors perpendicular to the line flip sign ($\lambda = -1$). Caption: "Projections P have eigenvalues 1 and 0. Exchanges E have λ = 1 and −1. A typical x changes direction, but an eigenvector Ax = λx stays on the line through x."Strang

An eigenvector is not a magical object. It is any nonzero vector in the null space of $A - \lambda I$, and the set of all such vectors (the eigenspace) is itself a subspace. The two structural theorems (triangular-matrix eigenvalues, distinct-eigenvalue independence) reduce two common checks to a few lines of algebra.

6. From one eigenvalue to all of them — characteristic equation, similarity, and the door to $A^k = P D^k P^{-1}$

The characteristic equation of $A$ is $\det(A - \lambda I) = 0$. Three if-and-only-ifs collapse into one line: $\lambda$ is an eigenvalue of $A$ if and only if $(A - \lambda I)\mathbf{x} = \mathbf{0}$ has a nonzero solution, if and only if $A - \lambda I$ is singular, if and only if $\det(A - \lambda I) = 0$. So eigenvalues are exactly the roots of the characteristic polynomial, and algebraic multiplicity is the multiplicity of $\lambda$ as a root.Lay

For a $2 \times 2$ matrix, the characteristic polynomial is $\lambda^2 - (\text{tr } A)\lambda + \det A$. This is the form most exam problems use, and you should be able to write it down in seconds: trace on the linear coefficient (negated), determinant on the constant term.

Similar matrices share eigenvalues. Matrices $A$ and $B$ are similar if $A = P B P^{-1}$ for some invertible $P$ (equivalently $B = P^{-1} A P$). The proof that similar matrices have the same characteristic polynomial is two algebraic steps:

$$\det(B - \lambda I) = \det(P^{-1} A P - \lambda I) = \det(P^{-1}(A - \lambda I) P) = \det(P^{-1}) \det(A - \lambda I) \det(P) = \det(A - \lambda I),$$

using $\det(P^{-1}) \det(P) = 1$ and the multiplicative property $\det(XY) = \det(X) \det(Y)$. So similar matrices share eigenvalues with the same multiplicities. The change of basis $P$ does not change the underlying linear transformation — it only re-describes the same operator in a different coordinate system.

Warning: row reduction is not similarity. If $A$ is row equivalent to $B$, then $B = E A$ for some invertible $E$, but $E$ is not conjugated by $A^{-1}$, so row operations generally change eigenvalues. Reducing $A$ to echelon form is fine for finding Nul $A$ and Row $A$, but to find eigenvalues you form $A - \lambda I$ and compute its determinant, not reduce it.

The Invertible Matrix Theorem (eigenvalue form) adds: $A$ is invertible if and only if $0$ is not an eigenvalue of $A$. The proof is one iff-chain: $0$ is an eigenvalue $\Longleftrightarrow$ there is nonzero $\mathbf{x}$ with $A\mathbf{x} = 0 \cdot \mathbf{x} = \mathbf{0}$ $\Longleftrightarrow$ $A$ is not invertible.

Two matrices can share eigenvalues without being similar (the Jordan-block distinction lives later; for Chapters 4–5 it suffices to know that the characteristic polynomial does not pin down similarity). The eigenvalues give you the algebraic content; similar matrices preserve that content.

7. When the eigenvectors form a basis — diagonalization and the killer application $A^k = P D^k P^{-1}$

An $n \times n$ matrix $A$ is diagonalizable if $A = P D P^{-1}$ for some invertible matrix $P$ and some diagonal matrix $D$. The Diagonalization Theorem says this is possible if and only if $A$ has $n$ linearly independent eigenvectors: $A$ is diagonalizable if and only if there exists a basis of $\mathbb{R}^n$ consisting of eigenvectors of $A$, equivalently if and only if $\sum_k \dim E_{\lambda_k} = n$.Lay

The proof of the theorem is a one-direction argument that ties the matrix $P$ to its columns. Suppose $A = P D P^{-1}$. Multiply on the right by $P$:

$$A P = P D P^{-1} P = P D.$$

Let $\mathbf{v}_1, \dots, \mathbf{v}_n$ be the columns of $P$ and $\lambda_1, \dots, \lambda_n$ the diagonal entries of $D$. The $j$-th column of $A P$ is $A \mathbf{v}_j$; the $j$-th column of $P D$ is $\lambda_j \mathbf{v}_j$. So $A \mathbf{v}_j = \lambda_j \mathbf{v}_j$ for each $j$. Each column of $P$ is an eigenvector of $A$, and the columns are linearly independent because $P$ is invertible. Conversely, if $A$ has $n$ linearly independent eigenvectors $\mathbf{v}_1, \dots, \mathbf{v}_n$, then $P = [\mathbf{v}_1 ; \cdots ; \mathbf{v}_n]$ is invertible, $D = \text{diag}(\lambda_1, \dots, \lambda_n)$ is diagonal, and the column-by-column equation $A\mathbf{v}_j = \lambda_j \mathbf{v}_j$ stacks into $A P = P D$, so $A = P D P^{-1}$. This is the iff.Sanderson

The four-step recipe for diagonalization:

1. Find the eigenvalues of $A$ from the characteristic polynomial.
2. Find a basis for each eigenspace by row reducing $A - \lambda_k I$ and writing the general solution as a span.
3. Construct $P = [\mathbf{v}_1 ; \mathbf{v}_2 ; \cdots ; \mathbf{v}_n]$ from the eigenvectors (the order is arbitrary but should match step 4).
4. Construct $D = \text{diag}(\lambda_1, \dots, \lambda_n)$ with eigenvalues matching the eigenvector order. To verify without computing $P^{-1}$, just confirm $A P = P D$ column-by-column.

Critical step: if you cannot find $n$ linearly independent eigenvectors in step 2 (i.e., $\sum_k \dim E_{\lambda_k} < n$), $A$ is not diagonalizable, and no further algebra will rescue it. A sufficient condition: $n$ distinct eigenvalues (each contributes at least one independent eigenvector).

Geometrically, on each eigenspace, $A$ acts as multiplication by $\lambda$. The Diagonalization Theorem says: if those eigenspaces span $\mathbb{R}^n$, then $A$ is, in the right coordinate system, a list of stretches — nothing else.

图注:A grayscale 2D plane region with several blue vectors emanating from the center. Curved grey arrows show the action of $A$: each vector on the eigenspace is stretched to a longer vector, all lying in the same plane. The diagram illustrates that $A$ acts as a dilation by $\lambda = 2$ on this eigenspace — every vector is doubled in length while staying in the same plane. The geometric motivation for the Diagonalization Theorem.Lay

图注:Two side-by-side 3D coordinate systems connected by a horizontal arrow labelled "Multiplication by $A$" (green-to-red gradient). Left: the eigenspace for $\lambda = 2$ as a translucent green plane with five blue vectors fanning out from the center. Right: the same plane with the same five vectors, all visibly longer but still pointing in the same directions and lying in the same plane. The 3D version of the eigenspace-dilation idea.

The visual intuition for why the Diagonalization Theorem works comes from changing the coordinate system to eigenvectors.

图注:Two stacked planes separated by a grey band. Upper plane: a standard blue square grid with three arrows from the origin (purple, red, green). The matrix $\begin{bmatrix} 3 & 1 \ 0 & 2 \end{bmatrix}$ sits in the middle, with a label "Use eigenvectors [1;0] and [-1;1] as basis". Lower plane: the same three arrows under the matrix, but in the new (skewed) coordinate system. The image illustrates the idea that switching the basis to eigenvectors turns the matrix into a simpler form.Sanderson

The killer application. Once $A = P D P^{-1}$ with $D$ diagonal, computing $A^k$ is one line:

$$A^k = P D^k P^{-1},$$

where $D^k = \text{diag}(\lambda_1^k, \dots, \lambda_n^k)$. So taking any power of $A$ reduces to three matrix multiplications (row, diagonal, row) instead of multiplying $A$ by itself $k$ times. For $A^{100}$ this is the difference between a hopeless direct computation and a trivial one.

This is exactly the structure behind the recursive-sequence motivating example from Chapter 5: sequences defined by $\mathbf{v}_{n+1} = A \mathbf{v}_n$ can be computed in closed form because $\mathbf{v}_n = A^n \mathbf{v}_0 = P D^n P^{-1} \mathbf{v}_0$, and $D^n$ is just $\lambda_i^n$ in each coordinate. The Fibonacci-like recurrences fall out of the eigenbasis in one multiplication per step.

The full chain, end to end:

$$\text{axioms} \to \text{subspaces} \to \text{basis and dimension} \to \text{coordinate mapping} \to \text{change of basis} \to \text{eigenvalues} \to \text{characteristic equation} \to \text{diagonalization} \to A^k = P D^k P^{-1}.$$

Each link is one or two theorems. Once you can re-derive every step, the procedures stop being a checklist and start being the consequence of a single chain of if-and-only-ifs. That is what exam fluency looks like.

References

The tutorial draws on the user's uploaded lecture slides (Chapters 4 and 5, Lay 5th edition), the Lay textbook itself, and the following supplementary resources for visual intuition and parallel formal treatment:

1. Lay, Lay, McDonald, Linear Algebra and Its Applications, 5th Edition (Pearson, 2016) — Pearson product page — Spine: ten axioms, subspace test, Nul/Col/Row spaces, coordinate mapping, change of basis, eigenvalue/eigenspace, characteristic equation, diagonalization, $A^k = P D^k P^{-1}$.
2. Strang, Lecture Notes for Linear Algebra (MIT 18.06, 2021 ed.) — math.mit.edu/~gs/LectureNotes/ — Four-subspaces framing in Section 3.5; Figure 6.2 projection/reflection canonical eigenvector examples.
3. Strang, Introduction to Linear Algebra, 5th ed., Section 6.1 — ila6_6_1.pdf — Figure 6.2 (projection $P$ and reflection $E$) and the "vector stays on its span" framing.
4. Sanderson (3Blue1Brown), Essence of Linear Algebra, Chapters 13 and 14 — Change of basis and Eigenvectors and eigenvalues — Visual intuition for change of basis (Jennifer's grid) and eigenbasis (use eigenvectors as basis).
5. Strang, "The Four Fundamental Subspaces: 4 Lines" essay (MIT 18.06 ver3, 2008) — newpaper_ver3.pdf — Canonical Figure 1 with dimensions $r, n-r, r, m-r$ and orthogonal-complement pairing.
6. Axler, Linear Algebra Done Right, 4th ed., Theorem 5.19 (Springer, 2024) — linear.axler.net/LADR4e.pdf — Determinant-free constructive proof that every operator on a finite-dimensional complex vector space has an eigenvalue.
7. Beezer, A First Course in Linear Algebra (FCLA Version 3.50, 2017) — linear.pugetsound.edu/html/ — Formal theorem-proof statements for the coordinate-mapping isomorphism and distinct-eigenvalue eigenvector independence.
8. Dawkins, Paul's Online Math Notes, Differential Equations Section 5.3 — tutorial.math.lamar.edu/Classes/DE/LA_Eigen.aspx — Worked-example bank: four fully worked eigenvalue/eigenspace examples covering distinct real, repeated real, complex, and 3×3 mixed multiplicities.