Ch.07

Eigenvalues and Eigenvectors: Principal Axes Unchanged by Transformation

Math diagram by chapter

Select a chapter to see its diagram below. View the flow of intermediate math at a glance.

Eigenvalues and Eigenvectors: Directions That Stay Put

Eigenvectors stay on one fixed line through the origin: the map does not bend them onto a new line—it only stretches or shrinks (and may flip) along that line . The eigenvalue is the stretch factor there.

Matrix

Eigenvalue

A

Eigenvector

Brown badge — the rule of the transform. Mint badge — the stretch on that line . Coral underlines — the same direction on both sides of “=”. Curved arrows — connect each label to the matching piece in the line.

Picture a washer tumbling laundry: water swirls, clothes tangle, and paths bend . When a matrix transforms space, most vectors leave the line they started on . But the washer’s spin axis stays on its line—its direction does not “turn a corner.” In math, some nonzero vectors still lie on the same line after the map; only their length stretches or shrinks (like a rubber band). Those directions are eigenvectors, and the stretch factors are eigenvalues . This chapter finds that steady line inside messy-looking data and uses it to split ideas into simpler pieces .

Eigenvalues and Eigenvectors: Directions That Stay Put

A\mathbf{v}=\lambda\mathbf{v}

2. Eigenvalue — how length changes on that line For an eigenvector, the eigenvalue is the factor by which length changes along that line. If it is greater than 1, it stretches; between 0 and 1, it shrinks. If it is negative, you still stay on the line but point the other way.

3. Characteristic equation — the usual way to find eigenvalues Build the characteristic equation and solve for candidates. Intuitively: pick a trial value until the map squishes enough that “volume” collapses to zero—that trial is a candidate, and the directions you recover are eigenvector candidates.

4. Diagonalization — turn a tangled map into simple scales If you have enough independent eigen-directions for a new basis, you can rewrite the map as “scale each axis on its own” —much easier to read than a fully mixed map.

\det(A)

A\mathbf{v}=\lambda\mathbf{v}

Eigenvalues and eigenvectors help you pick out the most meaningful directions when many directions are mixed together. In very high-dimensional data, some directions change a lot (important) and others barely change (less important). Eigenvectors of the covariance matrix show where the data spreads the most; eigenvalues say how large that spread is.

They also help when the same matrix is multiplied over and over (weather, markets, web ranking). You can often tell, without simulating every step, whether values explode, shrink toward zero, or settle down .

1. PCA and compression (finding the features that matter) Face photos have millions of pixels, but not every pixel matters equally. After building a covariance matrix, eigenvectors show directions of change. The direction with the largest eigenvalue often matches big cues like eyes or outline. Keeping a few top directions lets you shrink the dimension while keeping most of the information.

2. PageRank — links and steady scores Web pages link to each other. If you model “keep clicking links at random,” the long-run importance scores solve a problem very much like finding a steady score pattern that does not change when the same rule is applied again. 3. Deep nets and dynamics (gradient blow-ups) Models like RNNs multiply the same map many times . If the largest eigen-magnitude is far above 1, values can explode; if all are safely below 1, signals can fade . Designers try to keep spectra near 1 so training stays stable.

A\mathbf{v}=\lambda\mathbf{v}

A\mathbf{v}=\lambda\mathbf{v}

Worked patterns

Example 1 — Definition Question: On an eigen-line, what do we call the stretch number and the direction vector ? Solution: Eigenvalue and eigenvector . Example 2 — True / false Question: Similar matrices share the same characteristic polynomial. Solution: True . Example 3 — Diagonal matrix Question: A 2\times2 diagonal matrix with 3 and 2 on the diagonal—what are the eigenvalues? Solution: 3 and 2 (read the diagonal). Example 4 — Repeated map Question: If the stretch on a line is λ, what happens to that stretch after three identical steps? Solution: Cube the stretch (three repeated scalings). Example 5 — Sum & product Question: Eigenvalues 1 and 4 —what are trace and determinant? Solution: Sum 5, product 4 . Example 6 — PCA Question: Direction for the largest eigenvalue of a covariance matrix? Solution: Direction of maximal variance (first principal component).

Practice

R=\begin{pmatrix}1&0\\0&-1\end{pmatrix}

1 / 10

Eigenvalues and Eigenvectors: Directions That Stay Put

A\mathbf{v}=\lambda\mathbf{v}

2. Eigenvalue — how length changes on that line For an eigenvector, the eigenvalue is the factor by which length changes along that line. If it is greater than 1, it stretches; between 0 and 1, it shrinks. If it is negative, you still stay on the line but point the other way.

3. Characteristic equation — the usual way to find eigenvalues Build the characteristic equation and solve for candidates. Intuitively: pick a trial value until the map squishes enough that “volume” collapses to zero—that trial is a candidate, and the directions you recover are eigenvector candidates.

4. Diagonalization — turn a tangled map into simple scales If you have enough independent eigen-directions for a new basis, you can rewrite the map as “scale each axis on its own” —much easier to read than a fully mixed map.

\det(A)

A\mathbf{v}=\lambda\mathbf{v}

Eigenvalues and eigenvectors help you pick out the most meaningful directions when many directions are mixed together. In very high-dimensional data, some directions change a lot (important) and others barely change (less important). Eigenvectors of the covariance matrix show where the data spreads the most; eigenvalues say how large that spread is.

They also help when the same matrix is multiplied over and over (weather, markets, web ranking). You can often tell, without simulating every step, whether values explode, shrink toward zero, or settle down .

1. PCA and compression (finding the features that matter) Face photos have millions of pixels, but not every pixel matters equally. After building a covariance matrix, eigenvectors show directions of change. The direction with the largest eigenvalue often matches big cues like eyes or outline. Keeping a few top directions lets you shrink the dimension while keeping most of the information.

2. PageRank — links and steady scores Web pages link to each other. If you model “keep clicking links at random,” the long-run importance scores solve a problem very much like finding a steady score pattern that does not change when the same rule is applied again. 3. Deep nets and dynamics (gradient blow-ups) Models like RNNs multiply the same map many times . If the largest eigen-magnitude is far above 1, values can explode; if all are safely below 1, signals can fade . Designers try to keep spectra near 1 so training stays stable.

A\mathbf{v}=\lambda\mathbf{v}

A\mathbf{v}=\lambda\mathbf{v}

Worked patterns

Example 1 — Definition Question: On an eigen-line, what do we call the stretch number and the direction vector ? Solution: Eigenvalue and eigenvector . Example 2 — True / false Question: Similar matrices share the same characteristic polynomial. Solution: True . Example 3 — Diagonal matrix Question: A 2\times2 diagonal matrix with 3 and 2 on the diagonal—what are the eigenvalues? Solution: 3 and 2 (read the diagonal). Example 4 — Repeated map Question: If the stretch on a line is λ, what happens to that stretch after three identical steps? Solution: Cube the stretch (three repeated scalings). Example 5 — Sum & product Question: Eigenvalues 1 and 4 —what are trace and determinant? Solution: Sum 5, product 4 . Example 6 — PCA Question: Direction for the largest eigenvalue of a covariance matrix? Solution: Direction of maximal variance (first principal component).

Practice

R=\begin{pmatrix}1&0\\0&-1\end{pmatrix}

1 / 10

In words	Meaning
Eigenpair	Same line; length scales ( $A\mathbf{v}=\lambda\mathbf{v}$ )
Characteristic equation	The standard recipe to list eigenvalue candidates
Find directions	After a candidate, solve the shifted linear system for a direction
Geo. vs alg. multiplicity	How many independent directions vs how many repeated roots
Diagonalization	Enough independent eigen-directions ⇒ axis-only scaling form
$\det(A)$	Volume/area scaling of the map (Ch.05); equals product of eigenvalues
Trace & determinant	Quick sum and product checks for eigenvalues

In words	Meaning
Eigenpair	Same line; length scales ( $A\mathbf{v}=\lambda\mathbf{v}$ )
Characteristic equation	The standard recipe to list eigenvalue candidates
Find directions	After a candidate, solve the shifted linear system for a direction
Geo. vs alg. multiplicity	How many independent directions vs how many repeated roots
Diagonalization	Enough independent eigen-directions ⇒ axis-only scaling form
$\det(A)$	Volume/area scaling of the map (Ch.05); equals product of eigenvalues
Trace & determinant	Quick sum and product checks for eigenvalues