What is the spectral decomposition?

The spectral decomposition factors a real symmetric matrix A as A = QDQᵀ, where Q is orthogonal (columns are orthonormal eigenvectors) and D is diagonal (eigenvalues on the diagonal). In outer product form, A is a sum of rank-one projections weighted by eigenvalues.

What does the spectral theorem guarantee?

The spectral theorem guarantees three things for real symmetric matrices: all eigenvalues are real, eigenvectors from distinct eigenvalues are automatically orthogonal, and the matrix is always orthogonally diagonalizable — even when eigenvalues are repeated.

How does the spectral decomposition classify quadratic forms?

A quadratic form xᵀAx diagonalizes to λ₁y₁² + λ₂y₂² + ⋯ + λₙyₙ² in eigenvector coordinates. All positive eigenvalues mean positive definite (ellipsoid), all non-negative mean positive semi-definite, and mixed signs mean indefinite (hyperboloid). Classification reduces to checking eigenvalue signs.

How is the spectral decomposition used in PCA?

PCA applies the spectral decomposition to a covariance matrix. The eigenvectors are the principal component directions (axes of maximum variance), and the eigenvalues measure the variance along each direction. Projecting data onto the top k eigenvectors achieves optimal dimensionality reduction.

What is the difference between spectral decomposition and SVD?

The spectral decomposition A = QDQᵀ applies only to symmetric matrices and uses a single orthogonal matrix Q. The SVD A = UΣVᵀ applies to any matrix and uses two orthogonal matrices. For symmetric positive semi-definite matrices, the two coincide with U = V = Q and singular values equal to eigenvalues.

Page Title

What the Spectral Decomposition Is

The Spectral Theorem

The Outer Product Form

Computing the Spectral Decomposition

Properties of the Factors

Quadratic Forms

Principal Component Analysis

Spectral Decomposition vs. General Eigendecomposition

Spectral Decomposition and SVD

A Symmetric Matrix as a Sum of Projections

The spectral decomposition factors a real symmetric matrix as QDQᵀ — an orthogonal matrix of eigenvectors times a diagonal matrix of eigenvalues times the transpose. In outer product form, this becomes a sum of rank-one projections weighted by eigenvalues. The decomposition is the factorization form of the spectral theorem and the foundation of principal component analysis, quadratic form classification, and positive definiteness testing.

What the Spectral Decomposition Is

Every real symmetric matrix

A

factors as

A = QDQ^T

where

Q

is orthogonal (

Q^TQ = QQ^T = I

) and

D = \text{diag}(\lambda_1, \dots, \lambda_n)

is the diagonal matrix of eigenvalues. The columns of

Q

are orthonormal eigenvectors:

Q = [\mathbf{q}_1 \; \mathbf{q}_2 \; \cdots \; \mathbf{q}_n]

with

A\mathbf{q}_i = \lambda_i\mathbf{q}_i

and

\mathbf{q}_i \cdot \mathbf{q}_j = \delta_{ij}

.

This is the diagonalization

A = PDP^{-1}

specialized to symmetric matrices, where the crucial bonus is that

P

can be chosen orthogonal — so

P^{-1} = P^T

. The orthogonality of

Q

is not a convenience; it is a structural guarantee that holds for every real symmetric matrix, regardless of eigenvalue multiplicities.

The Spectral Theorem

The spectral theorem for real symmetric matrices states three facts.

All eigenvalues of a real symmetric matrix are real. No complex eigenvalues can appear.

Eigenvectors corresponding to distinct eigenvalues are orthogonal. If

\lambda_i \neq \lambda_j

, then

\mathbf{q}_i \cdot \mathbf{q}_j = 0

automatically — no Gram-Schmidt is needed between different eigenspaces.

Every real symmetric matrix is orthogonally diagonalizable. There always exist

n

orthonormal eigenvectors forming a basis for

\mathbb{R}^n

, even when eigenvalues are repeated. For a repeated eigenvalue with geometric multiplicity

k

, the eigenspace is

k

-dimensional, and Gram-Schmidt within that eigenspace produces

k

orthonormal eigenvectors.

Together these three facts guarantee that

A = QDQ^T

exists for every real symmetric

A

The Outer Product Form

Expanding

A = QDQ^T

column by column produces the spectral decomposition in outer product form:

A = \lambda_1 \mathbf{q}_1\mathbf{q}_1^T + \lambda_2 \mathbf{q}_2\mathbf{q}_2^T + \cdots + \lambda_n \mathbf{q}_n\mathbf{q}_n^T

Each term

\lambda_i \mathbf{q}_i\mathbf{q}_i^T

is a rank-one matrix. The matrix

\mathbf{q}_i\mathbf{q}_i^T

is the projection matrix onto the line spanned by

\mathbf{q}_i

: it sends any vector

\mathbf{x}

(\mathbf{q}_i \cdot \mathbf{x})\mathbf{q}_i

. Multiplying by

\lambda_i

scales the projection by the eigenvalue.

The spectral decomposition says that

A

acts by projecting onto each eigenvector direction independently, scaling each projection by the corresponding eigenvalue, and summing the results. There is no interaction between different eigenvector directions — the orthogonality of the

\mathbf{q}_i

's ensures complete decoupling.

Computing the Spectral Decomposition

The computation follows the standard eigenvalue workflow, with one additional step for repeated eigenvalues.

Find the eigenvalues by solving the characteristic equation

\det(A - \lambda I) = 0

. All roots are real.

For each eigenvalue

\lambda_i

, find the eigenspace by solving

(A - \lambda_i I)\mathbf{v} = \mathbf{0}

via row reduction.

If an eigenvalue has multiplicity greater than

1

, apply Gram-Schmidt within its eigenspace to produce an orthonormal basis. Eigenvectors from different eigenspaces are already orthogonal — no cross-eigenspace orthogonalization is needed.

Assemble

Q

(orthonormal eigenvectors as columns) and

D

(eigenvalues on the diagonal in matching order).

Worked Example

For

A = \begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix}

: eigenvalues are

\lambda_1 = 4

\lambda_2 = 2

. Eigenvectors:

\mathbf{q}_1 = \frac{1}{\sqrt{2}}(1, 1)^T

\mathbf{q}_2 = \frac{1}{\sqrt{2}}(1, -1)^T

. Then

A = 4 \cdot \frac{1}{2}\begin{pmatrix} 1 & 1 \\ 1 & 1 \end{pmatrix} + 2 \cdot \frac{1}{2}\begin{pmatrix} 1 & -1 \\ -1 & 1 \end{pmatrix} = \begin{pmatrix} 2 & 2 \\ 2 & 2 \end{pmatrix} + \begin{pmatrix} 1 & -1 \\ -1 & 1 \end{pmatrix} = \begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix}

Properties of the Factors

The orthogonality of

Q

makes every operation on the spectral decomposition cheap and clean.

The inverse is immediate:

A^{-1} = QD^{-1}Q^T = Q\,\text{diag}(1/\lambda_1, \dots, 1/\lambda_n)\,Q^T

, valid when all eigenvalues are nonzero.

Powers are diagonal:

A^k = QD^kQ^T = Q\,\text{diag}(\lambda_1^k, \dots, \lambda_n^k)\,Q^T

.

The matrix exponential is:

e^{At} = Qe^{Dt}Q^T = Q\,\text{diag}(e^{\lambda_1 t}, \dots, e^{\lambda_n t})\,Q^T

.

The trace is

\text{tr}(A) = \lambda_1 + \cdots + \lambda_n

. The determinant is

\det(A) = \lambda_1 \cdots \lambda_n

. The rank is the number of nonzero eigenvalues.

Every property of

A

reduces to a property of the diagonal

D

, mediated by the orthogonal rotation

Q

. This is the computational power of diagonalization — and for symmetric matrices, the orthogonality of

Q

ensures numerical stability as well.

Quadratic Forms

A quadratic form

f(\mathbf{x}) = \mathbf{x}^TA\mathbf{x}

with

A

symmetric can be diagonalized by the change of variables

\mathbf{x} = Q\mathbf{y}

f = \mathbf{y}^TQ^TAQ\mathbf{y} = \mathbf{y}^TD\mathbf{y} = \lambda_1 y_1^2 + \lambda_2 y_2^2 + \cdots + \lambda_n y_n^2

In the eigenvector coordinate system, the quadratic form decouples into a sum of independent squared terms. The eigenvectors define the principal axes of the quadratic surface (ellipsoid, hyperboloid, etc.), and the eigenvalues determine the curvature along each axis.

Positive definite (

f > 0

for

\mathbf{x} \neq \mathbf{0}

) means all

\lambda_i > 0

— the surface is an ellipsoid. Positive semi-definite (

f \geq 0

) means all

\lambda_i \geq 0

. Indefinite (eigenvalues of both signs) means the surface is a hyperboloid —

f

takes both positive and negative values.

Classification of quadratic forms reduces entirely to checking the signs of the eigenvalues.

Principal Component Analysis

The spectral decomposition of a covariance matrix is the mathematical core of principal component analysis (PCA).

The covariance matrix

\Sigma

of a dataset is symmetric positive semi-definite. Its spectral decomposition

\Sigma = QDQ^T

identifies the eigenvectors

\mathbf{q}_i

as the principal component directions — the orthogonal axes along which the data varies most — and the eigenvalues

\lambda_i

as the variance captured by each direction.

The first principal component

\mathbf{q}_1

(corresponding to the largest eigenvalue

\lambda_1

) is the direction of maximum variance. The second

\mathbf{q}_2

is the direction of maximum variance orthogonal to

\mathbf{q}_1

, and so on.

Dimensionality reduction follows: projecting the data onto the top

k

eigenvectors (those with the

k

largest eigenvalues) captures as much variance as possible in

k

dimensions. The discarded directions have small eigenvalues and contribute little information. This is PCA — the spectral decomposition applied to the covariance matrix.

Spectral Decomposition vs. General Eigendecomposition

The general eigendecomposition writes a diagonalizable matrix as

A = PDP^{-1}

with

P

invertible but not necessarily orthogonal. The spectral decomposition writes a symmetric matrix as

A = QDQ^T

with

Q

orthogonal.

The orthogonality of

Q

provides three advantages. Inversion is free:

Q^{-1} = Q^T

, no computation needed. Multiplication preserves norms:

\|Q\mathbf{x}\| = \|\mathbf{x}\|

, so numerical errors are not amplified. Projections are orthogonal: the rank-one terms

\mathbf{q}_i\mathbf{q}_i^T

are orthogonal projection matrices, making the outer product form geometrically transparent.

The spectral decomposition exists only for symmetric matrices (real case) or Hermitian matrices (complex case). For non-symmetric matrices, the eigendecomposition

PDP^{-1}

may exist (when the matrix is diagonalizable) but

P

is not orthogonal, and the computational and geometric advantages are lost.

Spectral Decomposition and SVD

For a symmetric positive semi-definite matrix

A

(all eigenvalues

\geq 0

), the spectral decomposition and the singular value decomposition coincide. The singular values are the eigenvalues, and

U = V = Q

A = QDQ^T = Q\Sigma Q^T

.

For a general symmetric matrix with negative eigenvalues, the relationship requires a sign adjustment. The singular values are the absolute values

|\lambda_i|

, and the signs are absorbed into

U

V

. If

\lambda_i < 0

, the corresponding column of

U

is negated relative to the corresponding column of

V

.

The SVD generalizes the spectral decomposition to non-symmetric and non-square matrices. Every property that the spectral decomposition provides for symmetric matrices — rank, pseudoinverse, best low-rank approximation, condition number — the SVD provides for arbitrary matrices. The spectral decomposition is the special case where symmetry allows

U

and

V

to coincide.