Spectral Decomposition: QDQᵀ & PCA

What the Spectral Decomposition Is

A

factors as

A = QDQ^T

where

Q

is orthogonal (

Q^TQ = QQ^T = I

) and

D = \text{diag}(\lambda_1, \dots, \lambda_n)

is the diagonal matrix of eigenvalues. The columns of

Q

are orthonormal eigenvectors:

Q = [\mathbf{q}_1 \; \mathbf{q}_2 \; \cdots \; \mathbf{q}_n]

with

A\mathbf{q}_i = \lambda_i\mathbf{q}_i

and

\mathbf{q}_i \cdot \mathbf{q}_j = \delta_{ij}

.

This is the diagonalization

A = PDP^{-1}

specialized to symmetric matrices, where the crucial bonus is that

P

can be chosen orthogonal — so

P^{-1} = P^T

. The orthogonality of

Q

is not a convenience; it is a structural guarantee that holds for every real symmetric matrix, regardless of eigenvalue multiplicities.

Spectral Decomposition Notation

Notation

Spectral Decomposition Notation

A single superscript carries the whole theorem, the same decomposition is written twice in shapes that look unrelated, and the letters collapse into one another when the matrix is symmetric.

The letter

\lambda

, the spectrum and the characteristic determinant are set out at the characteristic equation; the factorisation

A = PDP^{-1}

and the column-to-diagonal pairing at diagonalization. The reserved

Q

with

Q^{T}Q = I

and the Kronecker delta belong to orthogonal sets, the transpose position that separates an outer product from an inner one to QR, and the condition

\mathbf{x}^{T}A\mathbf{x} > 0

with its curved comparison signs to Cholesky.

A = QDQ^{T}

A equals Q D Q transpose

This is

A = PDP^{-1}

with one superscript changed, and the change is the entire content. Writing

Q^{T}

where an inverse is required asserts that

Q^{-1} = Q^{T}

, which holds only for an orthogonal matrix. The line therefore states the spectral theorem in a superscript: the eigenvector matrix can be chosen orthogonal, so no inversion is ever needed.

CasesThe guarantee is unconditional for a real symmetric

A

, including when eigenvalues repeat. Where general diagonalization can fail outright, this factorisation always exists, and the letter

Q

rather than

P

is what announces it.

Also writtenThe complex case replaces the transpose with the conjugate transpose,

A = UDU^{*}

for Hermitian

A

, keeping the shape and widening the hypothesis.

Do not confuseReading

Q^{T}

as a factor unrelated to

Q^{-1}

, which makes the line look like an unusual choice rather than a saving. The two are the same matrix here, and

A = QDQ^{-1}

is equally correct and never written.

Same glyph elsewhereThe exponent

T

marks an ordinary transpose throughout. Nothing about this equation makes it a special operation — the content is that the transpose happens to land on the inverse.

A = QDQ^{T} = \lambda_1\mathbf{q}_1\mathbf{q}_1^{T} + \cdots + \lambda_n\mathbf{q}_n\mathbf{q}_n^{T}

A equals Q D Q transpose, which also equals a sum of n rank-one terms

One decomposition written in two shapes that share no visible structure. On the left, three matrices multiplied. On the right,

n

matrices added. Nothing in either expression records that they are the same thing; the equality comes from expanding the product column by column, and it has to be supplied by the reader.

CasesEach term

\lambda_i\mathbf{q}_i\mathbf{q}_i^{T}

is a rank-one matrix, and the factor

\mathbf{q}_i\mathbf{q}_i^{T}

on its own is the projection onto the line through

\mathbf{q}_i

. The sum form is the one that shows what

A

does; the product form is the one that computes.

Also writtenCompressed to

A = \sum_{i=1}^{n}\lambda_iP_i

where each

P_i

is named as a projector, which hides the vectors and makes the decoupling explicit.

Do not confuseReading the sum as an approximation, in the way a truncated series usually is. With all

n

terms present the equality is exact; dropping terms is a separate decision, and it is what turns the same line into low-rank approximation.

\mathbf{x}^{T}A\mathbf{x} = \mathbf{y}^{T}D\mathbf{y} = \sum_{i=1}^{n}\lambda_iy_i^2, \qquad \mathbf{x} = Q\mathbf{y}

x transpose A x equals y transpose D y, which equals the sum of lambda i times y i squared, where x is Q y

The two sandwiches describe one number, and the substitution written to the right is what licenses the equality. Replacing

A

by

D

is not a simplification of the matrix; it is the same form measured in different coordinates, and

\mathbf{y}

names the coordinates of

\mathbf{x}

along the eigenvectors.

CasesBecause

D

is diagonal, the middle expression has no cross terms, which is what the final sum makes visible:

n

independent squares, each weighted by an eigenvalue. The cross terms did not cancel — they were absorbed into the change of variables.

Also writtenThe substitution is sometimes folded in as

\mathbf{y} = Q^{T}\mathbf{x}

, the same statement read in the other direction, which is available only because

Q^{-1} = Q^{T}

.

Do not confuseTreating

\mathbf{y}

as a new unknown to solve for. It is a relabelling of the same vector, and the sign of each

\lambda_i

then classifies the form — positive definite, indefinite and their relatives are set out at Cholesky.

Same glyph elsewhere

\Sigma

appears on this page as a covariance matrix, as the summation sign above, and at the singular value decomposition as the matrix of singular values — three unrelated jobs for one glyph, distinguished only by position.

U = V = Q, \qquad \sigma_i = |\lambda_i|

U and V both become Q, and each singular value is the absolute value of an eigenvalue

The three-letter form

A = U\Sigma V^{T}

and the two-letter form

A = QDQ^{T}

describe the same matrix when

A

is symmetric with non-negative eigenvalues. The separate names for the outer factors exist to allow them to differ; symmetry removes the need, and the notation contracts.

CasesWhen an eigenvalue is negative the collapse is incomplete. Singular values are never negative, so

\sigma_i = |\lambda_i|

, and the discarded sign is stored by negating the matching column of

U

relative to

V

. The bars are doing bookkeeping, not measurement.

Also writtenWritten

\Sigma = |D|

as a statement about the whole diagonal, which is entrywise here only because both matrices are diagonal.

Do not confuseConcluding that eigenvalues and singular values are the same quantity. They agree for symmetric positive semi-definite matrices and for nothing else, and the sign is the information the absolute value throws away.

All linear algebra symbols →↑ Matrix decompositions

The Spectral Theorem

The spectral theorem for real symmetric matrices states three facts.

All eigenvalues of a real symmetric matrix are real. No complex eigenvalues can appear.

Eigenvectors corresponding to distinct eigenvalues are orthogonal. If

\lambda_i \neq \lambda_j

, then

\mathbf{q}_i \cdot \mathbf{q}_j = 0

automatically — no Gram-Schmidt is needed between different eigenspaces.

Every real symmetric matrix is orthogonally diagonalizable. There always exist

n

orthonormal eigenvectors forming a basis for

\mathbb{R}^n

, even when eigenvalues are repeated. For a repeated eigenvalue with geometric multiplicity

k

, the eigenspace is

k

-dimensional, and Gram-Schmidt within that eigenspace produces

k

orthonormal eigenvectors.

Together these three facts guarantee that

A = QDQ^T

exists for every real symmetric

A

.

The Outer Product Form

Expanding

A = QDQ^T

column by column produces the spectral decomposition in outer product form:

A = \lambda_1 \mathbf{q}_1\mathbf{q}_1^T + \lambda_2 \mathbf{q}_2\mathbf{q}_2^T + \cdots + \lambda_n \mathbf{q}_n\mathbf{q}_n^T

Each term

\lambda_i \mathbf{q}_i\mathbf{q}_i^T

is a rank-one matrix. The matrix

\mathbf{q}_i\mathbf{q}_i^T

is the projection matrix onto the line spanned by

\mathbf{q}_i

: it sends any vector

\mathbf{x}

to

(\mathbf{q}_i \cdot \mathbf{x})\mathbf{q}_i

. Multiplying by

\lambda_i

scales the projection by the eigenvalue.

The spectral decomposition says that

A

acts by projecting onto each eigenvector direction independently, scaling each projection by the corresponding eigenvalue, and summing the results. There is no interaction between different eigenvector directions — the orthogonality of the

\mathbf{q}_i

's ensures complete decoupling.

One eigenvalue times one rank-one piece

Each eigenvector has been multiplied by its own transpose to give a rank-one matrix, and each of those is weighted by its eigenvalue. The sum of these pieces reconstructs the original exactly. Dropping the pieces with the smallest weights is how the decomposition turns into an approximation. Add the pieces back one at a time on the spectral decomposition visualizer.

This is the form principal component analysis uses, where the weights are variances and the pieces are the components.

Computing the Spectral Decomposition

The computation follows the standard eigenvalue workflow, with one additional step for repeated eigenvalues.

Find the eigenvalues by solving the characteristic equation

\det(A - \lambda I) = 0

. All roots are real.

For each eigenvalue

\lambda_i

, find the eigenspace by solving

(A - \lambda_i I)\mathbf{v} = \mathbf{0}

via row reduction.

If an eigenvalue has multiplicity greater than

1

, apply Gram-Schmidt within its eigenspace to produce an orthonormal basis. Eigenvectors from different eigenspaces are already orthogonal — no cross-eigenspace orthogonalization is needed.

Assemble

Q

(orthonormal eigenvectors as columns) and

D

(eigenvalues on the diagonal in matching order).

Worked Example

For

A = \begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix}

: eigenvalues are

\lambda_1 = 4

,

\lambda_2 = 2

. Eigenvectors:

\mathbf{q}_1 = \frac{1}{\sqrt{2}}(1, 1)^T

,

\mathbf{q}_2 = \frac{1}{\sqrt{2}}(1, -1)^T

. Then

A = 4 \cdot \frac{1}{2}\begin{pmatrix} 1 & 1 \\ 1 & 1 \end{pmatrix} + 2 \cdot \frac{1}{2}\begin{pmatrix} 1 & -1 \\ -1 & 1 \end{pmatrix} = \begin{pmatrix} 2 & 2 \\ 2 & 2 \end{pmatrix} + \begin{pmatrix} 1 & -1 \\ -1 & 1 \end{pmatrix} = \begin{pmatrix} 3 & 1 \\ 1 & 3 \end{pmatrix}

.

Q orthogonal, Λ diagonal, Q transposed

Because the eigenvectors of a symmetric matrix can be chosen orthonormal, the matrix of them is orthogonal — and the inverse of an orthogonal matrix is simply its transpose. That is the whole economy of the spectral form: no inverse ever has to be computed. Check the orthogonality yourself on the spectral decomposition visualizer.

Symmetry is doing all the work here; without it the eigenvector matrix need not be orthogonal and the transpose shortcut disappears.

Properties of the Factors

The orthogonality of

Q

makes every operation on the spectral decomposition cheap and clean.

The inverse is immediate:

A^{-1} = QD^{-1}Q^T = Q\,\text{diag}(1/\lambda_1, \dots, 1/\lambda_n)\,Q^T

, valid when all eigenvalues are nonzero.

Powers are diagonal:

A^k = QD^kQ^T = Q\,\text{diag}(\lambda_1^k, \dots, \lambda_n^k)\,Q^T

.

The matrix exponential is:

e^{At} = Qe^{Dt}Q^T = Q\,\text{diag}(e^{\lambda_1 t}, \dots, e^{\lambda_n t})\,Q^T

.

The trace is

\text{tr}(A) = \lambda_1 + \cdots + \lambda_n

. The determinant is

\det(A) = \lambda_1 \cdots \lambda_n

. The rank is the number of nonzero eigenvalues.

Every property of

A

reduces to a property of the diagonal

D

, mediated by the orthogonal rotation

Q

. This is the computational power of diagonalization — and for symmetric matrices, the orthogonality of

Q

ensures numerical stability as well.

Quadratic Forms

A quadratic form

f(\mathbf{x}) = \mathbf{x}^TA\mathbf{x}

with

A

symmetric can be diagonalized by the change of variables

\mathbf{x} = Q\mathbf{y}

:

Quadratic Form Diagonalization

\mathbf{x}^T A \mathbf{x} = \mathbf{y}^T D \mathbf{y} = \sum_{i=1}^{n} \lambda_i y_i^2, \qquad \mathbf{x} = Q\mathbf{y}

Quadratic form diagonalization →

In the eigenvector coordinate system, the quadratic form decouples into a sum of independent squared terms. The eigenvectors define the principal axes of the quadratic surface (ellipsoid, hyperboloid, etc.), and the eigenvalues determine the curvature along each axis.

Positive definite (

f > 0

for

\mathbf{x} \neq \mathbf{0}

) means all

\lambda_i > 0

— the surface is an ellipsoid. Positive semi-definite (

f \geq 0

) means all

\lambda_i \geq 0

. Indefinite (eigenvalues of both signs) means the surface is a hyperboloid —

f

takes both positive and negative values.

Classification of quadratic forms reduces entirely to checking the signs of the eigenvalues. The full taxonomy — including the negative-definite and negative-semi-definite cases — collects into the table below.

Download

PNGRaster · 2×PDFPrint quality

Decompositions · quadratic forms

Definiteness, read from the signs

The quadratic form $\mathbf{x}^{\mathsf{T}}A\mathbf{x}$ is classified entirely by the signs of the eigenvalues — not their sizes. Five classes, and the surface each one describes follows immediately.

5classes

Type	Condition	eigenvalue signs	sign of the form	level surface
One sign throughout
Positive definite§ 6	all $\lambda_i > 0$	all strictly positive	$> 0$ for every $\mathbf{x} \neq \mathbf{0}$	ellipsoid
Negative definite§ 6	all $\lambda_i < 0$	all strictly negative	$< 0$ for every $\mathbf{x} \neq \mathbf{0}$	inverted ellipsoid
One sign, with zeros
Positive semi-definite§ 6	all $\lambda_i \geq 0$ , at least one $= 0$	non-negative, some zero	$\geq 0$ ; vanishes on the null space	degenerate ellipsoid — a cylinder
Negative semi-definite§ 6	all $\lambda_i \leq 0$ , at least one $= 0$	non-positive, some zero	$\leq 0$ ; vanishes on the null space	inverted cylinder
Both signs present
Indefinite§ 6	some $\lambda_i > 0$ and some $< 0$	mixed	takes both signs	saddle — hyperboloid

In the eigenbasis the form becomes

\sum \lambda_i y_i^2

, and the sign question answers itself: a sum of squares with all-positive weights is positive, with mixed weights it takes both signs. Everything below is that substitution read off. In practice Cholesky tests definiteness faster than computing the eigenvalues does.

Definiteness, read from the signs·/linear-algebra/decompositions/spectralLearn Math Class

The classification is immediate once the decomposition is in hand. Substituting

\mathbf{x} = Q\mathbf{y}

turns the form into

\sum \lambda_i y_i^2

— a weighted sum of squares — and the sign of a sum of squares is decided by the signs of the weights and nothing else. The magnitudes only stretch the surface; they never change which of the five classes it belongs to.

Principal Component Analysis

The spectral decomposition of a covariance matrix is the mathematical core of principal component analysis (PCA).

The covariance matrix

\Sigma

of a dataset is symmetric positive semi-definite. Its spectral decomposition

\Sigma = QDQ^T

identifies the eigenvectors

\mathbf{q}_i

as the principal component directions — the orthogonal axes along which the data varies most — and the eigenvalues

\lambda_i

as the variance captured by each direction.

The first principal component

\mathbf{q}_1

(corresponding to the largest eigenvalue

\lambda_1

) is the direction of maximum variance. The second

\mathbf{q}_2

is the direction of maximum variance orthogonal to

\mathbf{q}_1

, and so on.

Dimensionality reduction follows: projecting the data onto the top

k

eigenvectors (those with the

k

largest eigenvalues) captures as much variance as possible in

k

dimensions. The discarded directions have small eigenvalues and contribute little information. This is PCA — the spectral decomposition applied to the covariance matrix.

Spectral Decomposition vs. General Eigendecomposition

The general eigendecomposition writes a diagonalizable matrix as

A = PDP^{-1}

with

P

invertible but not necessarily orthogonal. The spectral decomposition writes a symmetric matrix as

A = QDQ^T

with

Q

orthogonal.

The orthogonality of

Q

provides three advantages. Inversion is free:

Q^{-1} = Q^T

, no computation needed. Multiplication preserves norms:

\|Q\mathbf{x}\| = \|\mathbf{x}\|

, so numerical errors are not amplified. Projections are orthogonal: the rank-one terms

\mathbf{q}_i\mathbf{q}_i^T

are orthogonal projection matrices, making the outer product form geometrically transparent.

The spectral decomposition exists only for symmetric matrices (real case) or Hermitian matrices (complex case). For non-symmetric matrices, the eigendecomposition

PDP^{-1}

may exist (when the matrix is diagonalizable) but

P

is not orthogonal, and the computational and geometric advantages are lost.

Spectral Decomposition and SVD

For a symmetric positive semi-definite matrix

A

(all eigenvalues

\geq 0

), the spectral decomposition and the singular value decomposition coincide. The singular values are the eigenvalues, and

U = V = Q

:

A = QDQ^T = Q\Sigma Q^T

.

For a general symmetric matrix with negative eigenvalues, the relationship requires a sign adjustment. The singular values are the absolute values

|\lambda_i|

, and the signs are absorbed into

U

or

V

. If

\lambda_i < 0

, the corresponding column of

U

is negated relative to the corresponding column of

V

.

The SVD generalizes the spectral decomposition to non-symmetric and non-square matrices. Every property that the spectral decomposition provides for symmetric matrices — rank, pseudoinverse, best low-rank approximation, condition number — the SVD provides for arbitrary matrices. The spectral decomposition is the special case where symmetry allows

U

and

V

to coincide.

Summary: What the Spectral Decomposition Gives You

Once

A = QDQ^T

is in hand, most matrix-level quantities of interest reduce to operations on the diagonal

D

, with the orthogonal

Q

rotating in and out of the eigenvector basis. The table below collects the six properties most commonly read off the decomposition — inverse, power, exponential, trace, determinant, and rank — alongside the formula in terms of

Q

and

D

, the parallel formula in terms of individual eigenvalues, and the conditions under which each formula applies.

Every entry below reads

D

and leaves

Q

alone. That is the structural fact worth carrying away: the orthogonal factor is fixed by the matrix, so anything asked of

A

becomes the same question asked of its eigenvalues. The first group applies a function to them, the second reduces them to a single number.

Download

PNGRaster · 2×PDFPrint quality

Decompositions · spectral

What the decomposition gives you

Once $A = QDQ^{\mathsf{T}}$ is in hand, every entry below is read off $D$ alone. The orthogonal factor never changes — which is the whole reason functions of the matrix become functions of its eigenvalues.

7readings

Functions of the matrix4

1

Powers§ 3

k \geq 0

integer; any

k

if all

\lambda_i > 0

A^k = QD^kQ^{\mathsf{T}}

The middle factors collapse:

QDQ^{\mathsf{T}}QDQ^{\mathsf{T}} = QD^2Q^{\mathsf{T}}

, since

Q^{\mathsf{T}}Q = I

. Raising a diagonal matrix to a power means raising each entry, so

A^{100}

costs no more than

A^2

— the reason diagonalization is worth the effort at all.

2

Inverse§ 5

all

\lambda_i \neq 0

A^{-1} = QD^{-1}Q^{\mathsf{T}}

Inverting a diagonal matrix means reciprocating its entries, so the whole inversion costs nothing once the factorization exists. A zero eigenvalue is precisely what makes it fail — the same condition as everywhere else, read off the diagonal.

3

Square root§ 5

all

\lambda_i \geq 0

— positive semi-definite

A^{1/2} = QD^{1/2}Q^{\mathsf{T}}

A symmetric square root exists exactly when no eigenvalue is negative, and it is unique among positive semi-definite matrices. This is the construction behind Cholesky and behind whitening a covariance matrix.

4

Exponential§ 5

always defined

e^{A} = Qe^{D}Q^{\mathsf{T}}

No condition at all, since

e^x

is defined for every real

x

. This is what solves the linear system of differential equations

\mathbf{x}' = A\mathbf{x}

— the solution is

e^{At}\mathbf{x}_0

, and the decomposition makes it computable.

Scalars, straight off the diagonal3

5

Trace§ 5

always

\operatorname{tr}(A) = \lambda_1 + \cdots + \lambda_n

The trace is similarity invariant, so

\operatorname{tr}(A) = \operatorname{tr}(D)

and

D

is diagonal — the sum is immediate.

6

Determinant§ 5

always

\det(A) = \lambda_1 \lambda_2 \cdots \lambda_n

Same argument with the multiplicative property:

\det Q \cdot \det D \cdot \det Q^{\mathsf{T}} = \det D

, since

\det Q = \pm 1

and the two cancel.

7

Rank§ 5

always

count of

\lambda_i \neq 0

Multiplying by an invertible matrix preserves rank, so

\operatorname{rank}(A) = \operatorname{rank}(D)

, which is the number of nonzero diagonal entries. For a symmetric matrix, algebraic and geometric multiplicity always agree — so this count is unambiguous in a way it is not for a general matrix.

The pattern is one rule:

f(A) = Qf(D)Q^{\mathsf{T}}

, and

f(D)

applies

f

to each diagonal entry. Powers, inverses, roots and exponentials are all the same statement with a different

f

, which is what makes a symmetric matrix genuinely easier to work with rather than merely nicer to look at.

What the decomposition gives you·/linear-algebra/decompositions/spectralLearn Math Class

The four function entries are one rule with four choices of

f

: powers, reciprocals, roots and exponentials all reduce to

f(A) = Qf(D)Q^{\mathsf{T}}

. What differs is only which eigenvalues

f

accepts — every real number for the exponential, nonzero for the inverse, non-negative for the square root. Read the condition column downward and it is a list of what each function needs, not a list of separate theorems.

This works because

A

is symmetric. A general matrix may fail to diagonalize at all, and even when it does the eigenvector matrix need not be orthogonal, so

P^{-1}

must be computed rather than transposed. The spectral theorem is what guarantees both the orthogonality and the existence — which is why this page can state everything unconditionally where the general case cannot.

Spectral Decomposition FAQ

Why is it written $QDQ^{T}$ rather than $QDQ^{-1}$ ?

+

Because for an orthogonal

Q

the two are the same matrix, and the transpose announces that no inversion is needed. Writing

A = QDQ^{-1}

is equally correct and essentially never seen, since it hides the saving. The letter

Q

rather than

P

is itself the signal that the change of basis is orthogonal.Read more →

Is the outer product sum an approximation?

+

Not with all its terms present, where the equality is exact. The instinct comes from truncated series, but nothing is being discarded here: each term

\lambda_i\mathbf{q}_i\mathbf{q}_i^{T}

is a rank-one piece and together they reconstruct

A

perfectly. Dropping terms is a separate decision, and that is precisely what turns the same line into low-rank approximation.Read more →

Are singular values the same as eigenvalues?

+

They coincide only for symmetric positive semi-definite matrices and for nothing else. Singular values are never negative, so where an eigenvalue is negative you get

\sigma_i = |\lambda_i|

and the discarded sign is stored by negating the matching column elsewhere in the factorisation. The absolute value is doing bookkeeping, not measurement.Read more →

Spectral Decompositions

A Symmetric Matrix as a Sum of Projections

What the Spectral Decomposition Is

Spectral Decomposition Notation

Spectral Decomposition Notation

The Spectral Theorem

The Outer Product Form

Computing the Spectral Decomposition

Worked Example

Properties of the Factors

Quadratic Forms

Definiteness, read from the signs

Principal Component Analysis

Spectral Decomposition vs. General Eigendecomposition

Spectral Decomposition and SVD

Summary: What the Spectral Decomposition Gives You

What the decomposition gives you

Spectral Decomposition FAQ

Why is it written $QDQ^{T}$ rather than $QDQ^{-1}$ ?

Is the outer product sum an approximation?

Are singular values the same as eigenvalues?

Spectral Decompositions

A Symmetric Matrix as a Sum of Projections

What the Spectral Decomposition Is

Spectral Decomposition Notation

Spectral Decomposition Notation

The Spectral Theorem

The Outer Product Form

Computing the Spectral Decomposition

Worked Example

Properties of the Factors

Quadratic Forms

Definiteness, read from the signs

Principal Component Analysis

Spectral Decomposition vs. General Eigendecomposition

Spectral Decomposition and SVD

Summary: What the Spectral Decomposition Gives You

What the decomposition gives you

Spectral Decomposition FAQ

Why is it written QDQTQDQ^{T}QDQT rather than QDQ−1QDQ^{-1}QDQ−1?

Is the outer product sum an approximation?

Are singular values the same as eigenvalues?

Why is it written $QDQ^{T}$ rather than $QDQ^{-1}$ ?