SVD: Singular Value Decomposition

What the Singular Value Decomposition Is

A matrix mixes coordinates. Every entry of the output is assembled from every entry of the input, so almost nothing worth knowing about

A

can be read from

A

itself. What is

A^{10}

? Multiply ten times and find out. What happens to a vector pushed through

A

a thousand times — does it grow, shrink, settle into some direction? There is no answer short of running it.

Factorization removes the obstacle. Split

A

into factors that each do one simple thing, and questions that required arithmetic can be settled by inspection.

Diagonalization and Its Requirement

Diagonalization does exactly that. Writing

A = PDP^{-1}

amounts to changing coordinates so that the mixing is absent: the columns of

P

are directions the matrix only stretches,

D

holds the stretching factors, and in that basis no coordinate influences any other.

Take

B = \begin{pmatrix} 2 & 1 \\ 0 & 1 \end{pmatrix}

.

Its eigenvalues are

2

and

1

, with eigenvectors

(1, 0)

and

(1, -1)

, so

B = PDP^{-1}, \qquad P = \begin{pmatrix} 1 & 1 \\ 0 & -1 \end{pmatrix}, \qquad D = \begin{pmatrix} 2 & 0 \\ 0 & 1 \end{pmatrix}

The tenth power then costs nothing:

B^{10} = PD^{10}P^{-1}

, and

D^{10} = \begin{pmatrix} 1024 & 0 \\ 0 & 1 \end{pmatrix}

because

2^{10} = 1024

. Restoring the original coordinates gives

B^{10} = \begin{pmatrix} 1024 & 1023 \\ 0 & 1 \end{pmatrix}

without a single matrix multiplication. The inverse is

PD^{-1}P^{-1}

, with no elimination anywhere. Long-run behaviour is governed by whichever

\lambda

is largest, here

2

. Nothing is approximated and nothing is recomputed — the work was done once, when the factorization was found.

That performance comes with a condition, and the condition is visible in the formula. The same matrix

P

stands on both sides of

A = PDP^{-1}

: its columns are used to read the input and again to write the output. The factorization requires that both ends be described in identical terms — one basis, serving twice.

Two matrices, two ways the requirement fails.

A

3 \times 2

matrix takes vectors from

\mathbb{R}^2

and returns vectors in

\mathbb{R}^3

, so no single basis can describe both ends, and the question of an eigenvector cannot even be posed:

A\mathbf{v} = \lambda\mathbf{v}

compares vectors living in different spaces. A defective matrix fails otherwise. It is square, but every eigenvector is a multiple of one direction, so no basis of eigenvectors exists and

P

cannot be assembled. Rectangular shape excludes diagonalization in principle, defective structure excludes it in practice, and between them they account for most matrices anyone meets.

The second failure takes three lines to verify.

Change one entry of

B

and take

A = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}

. Its characteristic polynomial is

\det(A - \lambda I) = (1 - \lambda)^2

so

\lambda = 1

is a repeated root. Solving

(A - I)\mathbf{v} = \mathbf{0}

means solving

\begin{pmatrix} 0 & 1 \\ 0 & 0 \end{pmatrix}\mathbf{v} = \mathbf{0}

, which forces

v_2 = 0

and leaves

v_1

free. Every eigenvector is a multiple of

(1, 0)

: one direction, in two dimensions. There is no second column for

P

, so

A

has no diagonalization — not because the arithmetic is hard, but because the object being asked for does not exist.

Two Bases Instead of One

Nothing forced the two bases to coincide. That was an additional demand, and it can be withdrawn. Admit one orthonormal basis for the input space and a separate one for the output space, and require only that the matrix carry the first to the second with a scaling in between. Equal dimensions are then unnecessary, fixed directions are unnecessary, and both obstructions vanish together.

The unit circle becomes an ellipse. Each

\mathbf{v}_i

is carried to a multiple of

\mathbf{u}_i

, and

\sigma_i

is that multiple.

The picture states the entire factorization in one equation. There is an orthonormal basis

\mathbf{v}_1, \dots, \mathbf{v}_n

of the input space and an orthonormal basis

\mathbf{u}_1, \dots, \mathbf{u}_m

of the output space satisfying

A\mathbf{v}_i = \sigma_i \mathbf{u}_i

for every

i

. Each input basis vector is sent to a multiple of a single output basis vector, and to nothing else.

The matrix that had no diagonalization has this one.

For the same

A = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix}

, form

A^TA = \begin{pmatrix} 1 & 1 \\ 1 & 2 \end{pmatrix}, \qquad \det(A^TA - \lambda I) = \lambda^2 - 3\lambda + 1

whose roots are

\lambda = \frac{3 \pm \sqrt{5}}{2}

. The scaling factors are their square roots,

\sigma_1 = \frac{1 + \sqrt{5}}{2} \approx 1.618, \qquad \sigma_2 = \frac{\sqrt{5} - 1}{2} \approx 0.618

Two of them, both real, both positive, for a matrix with only one eigenvector. As a check,

\sigma_1\sigma_2 = 1

, which is

|\det A|

— the area scaling has to agree however it is computed. The eigenvectors of

A^TA

supply

\mathbf{v}_1

and

\mathbf{v}_2

, and

\mathbf{u}_i = A\mathbf{v}_i/\sigma_i

supplies the other basis.

The definition

Collecting those

n

statements into one matrix equation gives the singular value decomposition: an orthonormal input basis in

V

, an orthonormal output basis in

U

, and the scaling factors between them along the diagonal of

\Sigma

.

SVD

A = U\Sigma V^T

Svd →

It is a factorization, not an approximation: the three matrices multiply back to

A

exactly, with no error term and no conditions attached.

U

is

m \times m

orthogonal and its columns are the left singular vectors.

V

is

n \times n

orthogonal and its columns are the right singular vectors.

\Sigma

is

m \times n

, zero everywhere off the diagonal, carrying

\sigma_1 \geq \sigma_2 \geq \cdots \geq 0

along it. Those are the singular values, and they are the lengths of the semi-axes of the ellipse above.

What Follows and What Does Not

Several properties follow from the construction rather than from any further assumption. Both bases are orthonormal rather than merely independent, so

U^{-1} = U^T

and

V^{-1} = V^T

and neither factor is ever inverted. The scaling factors are real and non-negative, since they measure lengths, and can therefore be ordered from largest to smallest — an ordering no set of eigenvalues admits, eigenvalues being possibly complex. And the factorization exists for every matrix without exception: any shape, any rank, symmetric or not, invertible or not.

The same relaxation imposes a limitation, and the algebra shows exactly where.

Applying the matrix twice gives

A^2 = U\Sigma V^T \cdot U\Sigma V^T

and everything depends on the interior

V^TU

. Under diagonalization the corresponding product is

P^{-1}P = I

, which cancels and leaves

PD^2P^{-1}

. Here

V

and

U

are different orthogonal matrices, so

V^TU

is not the identity but a rotation — for the shear above, by about

26.6^{\circ}

— and nothing collapses. The question this section opened with, the tenth power and the long run, is therefore the one question the singular value decomposition cannot answer and diagonalization can. The two factorizations address different questions: one invariant directions, the other stretching. What the three factors do to a vector geometrically is the subject of the next section.

The Geometric Interpretation

Every linear transformation

\mathbf{x} \mapsto A\mathbf{x}

decomposes into three geometric steps:

V^T

rotates (or reflects) the input space, aligning the input with the "natural axes" of the transformation — the directions along which

A

stretches most and least.

\Sigma

scales each axis independently by the corresponding singular value. Axes with

\sigma_i = 0

are annihilated — those directions are collapsed to zero.

U

rotates (or reflects) the scaled result into the output space.

The singular values measure the stretching in each orthogonal direction.

\sigma_1

is the maximum stretching:

\sigma_1 = \max_{\|\mathbf{x}\|=1}\|A\mathbf{x}\|

. The smallest nonzero singular value

\sigma_r

is the minimum stretching on the row space. For a full-rank matrix, the ratio

\sigma_1/\sigma_r

is the condition number — it measures how distorted the transformation is.

Even the most complex-looking matrix is geometrically just two rotations or reflections sandwiching a coordinate-axis scaling.

SVD Notation

Notation

SVD Notation

Letters that are contracts, a summation sign moonlighting as a matrix, and the lowercase sigma with a set-valued cousin one chapter away.

A^{T}

and matrix products — matrix notation;

\lambda

and

\sigma(A)

— eigenvalue notation;

\|\cdot\|

— norm notation.

A = U\Sigma V^{T}

A equals U, Sigma, V-transpose

In decomposition notation the letters are the theorem: writing

U\Sigma V^{T}

asserts

U

and

V

orthogonal and

\Sigma

diagonal — three contracts in three glyphs. The whole family works this way:

LU

promises triangles,

QR

promises orthogonal-times-triangular,

PDP^{-1}

promises a diagonal

D

.

CasesThe contracts extend to the derived marks: the pseudoinverse of The Pseudoinverse below inverts the letters that can be inverted —

A^{+} = V\Sigma^{+}U^{T}

— with the superscript

+

joining

n

,

-1

,

T

and

\perp

in the crowded label corner.

Also written

U\Sigma V^{*}

over the complex numbers, the star being the conjugate transpose — the modern adjoint of the adjugate's naming hazard.

Do not confuseThe family's own letters.

LU

's

U

is upper triangular; SVD's

U

is orthogonal — the same letter signs two different contracts two pages apart, and only the decomposition's name says which is in force.

\Sigma

Sigma — here, a diagonal matrix

Capital sigma as an object, not an operator: the rectangular diagonal matrix carrying the singular values, doing the scaling step of The Geometric Interpretation above. Maximum-contrast glyph reuse — everywhere else on this site,

\Sigma

sums things.

CasesShape bookkeeping: for an

m \times n

matrix

A

,

\Sigma

is

m \times n

— diagonal entries

\sigma_1, \ldots, \sigma_r

, then zeros padding to the rectangle. The compact form of Compact and Thin Forms below trims the padding.

Also written

\operatorname{diag}(\sigma_1, \ldots, \sigma_r)

— the constructor spelling, with the shape stated alongside;

D

or

S

in software documentation, where sigma is hard to type.

Do not confuseThe summation sign. No bounds above or below, no summand after — a bare

\Sigma

between two matrix letters is a factor, not an instruction to add.

\sigma_1 \geq \sigma_2 \geq \cdots \geq 0

The singular values, in decreasing order

Lowercase sigma with an index — and the ordering is part of the notation: writing

\sigma_1

means the largest, by the standing convention of Singular Values below. No other subscripted family on this site carries a built-in sort.

CasesThe extremes have second names:

\sigma_1 = \|A\|_2

, the operator norm, and, for full-rank

A

,

\sigma_1/\sigma_r = \kappa(A)

, the condition number — kappa being numerical analysis's own reserved letter.

Also written

s_i

in some numerical texts and most software output, freeing sigma for statistics.

Do not confuseThe set-valued cousin:

\sigma(A)

with no subscript is the spectrum — the set of eigenvalues. Subscripted sigma is a singular value, bare sigma-of is a set; eigenvalues and singular values agree only for special matrices, so the two sigmas rarely even hold the same numbers.

All linear algebra symbols →↑ Decompositions

DownloadPNG2×PDF

Notation · Cheat sheet

Decomposition Notation at a Glance

The marks from the notation cards across the decomposition pages: how each is read and what it means. Every group links to the full card on its page.

LU decomposition

A = LU

A equals L U

Lower triangular times upper triangular.

PA = LU

permute, then factor

P

on the left reorders the rows of

A

.

l_{ij}, \; i > j

entries of L below the diagonal

The multipliers used in elimination; ones on the diagonal.

QR decomposition

A = QR

A equals Q R

Orthonormal columns times upper triangular;

Q_1R_1

is the thin form.

Q^{T}Q = I

but not always Q Q transpose

For a thin

Q

,

QQ^{T} \neq I

.

A_{k+1} = R_kQ_k

the QR iteration

Factor, then multiply back in reverse order.

Cholesky decomposition

A = LL^{T}

L times L transpose

One factor, used twice;

LL \neq A

.

\mathbf{x}^{T}A\mathbf{x} > 0

positive definite

Required for all

\mathbf{x} \neq \mathbf{0}

.

A = LDL^{T}

L D L transpose

The diagonal carried separately; no square roots.

Spectral decomposition

A = QDQ^{T}

Q D Q transpose

For symmetric

A

:

Q^{T}

stands where

Q^{-1}

would be.

\sum \lambda_i\mathbf{q}_i\mathbf{q}_i^{T}

a sum of rank-one terms

The same matrix, written by eigenvalues.

Singular value decompositionThis page

A = U\Sigma V^{T}

U Sigma V transpose

Orthogonal

U

and

V

, diagonal

\Sigma

.

\Sigma

capital sigma

A diagonal matrix here, not a sum.

\sigma_1 \geq \sigma_2 \geq \cdots \geq 0

the singular values

\sigma_1

always means the largest.

Same marks, different job

\Sigma

A diagonal matrix in the SVD; elsewhere the summation sign.

Q

Always orthonormal columns;

Q^{T}

then acts as the inverse.

L

Lower triangular in both LU and Cholesky, but with ones on the diagonal only in LU.

Decomposition symbols table Decompositions

Decomposition Notation at a Glance·/linear-algebra/decompositions/lower-upperLearn Math Class

Singular Values

The singular values of

A

are the square roots of the eigenvalues of

A^TA

(or equivalently

AA^T

):

Singular Values

\sigma_i = \sqrt{\lambda_i(A^TA)} = \sqrt{\lambda_i(AA^T)}

Singular values →

Since

A^TA

is symmetric positive semi-definite, its eigenvalues are all

\geq 0

, so the singular values are real and non-negative. They are ordered

\sigma_1 \geq \sigma_2 \geq \cdots \geq 0

.

The number of nonzero singular values equals the rank of

A

:

SVD Rank

\text{rank}(A) = \#\{i : \sigma_i > 0\}

Svd rank →

This is the most numerically stable method for determining rank: compute the SVD and count singular values above a tolerance.

Pitfall: Forgetting the square root

Taking the eigenvalues of

A^TA

as the singular values gives

3

and

1

for

A = \begin{pmatrix} 1 & 0 \\ 0 & 1 \\ 1 & 1 \end{pmatrix}

, so

\sigma_1^2 + \sigma_2^2 = 10

, while the squared entries of

A

add up to

4

. The singular values are

\sqrt{3}

and

1

.

The largest singular value

\sigma_1

is the operator norm

\|A\|_2 = \max_{\|\mathbf{x}\|=1}\|A\mathbf{x}\|

. The Frobenius norm is

\|A\|_F = \sqrt{\sigma_1^2 + \sigma_2^2 + \cdots + \sigma_r^2}

. For full-rank

A

, the condition number is

\kappa(A) = \sigma_1/\sigma_r

— a large condition number means the matrix is nearly singular and small perturbations in the input cause large changes in the output.

Singular values as roots of eigenvalues

The eigenvalues of the symmetric product have been found and their square roots taken. They are never negative, because that product cannot have negative eigenvalues, and they are listed largest first by convention. This is why every matrix has singular values even when it has no eigenvalues of its own. Compute them for a rectangular matrix on the SVD visualizer.

Their ordering is what makes truncation meaningful: cutting the list short always discards the least important part first.

Computing the SVD

The standard hand method computes the SVD through the eigenvalue decomposition of

A^TA

.

Form

A^TA

(symmetric,

n \times n

). Find its eigenvalues

\lambda_1 \geq \cdots \geq \lambda_n \geq 0

and orthonormal eigenvectors

\mathbf{v}_1, \dots, \mathbf{v}_n

using the spectral decomposition. These are the right singular vectors:

V = [\mathbf{v}_1 \; \cdots \; \mathbf{v}_n]

.

The singular values are

\sigma_i = \sqrt{\lambda_i}

. The left singular vectors are computed from the right ones:

\mathbf{u}_i = \frac{1}{\sigma_i}A\mathbf{v}_i

for each nonzero

\sigma_i

. If

r < m

, extend

\{\mathbf{u}_1, \dots, \mathbf{u}_r\}

to an orthonormal basis for

\mathbb{R}^m

.

Worked Example

For

A = \begin{pmatrix} 1 & 0 \\ 0 & 1 \\ 1 & 1 \end{pmatrix}

:

A^TA = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}

, eigenvalues

3

and

1

, eigenvectors

\frac{1}{\sqrt{2}}(1, 1)^T

and

\frac{1}{\sqrt{2}}(1, -1)^T

. Singular values:

\sqrt{3}

and

1

. Left singular vectors:

\mathbf{u}_1 = \frac{1}{\sqrt{3}}A\mathbf{v}_1 = \frac{1}{\sqrt{6}}(1, 1, 2)^T

,

\mathbf{u}_2 = A\mathbf{v}_2 = \frac{1}{\sqrt{2}}(1, -1, 0)^T

. Extend with

\mathbf{u}_3 = \frac{1}{\sqrt{3}}(-1, -1, 1)^T

.

Pitfall: Choosing the left vectors separately

Taking each

\mathbf{u}_i

as an eigenvector of

AA^T

on its own can flip a sign:

\mathbf{u}_1 = -\frac{1}{\sqrt{6}}(1, 1, 2)^T

is also a unit eigenvector, but with it

U\Sigma V^T = \begin{pmatrix} 0 & -1 \\ -1 & 0 \\ -1 & -1 \end{pmatrix} \neq A

. With

\mathbf{u}_1 = \frac{1}{\sigma_1}A\mathbf{v}_1

the product is

A

.

In the example, the columns of

V

came from the eigenvectors of

A^TA

, and the first two columns of

U

were then produced from them one at a time by

\mathbf{u}_i = \frac{1}{\sigma_i}A\mathbf{v}_i

. Each such pair satisfies

A\mathbf{v}_i = \sigma_i\mathbf{u}_i

, which is the column-by-column form of

AV = U\Sigma

. Only

\mathbf{u}_3

, which meets the zero row of

\Sigma

, is fixed by orthogonality to

\mathbf{u}_1

and

\mathbf{u}_2

alone.

The three factors assembled

Two orthogonal matrices with a rectangular diagonal between them, and the whole product reconstructs the original. Unlike diagonalisation this asks nothing of the matrix — not squareness, not symmetry, not a full set of eigenvectors. Every matrix without exception has this factorisation. Build it for a matrix of your own on the SVD visualizer.

That universality is why the SVD, rather than the eigendecomposition, is the factorisation numerical work reaches for by default.

Compact and Thin Forms

The full SVD has

U

of size

m \times m

,

\Sigma

of size

m \times n

, and

V

of size

n \times n

. Two economical alternatives retain only the essential information.

The thin SVD keeps only the first

n

columns of

U

(call them

U_1

) and the top

n \times n

block of

\Sigma

(call it

\Sigma_1

):

A = U_1 \Sigma_1 V^T

. This drops the columns of

U

corresponding to the left null space.

The compact SVD keeps only the first

r

columns of

U

and

V

(where

r = \text{rank}(A)

) and the

r \times r

diagonal block of nonzero singular values:

A = U_r \Sigma_r V_r^T

. This is the most economical representation — it captures only the rank-

r

content of

A

, discarding everything associated with zero singular values.

All three forms represent the same matrix

A

. The compact form uses the least storage; the full form provides bases for all four fundamental subspaces. The three variants line up cleanly on factor dimensions and on what each one chooses to keep.

DownloadPNG2×PDF

Decompositions · SVD

Three forms, one factorization

All three reconstruct $A$ exactly. They differ in how much of $U$ and $\Sigma$ is kept — and what is dropped is precisely the part that multiplies against zero singular values.

3forms

Everything kept1

Full SVD§ 1

U

and

V

both square and orthogonal

shapes

U: m \times m

,

\Sigma: m \times n

,

V: n \times n

what is keptall of

U

,

V

, and a rectangular

\Sigma

what is droppednothing

used fortheory, the four subspaces

The only form where

U

and

V

are genuinely orthogonal matrices, so

UU^{\mathsf{T}} = I

as well as

U^{\mathsf{T}}U = I

. That is what makes it the form to state results with — the last

m - r

columns of

U

span the left null space and would otherwise be gone.

Trimmed to what multiplies2

Thin SVD§ 5

m > n

— keep

n

columns of

U

shapes

U_n: m \times n

,

\Sigma_n: n \times n

,

V: n \times n

what is keptthe first

n

columns of

U

what is droppedthe last

m - n

columns of

U

used forleast squares on tall matrices

The dropped columns multiply rows of

\Sigma

that are entirely zero, so removing them changes nothing about the product. Note

U_n

is no longer square:

U_n^{\mathsf{T}}U_n = I_n

still holds, but

U_nU_n^{\mathsf{T}}

does not.

Compact SVD§ 5keep only the

r

nonzero singular values

shapes

U_r: m \times r

,

\Sigma_r: r \times r

,

V_r: n \times r

what is keptthe rank-

r

content only

what is droppedevery zero singular value and its vectors

used forstorage, rank-deficient matrices

The most economical exact form:

\Sigma_r

is square with a strictly positive diagonal, so it is invertible — which is what makes the pseudoinverse computable as

V_r\Sigma_r^{-1}U_r^{\mathsf{T}}

.

Nothing is approximated anywhere in this table. The full form carries orthonormal bases for all four fundamental subspaces, which is why it is the one to state theorems with; the compact form carries only what contributes to

A

, which is why it is the one to store. Approximation begins only when the compact form is truncated below rank

r

.

Three forms, one factorization·/linear-algebra/decompositions/svdLearn Math Class

The distinction to keep hold of is that none of these is an approximation. Each drops columns that multiply against zero singular values, so the product is unchanged and

A

is reconstructed exactly in all three. What is lost is not accuracy but the orthogonality of

U

as a square matrix —

U_r^{\mathsf{T}}U_r = I_r

still holds, but

U_rU_r^{\mathsf{T}}

no longer does, and any argument relying on the left null space needs the full form.

SVD and the Four Fundamental Subspaces

The SVD simultaneously provides orthonormal bases for all four fundamental subspaces of

A

:

SVD Four Fundamental Subspaces

\begin{aligned} \text{Col}(A) &= \text{Span}\{\mathbf{u}_1, \ldots, \mathbf{u}_r\} \\ \text{Null}(A^T) &= \text{Span}\{\mathbf{u}_{r+1}, \ldots, \mathbf{u}_m\} \\ \text{Row}(A) &= \text{Span}\{\mathbf{v}_1, \ldots, \mathbf{v}_r\} \\ \text{Null}(A) &= \text{Span}\{\mathbf{v}_{r+1}, \ldots, \mathbf{v}_n\} \end{aligned}

Svd four fundamental subspaces →

The first

r

columns of

V

(

\mathbf{v}_1, \dots, \mathbf{v}_r

) form an orthonormal basis for the row space of

A

.

The last

n - r

columns of

V

(

\mathbf{v}_{r+1}, \dots, \mathbf{v}_n

) form an orthonormal basis for the null space of

A

.

The first

r

columns of

U

(

\mathbf{u}_1, \dots, \mathbf{u}_r

) form an orthonormal basis for the column space of

A

.

The last

m - r

columns of

U

(

\mathbf{u}_{r+1}, \dots, \mathbf{u}_m

) form an orthonormal basis for the left null space of

A

.

The SVD provides all four bases simultaneously, all orthonormal, and pairs them so that

A\mathbf{v}_i = \sigma_i\mathbf{u}_i

. The SVD is the complete structural portrait of any matrix.

Fundamental subspace	Basis from	Column indices	Dimension
Row space of A	V	1, ..., r	r
Null space of A	V	r+1, ..., n	n − r
Column space of A	U	1, ..., r	r
Left null space of A	U	r+1, ..., m	m − r

The Pseudoinverse

The Moore-Penrose pseudoinverse

A^+

is computed directly from the SVD:

Moore-Penrose Pseudoinverse

A^+ = V\Sigma^+ U^T

Moore penrose pseudoinverse →

The matrix

\Sigma^+

is formed by reciprocating each nonzero singular value and transposing the shape: if

\Sigma

is

m \times n

with diagonal entries

\sigma_1, \dots, \sigma_r, 0, \dots, 0

, then

\Sigma^+

is

n \times m

with diagonal entries

1/\sigma_1, \dots, 1/\sigma_r, 0, \dots, 0

.

The pseudoinverse satisfies four defining properties:

AA^+A = A

,

A^+AA^+ = A^+

,

(AA^+)^T = AA^+

,

(A^+A)^T = A^+A

.

THEOREM

For a full-rank overdetermined system (

m > n

, rank

= n

),

A^+\mathbf{b}

gives the least-squares solution. For a rank-deficient system,

A^+\mathbf{b}

gives the minimum-norm least-squares solution — the solution of smallest length among all minimizers of

\|A\mathbf{x} - \mathbf{b}\|

.

Pitfall: Keeping U and V in place

Writing

U\Sigma^+V^T

instead of

V\Sigma^+U^T

: for

A = \begin{pmatrix} 0 & 2 \\ 0 & 0 \end{pmatrix}

, with

\mathbf{u}_1 = \mathbf{e}_1

,

\mathbf{v}_1 = \mathbf{e}_2

and

\sigma_1 = 2

, it gives

\begin{pmatrix} 0 & 1/2 \\ 0 & 0 \end{pmatrix}

, and then

AA^+A = O

. The pseudoinverse is

\begin{pmatrix} 0 & 0 \\ 1/2 & 0 \end{pmatrix}

.

Assembling

A^+

takes three steps: reciprocate the nonzero singular values, transpose the shape of

\Sigma

to get

\Sigma^+

, and multiply by the two orthogonal factors. When

A

is square and invertible, the result is the ordinary inverse

A^{-1} = V\Sigma^{-1}U^T

. In every case the result can be tested against the four defining properties above.

The pseudoinverse inverts the matrix on every direction it keeps and sends the directions it cannot reach to zero.

Low-Rank Approximation

The best rank-

k

approximation to

A

in either the operator norm or the Frobenius norm is obtained by truncating the SVD at

k

terms:

Eckart-Young Low-Rank Approximation

A_k = \sum_{i=1}^{k} \sigma_i \mathbf{u}_i \mathbf{v}_i^T, \qquad \|A - A_k\|_2 = \sigma_{k+1}, \quad \|A - A_k\|_F = \sqrt{\sum_{i=k+1}^{r}\sigma_i^2}

Eckart young low rank approximation →

This is the Eckart-Young-Mirsky theorem.

THEOREMEckart–Young–Mirsky theorem

Among all matrices of rank at most

k

,

A_k

is the closest to

A

.

Fact: Published in a psychology journal

Eckart and Young proved in 1936 that truncating the SVD gives the closest lower-rank matrix, in the first volume of Psychometrika. Psychologists needed it for factor analysis: explaining a large table of test scores by a few underlying abilities.

The approximation error equals

\sigma_{k+1}

(the first discarded singular value) in the operator norm, and

\sqrt{\sigma_{k+1}^2 + \cdots + \sigma_r^2}

in the Frobenius norm.

When the singular values decay rapidly —

\sigma_1 \gg \sigma_2 \gg \cdots

— a small number of terms captures most of the matrix. This is the basis of image compression (store

k

singular value triples instead of

mn

entries), noise reduction (discard small singular values as noise), latent semantic analysis (retain the top-

k

"concepts" in a document-term matrix), and dimensionality reduction more broadly.

How fast the singular values fall decides how few numbers can stand in for the whole matrix.

SVD and Norms

The singular values provide the complete "size profile" of a matrix.

The operator (spectral) norm is the largest singular value:

Operator Norm

\|A\|_2 = \sigma_1 = \max_{\|\mathbf{x}\|=1} \|A\mathbf{x}\|

Operator norm →

It measures the maximum factor by which

A

can stretch a unit vector.

The Frobenius norm is the root-sum-of-squares of all singular values:

Frobenius Norm via Singular Values

\|A\|_F = \sqrt{\sum_{i=1}^{r} \sigma_i^2}

Frobenius norm via singular values →

It measures the total "energy" in the matrix.

The condition number quantifies sensitivity to perturbation:

Condition Number

\kappa(A) = \frac{\sigma_1}{\sigma_n}

Condition number →

A matrix with

\kappa = 10^k

loses roughly

k

digits of accuracy in solving

A\mathbf{x} = \mathbf{b}

with floating-point arithmetic. A perfectly conditioned matrix (

\kappa = 1

) is a nonzero multiple of an orthogonal matrix. A singular square matrix (

\sigma_n = 0

) has

\kappa = \infty

.

The singular values are the natural measuring tool for matrices, just as eigenvalues are the natural measuring tool for symmetric matrices and linear operators. For non-symmetric matrices, singular values (not eigenvalues) govern norms and conditioning.

Quantity	Formula via singular values	Interpretation
Operator (spectral) norm ‖A‖₂	σ₁	maximum stretching factor on the unit ball
Frobenius norm ‖A‖_F	√(σ₁² + σ₂² + ··· + σ_r²)	total "energy" — root-sum-of-squares of singular values
Condition number κ(A)	σ₁ / σ_r	sensitivity to perturbation: κ = 10^k loses ~k digits

SVD and the Spectral Decomposition

COROLLARY

For a symmetric positive semi-definite matrix

A

with eigenvalues

\lambda_1 \geq \cdots \geq \lambda_n \geq 0

, the spectral decomposition

A = QDQ^T

is also the SVD:

U = V = Q

and

\Sigma = D

.

The singular values are the eigenvalues.

For a general symmetric matrix with some negative eigenvalues, the singular values are

|\lambda_i|

. The signs are absorbed into

U

or

V

: if

\lambda_i < 0

, one of the corresponding singular vectors is negated so that

\sigma_i = |\lambda_i| > 0

.

Pitfall: Using the eigenvalues of A

Reading the singular values of

A = \begin{pmatrix} 1 & 2 \\ 0 & 1 \end{pmatrix}

off its eigenvalues

1, 1

gives

\|A\|_2 = 1

and

\kappa(A) = 1

. From

A^TA = \begin{pmatrix} 1 & 2 \\ 2 & 5 \end{pmatrix}

,

\sigma_{1,2} = \sqrt{2} \pm 1

, so

\|A\|_2 = 1 + \sqrt{2}

and

\kappa(A) = 3 + 2\sqrt{2}

.

For non-symmetric or rectangular matrices, the eigendecomposition does not apply (it requires square matrices and may not exist even then), but the SVD always does. The SVD is the correct generalization of the spectral decomposition to the broadest possible class of matrices.

DownloadPNG2×PDF

Before you read

How far the SVD reaches

The spectral decomposition needs a symmetric matrix; the SVD works for every matrix. Moving outward, the two decompositions part ways.

S₊⊂S⊂M_n⊂M_{m×n}

M_{m×n}

All matrices

adds rectangular shapes · only the SVD applies

M_n

Square

eigendecomposition may not exist

S

Symmetric

adds negative eigenvalues ·

\sigma_i = |\lambda_i|

, signs absorbed into

U

or

V

S₊

Symmetric positive semi-definite

spectral decomposition and SVD coincide:

U = V = Q

,

\Sigma = D

Takeaway

The broadest possible class

Where eigendecomposition may fail or not apply, the SVD always does: it covers non-symmetric and rectangular matrices too.

How far the SVD reaches·/linear-algebra/decompositions/svdLearn Math Class

The Outer Product Form

The SVD can be written as a sum of rank-one matrices:

SVD Outer Product Form

A = \sum_{i=1}^{r} \sigma_i \, \mathbf{u}_i \mathbf{v}_i^T

Svd outer product form →

Fact: A million dollars for a low-rank guess

The Netflix Prize data held about 100 million ratings from 480,189 users of 17,770 films, so under 1.2% of the table was filled. Fitting the known ratings with a short sum of rank-one terms, called SVD by the contestants, was central to the entry that won the one-million-dollar prize in 2009.

Each term

\sigma_i \mathbf{u}_i\mathbf{v}_i^T

is an

m \times n

rank-one matrix. The singular value

\sigma_i

weights its contribution. The terms are ordered by importance: the first term captures the most of

A

(in the norm sense), the second captures the most of the remainder, and so on.

Truncating this sum at

k

terms gives the best rank-

k

approximation

A_k

. The fraction of the squared Frobenius norm captured by the first

k

terms is

(\sigma_1^2 + \cdots + \sigma_k^2)/(\sigma_1^2 + \cdots + \sigma_r^2)

.

This outer product perspective is the basis of nearly every matrix approximation method: keep the large singular values (signal) and discard the small ones (noise or redundancy).

A as a sum of rank-one pieces

The same matrix written term by term: A = √45 u₁v₁^T + √5 u₂v₂^T. Each term is a rank-one matrix — a column times a row — weighted by a singular value, largest first. Keeping only the first term gives the best rank-one approximation of A, which is why the order of the singular values matters. Build the terms on the SVD visualizer.

Truncating this sum after k terms is exactly the low-rank approximation described above.

Spectral Norm and Condition Number

The singular values answer two numerical questions that the eigenvalues cannot: how much can

A

stretch a vector at most, and how sensitive is

A\mathbf{x} = \mathbf{b}

to errors in the data.

The spectral norm is the worst-case stretch factor:

Spectral Norm and Condition Number

\|A\|_2 = \max_{\mathbf{x} \neq \mathbf{0}} \frac{\|A\mathbf{x}\|}{\|\mathbf{x}\|} = \sigma_1

Write

\mathbf{x}

in the orthonormal basis

\mathbf{v}_1, \dots, \mathbf{v}_n

of right singular vectors;

A

scales the

i

-th coefficient by

\sigma_i

and hands it to

\mathbf{u}_i

, so the ratio is largest when

\mathbf{x} = \mathbf{v}_1

and equals

\sigma_1

. Geometrically,

\sigma_1

is the longest semi-axis of the ellipse that the unit circle is mapped to. This differs from the Frobenius norm

\sqrt{\sigma_1^2 + \cdots + \sigma_r^2}

, which adds up every stretch rather than reporting the largest.

The condition number is the ratio of the largest stretch to the smallest:

Spectral Norm and Condition Number

\kappa(A) = \frac{\sigma_1}{\sigma_n} = \|A\|_2\,\|A^{-1}\|_2

for an invertible square matrix, since the largest singular value of

A^{-1}

is

1 / \sigma_n

. It bounds how a relative error in

\mathbf{b}

propagates to the solution of

A\mathbf{x} = \mathbf{b}

:

Spectral Norm and Condition Number

\frac{\|\delta\mathbf{x}\|}{\|\mathbf{x}\|} \leq \kappa(A)\,\frac{\|\delta\mathbf{b}\|}{\|\mathbf{b}\|}

A condition number near

1

means errors pass through unamplified, and orthogonal matrices achieve exactly

1

. A condition number of

10^k

means roughly

k

decimal digits of accuracy can be lost in solving the system. An infinite condition number means

\sigma_n = 0

: the matrix is singular.

The determinant says nothing about conditioning. The diagonal matrix with entries

10^{6}

and

10^{-6}

has

\det = 1

and

\kappa = 10^{12}

. Ill-conditioning is about the ratio of the stretches, not their product, and only the singular values expose it.

The SVD tool reports both numbers for any

2 \times 2

matrix alongside the ellipse whose axes they measure.

What the SVD Reveals

No other single factorization provides as much structural information about a matrix.

The rank: the number of nonzero singular values.

The four fundamental subspaces: orthonormal bases from the columns of

U

and

V

.

The pseudoinverse:

A^+ = V\Sigma^+ U^T

.

The best rank-

k

approximation: truncate at

k

terms.

Norms and the condition number: directly from the singular values.

The geometry of the linear map: rotation or reflection, scaling, rotation or reflection.

For symmetric positive semi-definite matrices, the SVD reduces to the spectral decomposition. For invertible square matrices, the singular values reveal the conditioning that the determinant alone cannot see (a matrix with

\det = 1

can still be poorly conditioned). For rectangular matrices, the SVD is one of the few factorizations that apply without modification.

The SVD is the culmination of the decomposition hierarchy — the most general, most informative, and most broadly applicable factorization in linear algebra.

The six structural quantities the SVD exposes — rank, the four fundamental subspaces, the pseudoinverse, the best rank-

k

approximation, norms and condition number, and the geometric decomposition — collect into a single reference card below.

Grouping these by which factor supplies the answer is what makes them one computation rather than seven. Three come from the singular values alone, two from the columns of the orthogonal factors, and two require reading all three factors together. Nothing below needs the matrix again once the factorization is in hand.

DownloadPNG2×PDF

Decompositions · SVD

What the SVD reveals

Every entry comes from the same three factors. What changes is which one is being read — the singular values for size, the columns of $U$ and $V$ for structure, and all three together for geometry.

7readings

From the singular values3

1

Rank§ 3

strictly positive only

r =

count of

\sigma_i > 0

The most numerically reliable rank there is. Row reduction decides rank by comparing entries against zero, which floating point makes arbitrary; the singular values instead show a gap, and where that gap falls is a judgement the numbers themselves support.

2

Norms§ 9

read straight off the list

\|A\|_2 = \sigma_1, \quad \|A\|_F^2 = \textstyle\sum \sigma_i^2

The largest singular value is the most any unit vector is stretched. The Frobenius norm is the whole list in quadrature — so both common matrix norms are functions of the same numbers.

3

Condition number§ 9

A

of full rank

\kappa(A) = \sigma_1 / \sigma_r

The ratio of most to least stretched. A large

\kappa

means the matrix is nearly singular in some direction, and small changes to

\mathbf{b}

produce large changes to the solution — which no determinant reports, since a matrix can have

\det = 1

and be badly conditioned.

From the columns of U and V2

4

The four subspaces§ 6

orthonormal bases, all four at once

partition

U

and

V

at index

r

First

r

columns of

V

span the row space and the rest the null space; first

r

of

U

span the column space and the rest the left null space. It gives all four orthonormally from one computation, with

A\mathbf{v}_i = \sigma_i\mathbf{u}_i

pairing them.

5

Pseudoinverse§ 7

reciprocate the nonzero

\sigma

, leave zeros alone

A^{+} = V\Sigma^{+}U^{\mathsf{T}}

Defined for every matrix, square or not, invertible or not. When

A

is invertible it coincides with

A^{-1}

; otherwise it returns the least-squares solution of minimum norm, which is the sense in which it is the closest thing to an inverse.

From all three factors2

6

Low-rank approximation§ 8

optimal for every unitarily invariant norm

A_k = \sum_{i=1}^{k} \sigma_i \mathbf{u}_i \mathbf{v}_i^{\mathsf{T}}

Truncating the outer-product sum gives the best rank-

k

approximation there is — the Eckart–Young theorem, and it is why the SVD underlies image compression and principal component analysis. The error in the operator norm is exactly

\sigma_{k+1}

, so the singular values say in advance how much is lost.

7

Geometric action§ 2

every matrix, no hypotheses

A = U\Sigma V^{\mathsf{T}}

— rotate, stretch, rotate

Read right to left:

V^{\mathsf{T}}

rotates,

\Sigma

scales along axes,

U

rotates again. Every linear map is those three steps, which is the claim that makes the factorization worth having as geometry rather than only as algebra.

No other factorization that exists for every matrix answers this many questions. That combination is why the SVD is the one to reach for when the matrix is rectangular, rank-deficient, or simply unknown — the conditions many of the other decompositions require are exactly the ones it does without.

What the SVD reveals·/linear-algebra/decompositions/svdLearn Math Class

Two of these deserve emphasis because nothing else supplies them. The condition number measures how badly a system amplifies error, and it is invisible to the determinant — a matrix can have determinant one and still be nearly singular in some direction. Low-rank approximation is optimal rather than merely reasonable: truncating the sum at

k

terms gives the closest rank-

k

matrix in every unitarily invariant norm, with error exactly

\sigma_{k+1}

in the operator norm.

The reason all of this comes from one factorization is that the SVD asks nothing of the matrix. It need not be square, invertible, symmetric or full rank — the factorization exists regardless, which is what separates it from every other decomposition in this section and why it is the one to reach for when the matrix is unknown.

Common Pitfalls

Four mistakes come up again and again when an SVD is worked out by hand. Each is marked with a box where it happens in the sections above; here each is worked through in full, with the wrong and the right working side by side.

Forgetting the square root

The eigenvalues of

A^TA

are found correctly and then used directly as the singular values. The square root that turns them into stretch factors is left out.

Wrong working: for

A = \begin{pmatrix} 1 & 0 \\ 0 & 1 \\ 1 & 1 \end{pmatrix}

from the worked example,

A^TA = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}, \quad \lambda_1 = 3, \; \lambda_2 = 1 \quad \Longrightarrow \quad \sigma_1 = 3, \; \sigma_2 = 1

Right working: each singular value is the square root of an eigenvalue.

\sigma_1 = \sqrt{3}, \quad \sigma_2 = \sqrt{1} = 1

Why it happens: the whole computation runs through the eigenvalue problem for

A^TA

, and its eigenvalues look like the final answer. When an eigenvalue is

0

or

1

the square root changes nothing, which hides the step.

How to check: the squares of the singular values must add up to the sum of the squared entries of

A

, which is

\|A\|_F^2

. Here the entries give

1 + 1 + 1 + 1 = 4

, and

(\sqrt{3})^2 + 1^2 = 4

, while

3^2 + 1^2 = 10

. A second test:

\|A\mathbf{v}_1\|

must equal

\sigma_1

, and

A\mathbf{v}_1 = \frac{1}{\sqrt{2}}(1, 1, 2)^T

has length

\sqrt{3}

.

Choosing the left vectors separately

The right singular vectors come from

A^TA

, and the left ones are taken as unit eigenvectors of

AA^T

, found independently. Each eigenvector is fixed only up to sign, so the two choices need not match, and

U\Sigma V^T

no longer equals

A

.

Wrong working: with

\mathbf{v}_1 = \frac{1}{\sqrt{2}}(1, 1)^T

and

\mathbf{v}_2 = \frac{1}{\sqrt{2}}(1, -1)^T

from the worked example, the unit eigenvector of

AA^T

for

\lambda = 3

is taken as

\mathbf{u}_1 = -\frac{1}{\sqrt{6}}(1, 1, 2)^T

, with

\mathbf{u}_2 = \frac{1}{\sqrt{2}}(1, -1, 0)^T

. Since

\mathbf{u}_3

meets a zero row of

\Sigma

,

U\Sigma V^T = \sigma_1\mathbf{u}_1\mathbf{v}_1^T + \sigma_2\mathbf{u}_2\mathbf{v}_2^T = -\frac{1}{2}\begin{pmatrix} 1 & 1 \\ 1 & 1 \\ 2 & 2 \end{pmatrix} + \frac{1}{2}\begin{pmatrix} 1 & -1 \\ -1 & 1 \\ 0 & 0 \end{pmatrix} = \begin{pmatrix} 0 & -1 \\ -1 & 0 \\ -1 & -1 \end{pmatrix}

Right working: compute

\mathbf{u}_1

from

\mathbf{v}_1

.

\mathbf{u}_1 = \frac{1}{\sqrt{3}}A\mathbf{v}_1 = \frac{1}{\sqrt{3}} \cdot \frac{1}{\sqrt{2}}(1, 1, 2)^T = \frac{1}{\sqrt{6}}(1, 1, 2)^T

U\Sigma V^T = \frac{1}{2}\begin{pmatrix} 1 & 1 \\ 1 & 1 \\ 2 & 2 \end{pmatrix} + \frac{1}{2}\begin{pmatrix} 1 & -1 \\ -1 & 1 \\ 0 & 0 \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ 0 & 1 \\ 1 & 1 \end{pmatrix} = A

Why it happens: the columns of

U

and

V

are eigenvectors of

AA^T

and

A^TA

, so it looks natural to compute both the same way. That description fixes each column only up to sign, and it says nothing about the pairing

A\mathbf{v}_i = \sigma_i\mathbf{u}_i

that holds the factorization together.

How to check: for each nonzero

\sigma_i

, compare

A\mathbf{v}_i

with

\sigma_i\mathbf{u}_i

. Here

A\mathbf{v}_1 = \frac{1}{\sqrt{2}}(1, 1, 2)^T

, while the wrong choice gives

\sigma_1\mathbf{u}_1 = -\frac{1}{\sqrt{2}}(1, 1, 2)^T

.

Keeping U and V in place

To build the pseudoinverse,

\Sigma

is replaced by

\Sigma^+

but the outer factors are left where they were, giving

U\Sigma^+V^T

. The order has to reverse, as it does for the inverse of any product.

Wrong working:

A = \begin{pmatrix} 0 & 2 \\ 0 & 0 \end{pmatrix}

sends

\mathbf{e}_2

to

2\mathbf{e}_1

and

\mathbf{e}_1

to

\mathbf{0}

, so

\sigma_1 = 2

,

\mathbf{v}_1 = \mathbf{e}_2

,

\mathbf{u}_1 = \mathbf{e}_1

, and

\sigma_2 = 0

,

\mathbf{v}_2 = \mathbf{e}_1

,

\mathbf{u}_2 = \mathbf{e}_2

. Then

U = I

,

V = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}

,

\Sigma^+ = \begin{pmatrix} 1/2 & 0 \\ 0 & 0 \end{pmatrix}

, and

U\Sigma^+V^T = \begin{pmatrix} 1/2 & 0 \\ 0 & 0 \end{pmatrix}\begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix} = \begin{pmatrix} 0 & 1/2 \\ 0 & 0 \end{pmatrix}

Right working:

V

goes on the left and

U^T

on the right.

A^+ = V\Sigma^+U^T = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}\begin{pmatrix} 1/2 & 0 \\ 0 & 0 \end{pmatrix} = \begin{pmatrix} 0 & 0 \\ 1/2 & 0 \end{pmatrix}

Why it happens:

\Sigma^+

takes the place of

\Sigma

in the middle, and it is tempting to change nothing else. For a symmetric positive semi-definite matrix

U = V

, so the slip gives the right answer there and goes unnoticed.

How to check: test the first defining property,

AA^+A = A

. With the correct

A^+

,

AA^+ = \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}

and

AA^+A = A

. With the wrong one,

A \begin{pmatrix} 0 & 1/2 \\ 0 & 0 \end{pmatrix} = O

, so

AA^+A = O

. For a rectangular

A

the sizes alone catch it:

A^+

must be

n \times m

, and

U\Sigma^+V^T

is not even defined when

m \neq n

.

Using the eigenvalues of A

For a square matrix, the singular values are read off the eigenvalues of

A

itself, and the norm and the condition number are computed from them. This works for symmetric matrices, after taking absolute values, but not in general.

Wrong working:

A = \begin{pmatrix} 1 & 2 \\ 0 & 1 \end{pmatrix}

is triangular, with eigenvalues

1

and

1

. Taking these as the singular values gives

\|A\|_2 = 1, \qquad \kappa(A) = \frac{1}{1} = 1

Right working: the singular values come from

A^TA

.

A^TA = \begin{pmatrix} 1 & 2 \\ 2 & 5 \end{pmatrix}, \quad \lambda^2 - 6\lambda + 1 = 0, \quad \lambda = 3 \pm 2\sqrt{2}

\sigma_1 = \sqrt{3 + 2\sqrt{2}} = 1 + \sqrt{2}, \qquad \sigma_2 = \sqrt{3 - 2\sqrt{2}} = \sqrt{2} - 1

\|A\|_2 = 1 + \sqrt{2} \approx 2.414, \qquad \kappa(A) = \frac{\sqrt{2} + 1}{\sqrt{2} - 1} = 3 + 2\sqrt{2} \approx 5.83

Why it happens: for symmetric matrices the singular values are the absolute values of the eigenvalues, as the section on the spectral decomposition shows, and the shortcut is carried over to matrices that are not symmetric.

How to check: one vector is enough.

A\mathbf{e}_2 = (2, 1)^T

has length

\sqrt{5} \approx 2.24

, more than

1

, so

\|A\|_2

cannot be

1

. The squares of the singular values must also add up to the squared entries,

1 + 4 + 0 + 1 = 6

; the eigenvalues give

1 + 1 = 2

, while

(1 + \sqrt{2})^2 + (\sqrt{2} - 1)^2 = 6

.

Singular Value Decomposition FAQ

Is the $U$ in the SVD the same kind of matrix as the $U$ in $LU$ ?

+

No, and the collision is worth watching. In

LU

the letter means upper triangular; in the SVD it means orthogonal. The same character signs two different contracts a couple of pages apart, and only the name of the decomposition says which is in force. Nothing about the symbol itself distinguishes them.Read more →

Is the $\Sigma$ in $U\Sigma V^{T}$ a summation sign?

+

No, it is a matrix. A summation sign carries bounds above and below and is followed by a summand, none of which appear here. A bare capital sigma sitting between two matrix letters is a factor in a product, and in the SVD it is the rectangular diagonal matrix holding the singular values with zero padding.Read more →

What is the difference between $\sigma_i$ and $\sigma(A)$ ?

+

The subscript changes the species entirely. A subscripted

\sigma_i

is one singular value, a single non-negative number, while a bare

\sigma(A)

is the spectrum, meaning the set of eigenvalues. Since singular values and eigenvalues agree only for special matrices, the two notations rarely even hold the same numbers.Read more →

Can you take $U$ from $AA^T$ and $V$ from $A^TA$ separately?

+

Not safely, because each eigenvector is fixed at best up to sign, and the two choices need not match. The factorization depends on the pairing

A\mathbf{v}_i = \sigma_i\mathbf{u}_i

, and an independently chosen

\mathbf{u}_i

can be its negative, giving a product that is not

A

. The reliable route takes

V

from

A^TA

and sets

\mathbf{u}_i = A\mathbf{v}_i / \sigma_i

for each nonzero singular value; only the remaining columns of

U

are free.Read more →

Can a matrix with determinant 1 be ill-conditioned?

+

Yes. Conditioning depends on the ratio of the largest singular value to the smallest, while the determinant, up to sign, is their product, and a product can stay at

1

while the ratio explodes. The diagonal matrix with entries

10^{6}

and

10^{-6}

has determinant

1

yet condition number

10^{12}

, enough to wipe out most of the digits a computer carries. Only the singular values show this; the determinant hides it.Read more →

Can the SVD be used to compute powers of a matrix?

+

Not in general. Squaring gives

A^2 = U\Sigma V^TU\Sigma V^T

, and the middle product

V^TU

is not the identity, because the input and output bases differ, so nothing cancels. Diagonalization answers questions about powers and long-run behaviour; the SVD answers questions about stretching, rank and approximation. The clean exception is a symmetric positive semi-definite matrix, where

U = V

and

A^k = U\Sigma^kU^T

.Read more →

Singular Value Decomposition

The Universal Matrix Factorization

What the Singular Value Decomposition Is

Diagonalization and Its Requirement

Two Bases Instead of One

What Follows and What Does Not

The Geometric Interpretation

SVD Notation

SVD Notation

Singular Values

Computing the SVD

Compact and Thin Forms

Three forms, one factorization

SVD and the Four Fundamental Subspaces

The Pseudoinverse

Low-Rank Approximation

SVD and Norms

SVD and the Spectral Decomposition

The Outer Product Form

Spectral Norm and Condition Number

What the SVD Reveals

What the SVD reveals

Common Pitfalls

Singular Value Decomposition FAQ

Is the UUU in the SVD the same kind of matrix as the UUU in LULULU?

Is the Σ\SigmaΣ in UΣVTU\Sigma V^{T}UΣVT a summation sign?

What is the difference between σi\sigma_iσi​ and σ(A)\sigma(A)σ(A)?

Can you take UUU from AATAA^TAAT and VVV from ATAA^TAATA separately?

Can a matrix with determinant 1 be ill-conditioned?

Can the SVD be used to compute powers of a matrix?

Is the $U$ in the SVD the same kind of matrix as the $U$ in $LU$ ?

Is the $\Sigma$ in $U\Sigma V^{T}$ a summation sign?

What is the difference between $\sigma_i$ and $\sigma(A)$ ?

Can you take $U$ from $AA^T$ and $V$ from $A^TA$ separately?