Visual Tools
Calculators
Tables
Mathematical Keyboard
Converters
Other Tools

How to Find the Singular Value Decomposition

Every matrix factors as A=UΣVTA = U\Sigma V^T with UU and VV orthogonal and Σ\Sigma diagonal, holding the singular values σ1≥σ2≥⋯≥0\sigma_1 \geq \sigma_2 \geq \cdots \geq 0. Get VV and the σi\sigma_i from the symmetric matrix ATAA^TA, then build UU from them.

Recognize the task

Any matrix, square or not, and the words SVD, singular values, pseudoinverse, low-rank approximation, condition number or 2-norm. Forms: (3045)\begin{pmatrix} 3 & 0 \\ 4 & 5 \end{pmatrix}, (32223−2)\begin{pmatrix} 3 & 2 & 2 \\ 2 & 3 & -2 \end{pmatrix}. Test: for a wide matrix, AATAA^T is the smaller product; start from it and swap the roles of UU and VV.
The SVD is defined in the SVD lesson; the singular values are the square roots of the eigenvalues of ATAA^TA.

Two ways to do it

Start from the columnstall or square
ATA vi=σi2vi,ui=1σiAviA^TA\,\mathbf{v}_i = \sigma_i^2\mathbf{v}_i, \quad \mathbf{u}_i = \frac{1}{\sigma_i}A\mathbf{v}_i
Eigenvectors of ATAA^TA give VV; each ui\mathbf{u}_i follows from its vi\mathbf{v}_i.
Start from the rowswide
AAT ui=σi2ui,vi=1σiATuiAA^T\,\mathbf{u}_i = \sigma_i^2\mathbf{u}_i, \quad \mathbf{v}_i = \frac{1}{\sigma_i}A^T\mathbf{u}_i
For a wide AA the matrix AATAA^T is smaller; get UU first, then each vi\mathbf{v}_i.

Steps

StepOn A=(3045)A = \begin{pmatrix} 3 & 0 \\ 4 & 5 \end{pmatrix}
1Form the product of the transpose with A. ATAA^TA is symmetric and its eigenvalues are never negative.ATA=(25202025)A^TA = \begin{pmatrix} 25 & 20 \\ 20 & 25 \end{pmatrix}
2Find its eigenvalues and take square roots. Order them from largest to smallest; the square roots are the singular values.λ=45,5\lambda = 45, 5, so σ1=45=35\sigma_1 = \sqrt{45} = 3\sqrt{5}, σ2=5\sigma_2 = \sqrt{5}
3Find orthonormal eigenvectors for V. This is the spectral decomposition of ATAA^TA. Keep the same order as the σi\sigma_i.v1=12(1,1)\mathbf{v}_1 = \frac{1}{\sqrt{2}}(1, 1), v2=12(1,−1)\mathbf{v}_2 = \frac{1}{\sqrt{2}}(1, -1)
4Compute each column of U from its partner in V. For each nonzero σi\sigma_i, ui=1σiAvi\mathbf{u}_i = \frac{1}{\sigma_i}A\mathbf{v}_i. If UU needs more columns, extend to an orthonormal basis.u1=135⋅12(3,9)=110(1,3)\mathbf{u}_1 = \frac{1}{3\sqrt{5}} \cdot \frac{1}{\sqrt{2}}(3, 9) = \frac{1}{\sqrt{10}}(1, 3), u2=110(3,−1)\mathbf{u}_2 = \frac{1}{\sqrt{10}}(3, -1)
5Assemble and check. UΣVTU\Sigma V^T must give back AA, and the squared singular values must add up to the sum of the squared entries (Frobenius norm).U=110(133−1)U = \frac{1}{\sqrt{10}}\begin{pmatrix} 1 & 3 \\ 3 & -1 \end{pmatrix}, Σ=(35005)\Sigma = \begin{pmatrix} 3\sqrt{5} & 0 \\ 0 & \sqrt{5} \end{pmatrix}, 45+5=9+16+2545 + 5 = 9 + 16 + 25 ✓

Worked examples

Example 1 · a rank-one matrix

one singular value
A=(1111)A = \begin{pmatrix} 1 & 1 \\ 1 & 1 \end{pmatrix}: ATA=(2222)A^TA = \begin{pmatrix} 2 & 2 \\ 2 & 2 \end{pmatrix}, λ=4,0\lambda = 4, 0
trace 44, determinant 00
σ1=2\sigma_1 = 2, v1=12(1,1)\mathbf{v}_1 = \frac{1}{\sqrt{2}}(1, 1), u1=12Av1=12(1,1)\mathbf{u}_1 = \frac{1}{2}A\mathbf{v}_1 = \frac{1}{\sqrt{2}}(1, 1)
σ2=0\sigma_2 = 0 adds nothing
A=2 u1v1TA = 2\,\mathbf{u}_1\mathbf{v}_1^T, rank 11
one nonzero singular value

Example 2 · a wide matrix

rows first
A=(32223−2)A = \begin{pmatrix} 3 & 2 & 2 \\ 2 & 3 & -2 \end{pmatrix}: AAT=(178817)AA^T = \begin{pmatrix} 17 & 8 \\ 8 & 17 \end{pmatrix}, λ=25,9\lambda = 25, 9
2×22 \times 2 instead of 3×33 \times 3
σ1=5\sigma_1 = 5, σ2=3\sigma_2 = 3; u1=12(1,1)\mathbf{u}_1 = \frac{1}{\sqrt{2}}(1, 1), u2=12(1,−1)\mathbf{u}_2 = \frac{1}{\sqrt{2}}(1, -1)
eigenvectors of AATAA^T
v1=15ATu1=12(1,1,0)\mathbf{v}_1 = \frac{1}{5}A^T\mathbf{u}_1 = \frac{1}{\sqrt{2}}(1, 1, 0), v2=13ATu2=132(1,−1,4)\mathbf{v}_2 = \frac{1}{3}A^T\mathbf{u}_2 = \frac{1}{3\sqrt{2}}(1, -1, 4)
the roles of UU and VV swap
v3=13(2,−2,−1)\mathbf{v}_3 = \frac{1}{3}(2, -2, -1), Σ=(500030)\Sigma = \begin{pmatrix} 5 & 0 & 0 \\ 0 & 3 & 0 \end{pmatrix}
unit vector orthogonal to v1,v2\mathbf{v}_1, \mathbf{v}_2

Example 3 · best rank-one approximation

truncate
for the matrix in the steps, keep only σ1u1v1T\sigma_1\mathbf{u}_1\mathbf{v}_1^T
drop the smallest singular value
A1=35⋅110(13)12(11)A_1 = 3\sqrt{5} \cdot \frac{1}{\sqrt{10}}\begin{pmatrix} 1 \\ 3 \end{pmatrix} \frac{1}{\sqrt{2}}\begin{pmatrix} 1 & 1 \end{pmatrix}
3520=32\frac{3\sqrt{5}}{\sqrt{20}} = \frac{3}{2}
A1=(1.51.54.54.5)A_1 = \begin{pmatrix} 1.5 & 1.5 \\ 4.5 & 4.5 \end{pmatrix}, error ∥A−A1∥2=σ2=5\|A - A_1\|_2 = \sigma_2 = \sqrt{5}
no rank-one matrix is closer
Truncating the SVD is the low-rank approximation behind image compression.

See it happen

Every matrix maps the unit circle to an ellipse. VTV^T turns the circle so that the right singular vectors line up with the axes, Σ\Sigma stretches the axes by σ1\sigma_1 and σ2\sigma_2, and UU turns the result into place. The singular values are the semi-axes of the ellipse. Watch the circle deform in the SVD visualizer.

Common mistakes

Forgetting the square roots

✗ σ1=45\sigma_1 = 45, σ2=5\sigma_2 = 5
✓ σi=λi\sigma_i = \sqrt{\lambda_i}: 353\sqrt{5} and 5\sqrt{5}

Choosing the left vectors separately

✗ taking unit eigenvectors of AATAA^T for UU without looking at VV
✓ ui=1σiAvi\mathbf{u}_i = \frac{1}{\sigma_i}A\mathbf{v}_i; a separate choice can flip a sign and break UΣVT=AU\Sigma V^T = A

Confusing singular values with eigenvalues

✗ the singular values of (3045)\begin{pmatrix} 3 & 0 \\ 4 & 5 \end{pmatrix} are 33 and 55
✓ those are its eigenvalues; the singular values are 353\sqrt{5} and 5\sqrt{5}
They agree only for symmetric matrices with no negative eigenvalues.

Dropping the transpose on V

✗ A=UΣVA = U\Sigma V
✓ A=UΣVTA = U\Sigma V^T