Linear Algebra Terms and Definitions

AAdjugateDeterminants AAlgebraic MultiplicityEigen AAugmented MatrixLinear Systems BBack-SubstitutionLinear Systems BBasisVector Spaces CChange of Basis MatrixTransformations CCharacteristic PolynomialEigen CCholesky FactorizationOrthogonality CCoefficient MatrixLinear Systems CCofactorDeterminants CCofactor ExpansionDeterminants CCofactor Matrix (Adjugate)Determinants CColumn SpaceVector Spaces CCondition NumberMatrices CConformabilityMatrices CConsistent SystemLinear Systems CCoordinatesVector Spaces CCramer's RuleDeterminants CCross ProductVectors DDefective MatrixEigen DDesign MatrixLinear Systems DDeterminantDeterminants DDiagonal MatrixMatrices DDiagonalizable MatrixEigen DDiagonalizationEigen DDimensionVector Spaces DDirectionVectors DDominant EigenvalueEigen DDot ProductVectors EEigenspaceEigen EEigenvalueEigen EEigenvectorEigen EElement-wise OperationMatrices EElementary Row OperationLinear Systems FForward SubstitutionLinear Systems FFree VariableVector Spaces FFrobenius Inner ProductVectors FFrobenius NormVectors FFull RankVector Spaces GGaussian EliminationLinear Systems GGeometric MultiplicityEigen GGram-Schmidt ProcessOrthogonality HHadamard ProductVectors HHomogeneous SystemLinear Systems IIdempotent MatrixMatrices IIdentity MatrixMatrices IImage (Range)Transformations IInner ProductOrthogonality IInner Product SpaceVectors IInverse MatrixMatrices IInvolutory MatrixMatrices IIsometryVectors LLeast-Squares SolutionLinear Systems LLeft Null SpaceVector Spaces LLinear CombinationVectors LLinear IndependenceVector Spaces LLinear TransformationTransformations LLU DecompositionLinear Systems MMagnitude (Norm)Vectors MMain DiagonalMatrices MMatrixMatrices MMatrix RepresentationTransformations MMatrix Square RootMatrices MMinorDeterminants NNilpotent MatrixMatrices NNull Space (Kernel)Vector Spaces NNullityVector Spaces OOrientationDeterminants OOrthogonal ComplementOrthogonality OOrthogonal DecompositionVectors OOrthogonal MatrixOrthogonality OOrthogonal SetOrthogonality OOrthogonal VectorsOrthogonality OOrthonormal SetOrthogonality OOuter ProductVectors OOverdetermined SystemLinear Systems PPartial PivotingLinear Systems PPermutation MatrixMatrices PPivotLinear Systems PPivot ColumnVector Spaces PPositive Definite MatrixMatrices PProjection MatrixTransformations PPseudoinverseLinear Systems QQR DecompositionOrthogonality QQuadratic FormEigen RRankMatrices RRank-Nullity TheoremVector Spaces RRayleigh QuotientEigen RReduced Row Echelon FormLinear Systems RReflectionTransformations RResidualLinear Systems RRight-Hand RuleVectors RRotationTransformations RRow Echelon FormLinear Systems RRow SpaceVector Spaces SSarrus's RuleDeterminants SScalarVectors SScalar MatrixMatrices SSimilar MatricesTransformations SSingular MatrixMatrices SSingular ValueEigen SSingular Value DecompositionEigen SSkew-Symmetric MatrixMatrices SSpanVector Spaces SSpecial SolutionsVector Spaces SSpectral DecompositionEigen SSpectral NormMatrices SSquare MatrixMatrices SSubspaceVector Spaces SSymmetric MatrixMatrices SSystem of Linear EquationsLinear Systems TTraceMatrices TTransposeMatrices TTriangular MatrixMatrices UUnit VectorVectors VVectorVectors VVector ProjectionVectors VVector SpaceVector Spaces ZZero MatrixMatrices ZZero VectorVectors

Determinants(9)

Eigen(15)

Linear Systems(19)

Matrices(25)

Orthogonality(9)

Transformations(8)

Vector Spaces(17)

Vectors(18)

120 of 120 terms

Determinants

Determinant Minor Cofactor Cofactor Matrix (Adjugate)Cofactor Expansion Sarrus's Rule Adjugate Cramer's Rule Orientation

Eigen

Eigenvalue Eigenvector Eigenspace Characteristic Polynomial Algebraic Multiplicity Geometric Multiplicity Singular Value Diagonalizable Matrix Diagonalization Defective Matrix Dominant Eigenvalue Rayleigh Quotient Quadratic Form Spectral Decomposition Singular Value Decomposition

Linear Systems

System of Linear Equations Augmented Matrix Row Echelon Form Reduced Row Echelon Form Pivot Homogeneous System Coefficient Matrix Elementary Row Operation Gaussian Elimination Back-Substitution Forward Substitution Partial Pivoting Consistent System Overdetermined System LU Decomposition Least-Squares Solution Residual Design Matrix Pseudoinverse

Matrices

Matrix Square Matrix Identity Matrix Symmetric Matrix Inverse Matrix Singular Matrix Rank Trace Diagonal Matrix Positive Definite Matrix Transpose Main Diagonal Zero Matrix Scalar Matrix Skew-Symmetric Matrix Triangular Matrix Permutation Matrix Nilpotent Matrix Idempotent Matrix Involutory Matrix Conformability Element-wise Operation Matrix Square Root Spectral Norm Condition Number

Orthogonality

Inner Product Orthogonal Vectors Orthogonal Set Orthonormal Set Orthogonal Complement Orthogonal Matrix Gram-Schmidt Process QR Decomposition Cholesky Factorization

Transformations

Linear Transformation Image (Range)Matrix Representation Change of Basis Matrix Similar Matrices Rotation Reflection Projection Matrix

Vector Spaces

Vector Space Subspace Span Linear Independence Basis Dimension Column Space Null Space (Kernel)Row Space Left Null Space Coordinates Pivot Column Free Variable Special Solutions Nullity Rank-Nullity Theorem Full Rank

Vectors

Vector Scalar Magnitude (Norm)Unit Vector Dot Product Cross Product Linear Combination Zero Vector Direction Vector Projection Orthogonal Decomposition Outer Product Hadamard Product Frobenius Inner Product Frobenius Norm Inner Product Space Right-Hand Rule Isometry

120 terms

Vectors

(18 items)

Vector

An ordered list of

n

real numbers:

\mathbf{v} = (v_1, v_2, \ldots, v_n) \in \mathbb{R}^n

Vectors

See details

↑ Back to top

intuitionnotationrelated conceptsrelated formulas

A quantity with both magnitude and direction. In

\mathbb{R}^2

and

\mathbb{R}^3

, vectors can be visualized as arrows from the origin to a point.

↑ Back to top

Scalar

An element of the underlying field — in standard linear algebra, a real number

c \in \mathbb{R}

Vectors

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

A single number used to scale vectors. Multiplying a vector by a scalar changes its length without altering its direction (unless the scalar is negative, which reverses direction).

↑ Back to top

Magnitude (Norm)

The length of a vector, measured as its distance from the origin:

\|\mathbf{v}\| = \sqrt{v_1^2 + v_2^2 + \cdots + v_n^2}

Vector Magnitude

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

\mathbb{R}^2

, this reduces to the hypotenuse given by the Pythagorean theorem. The concept generalizes to any

\mathbb{R}^n

↑ Back to top

Unit Vector

A vector

\hat{\mathbf{u}}

with

\|\hat{\mathbf{u}}\| = 1

Vector Magnitude

See details

↑ Back to top

intuitionnotationrelated conceptsrelated formulas

A vector that encodes direction only, with all length information removed. Any nonzero vector

\mathbf{v}

can be normalized to a unit vector by dividing by its magnitude:

\hat{\mathbf{v}} = \frac{\mathbf{v}}{\|\mathbf{v}\|}

↑ Back to top

Dot Product

An operation that takes two vectors and returns a scalar, computed by summing the products of corresponding components:

\mathbf{u} \cdot \mathbf{v} = u_1 v_1 + u_2 v_2 + \cdots + u_n v_n = \|\mathbf{u}\|\,\|\mathbf{v}\|\cos\theta

Dot Product

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

A scalar measure of how much two vectors point in the same direction. Positive when the angle between them is acute, zero when perpendicular, negative when obtuse.

↑ Back to top

Cross Product

A binary operation on two vectors in

\mathbb{R}^3

that produces a vector perpendicular to both inputs:

\mathbf{u} \times \mathbf{v} = \begin{pmatrix} u_2 v_3 - u_3 v_2 \\ u_3 v_1 - u_1 v_3 \\ u_1 v_2 - u_2 v_1 \end{pmatrix}

Cross Product

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

Its magnitude equals the area of the parallelogram spanned by the two vectors. The direction follows the right-hand rule.

↑ Back to top

Linear Combination

A sum of vectors, each multiplied by a scalar coefficient:

c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \cdots + c_k\mathbf{v}_k

Linear Combinations

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

A new vector built by scaling given vectors and adding the results. The set of all possible linear combinations of a collection of vectors defines their span.

↑ Back to top

Zero Vector

The vector

\mathbf{0} = (0, 0, \ldots, 0)

whose every component is zero

Types of Vectors

See details

↑ Back to top

intuitionpropertiesrelated concepts

The "nothing" vector: no length and no direction. It is the additive identity, so

\mathbf{v} + \mathbf{0} = \mathbf{v}

for every

\mathbf{v}

, and it is the one vector that cannot be normalized.

↑ Back to top

Direction

The orientation of a non-zero vector, captured by its unit vector

\mathbf{v} / \|\mathbf{v}\|

Direction

See details

↑ Back to top

intuitionrelated concepts

What is left of a vector once its length is stripped away. Two vectors share a direction exactly when one is a positive multiple of the other; a negative multiple reverses it.

↑ Back to top

Vector Projection

The component of

\mathbf{u}

lying along

\mathbf{v}

\text{proj}_{\mathbf{v}}\mathbf{u} = \frac{\mathbf{u} \cdot \mathbf{v}}{\mathbf{v} \cdot \mathbf{v}}\,\mathbf{v}

Orthogonal Projection

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

The shadow

\mathbf{u}

casts onto the line through

\mathbf{v}

. It is always a multiple of

\mathbf{v}

, never of

\mathbf{u}

— the formula hides that asymmetry and the picture does not. The signed length of the shadow,

(\mathbf{u} \cdot \mathbf{v}) / \|\mathbf{v}\|

, is the scalar projection.

↑ Back to top

Orthogonal Decomposition

The unique splitting of a vector into a part inside a subspace

W

and a part in its orthogonal complement:

\mathbf{u} = \text{proj}_W\mathbf{u} + (\mathbf{u} - \text{proj}_W\mathbf{u})

The Orthogonal Decomposition

See details

↑ Back to top

intuitionrelated concepts

Every vector is "some of

W

plus something perpendicular to

W

", and there is only one way to do the split. The perpendicular part is the error a projection leaves behind, which is why least squares rests on this decomposition.

↑ Back to top

Outer Product

The matrix

\mathbf{u}\mathbf{v}^T

with entries

(\mathbf{u}\mathbf{v}^T)_{ij} = u_i v_j

Two More Products: Hadamard and Outer

See details

↑ Back to top

intuitionpropertiesrelated concepts

Where the dot product collapses two vectors into a number, the outer product spreads them into a table. Every column is a multiple of

\mathbf{u}

and every row a multiple of

\mathbf{v}^T

, so the result has rank one. Also called the dyadic or tensor product.

↑ Back to top

Hadamard Product

The entry-wise product of two same-shaped matrices:

(A \circ B)_{ij} = a_{ij}\,b_{ij}

Two More Products: Hadamard and Outer

See details

↑ Back to top

intuitionrelated concepts

Multiply each entry by the entry in the same position and nothing else. Unlike ordinary matrix multiplication it is commutative, needs identical shapes, and never mixes positions. Also called the Schur product.

↑ Back to top

Frobenius Inner Product

The inner product of two same-shaped matrices obtained by treating them as long vectors:

\langle A, B \rangle_F = \sum_{i,j} a_{ij}\,b_{ij} = \text{tr}(A^T B)

The Frobenius Inner Product

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

Flatten both matrices into vectors and take the ordinary dot product. The trace form

\text{tr}(A^T B)

is the same number written without flattening, which is what makes it useful in proofs.

↑ Back to top

Frobenius Norm

The length of a matrix measured as if its entries were one long vector:

\|A\|_F = \sqrt{\sum_{i,j} a_{ij}^2} = \sqrt{\text{tr}(A^T A)}

The Frobenius Inner Product

See details

↑ Back to top

intuitionrelated concepts

The most literal way to say how "big" a matrix is: square every entry, add, take the root. It equals the square root of the sum of the squared singular values, which ties it to the geometry of the matrix rather than only its entries.

↑ Back to top

Inner Product Space

A vector space equipped with an inner product

\langle \cdot, \cdot \rangle

that is symmetric, linear in each argument, and positive definite

General Inner Products

See details

↑ Back to top

intuitionrelated concepts

A vector space in which length, distance, angle and perpendicularity all make sense, because a single pairing supplies them:

\|\mathbf{v}\| = \sqrt{\langle \mathbf{v}, \mathbf{v} \rangle}

and orthogonality means

\langle \mathbf{u}, \mathbf{v} \rangle = 0

\mathbb{R}^n

with the dot product is the standard example.

↑ Back to top

Right-Hand Rule

The convention fixing the direction of a cross product

\mathbf{u} \times \mathbf{v}

Direction and the Right-Hand Rule

See details

↑ Back to top

intuitionrelated concepts

Curl the fingers of your right hand from

\mathbf{u}

toward

\mathbf{v}

; your thumb points along

\mathbf{u} \times \mathbf{v}

. Swapping the two vectors curls the other way, which is why the cross product changes sign when its arguments are swapped.

↑ Back to top

Isometry

A linear transformation that preserves lengths:

\|T\mathbf{v}\| = \|\mathbf{v}\|

for every

\mathbf{v}

Isometries: Maps That Preserve Length

See details

↑ Back to top

intuitionrelated concepts

A rigid motion of space about the origin: nothing is stretched, squashed or sheared, so distances and angles survive too. In

\mathbb{R}^n

the isometries are exactly the orthogonal matrices — rotations and reflections.

↑ Back to top

Vector Spaces

(17 items)

Vector Space

A set

V

equipped with vector addition and scalar multiplication satisfying the vector space axioms

Vector Spaces

See details

↑ Back to top

intuitionexamplesrelated conceptsrelated formulas

A collection of objects (vectors) that can be added together and scaled by numbers, where these operations behave predictably.

\mathbb{R}^n

is the most familiar example, but the concept extends to function spaces, polynomial spaces, and more.

↑ Back to top

Subspace

A nonempty subset

W \subseteq V

that is itself a vector space under the same operations

Subspaces

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

A subset of a vector space that is closed under addition and scalar multiplication. Every subspace must contain the zero vector.

↑ Back to top

Span

The set of all linear combinations of a given collection of vectors:

\text{Span}\{\mathbf{v}_1, \ldots, \mathbf{v}_k\} = \{c_1\mathbf{v}_1 + \cdots + c_k\mathbf{v}_k \mid c_i \in \mathbb{R}\}

Span

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

Geometrically, spanning two non-parallel vectors in

\mathbb{R}^3

gives a plane; spanning three linearly independent vectors fills the entire space.

↑ Back to top

Linear Independence

Vectors

\mathbf{v}_1, \ldots, \mathbf{v}_k

are linearly independent if the only solution to

c_1\mathbf{v}_1 + \cdots + c_k\mathbf{v}_k = \mathbf{0}

c_1 = c_2 = \cdots = c_k = 0

Linear Independence

See details

↑ Back to top

intuitioncommon errorsrelated conceptsrelated formulas

No vector in the set can be written as a linear combination of the others. Each vector contributes a genuinely new direction. Removing any one reduces the span.

↑ Back to top

Basis

A set

\{\mathbf{v}_1, \ldots, \mathbf{v}_n\}

that is linearly independent and spans the entire vector space

Vector Spaces

See details

↑ Back to top

intuitionexamplesrelated conceptsrelated formulas

A minimal set of vectors that can produce every vector in the space through linear combinations. Every vector has a unique representation as a linear combination of basis vectors.

↑ Back to top

Dimension

The number of vectors in any basis of a vector space

V

, denoted

\dim(V)

Dimension

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

The number of independent directions in a space.

\mathbb{R}^3

is three-dimensional because any basis has exactly three vectors.

↑ Back to top

Column Space

The set of all vectors expressible as

A\mathbf{x}

— equivalently, the span of the columns of

A

\text{Col}(A) = \{A\mathbf{x} \mid \mathbf{x} \in \mathbb{R}^n\}

Fundamental Spaces

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

The set of all vectors

\mathbf{b}

for which

A\mathbf{x} = \mathbf{b}

has a solution. It captures everything the matrix can "reach" as output.

↑ Back to top

Null Space (Kernel)

The set of all solutions to the homogeneous system

A\mathbf{x} = \mathbf{0}

\text{Nul}(A) = \{\mathbf{x} \in \mathbb{R}^n \mid A\mathbf{x} = \mathbf{0}\}

Fundamental Spaces

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

The set of all inputs that the matrix sends to the zero vector. If the null space contains only

\mathbf{0}

, the matrix is injective (one-to-one). A larger null space means the matrix "collapses" some directions.

↑ Back to top

Row Space

The span of the rows of a matrix, equivalently the column space of its transpose:

\text{Row}(A) = \text{Col}(A^T)

Fundamental Spaces

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

A subspace of

\mathbb{R}^n

spanned by the rows of

A

. Row operations change individual rows but preserve the row space, making echelon form useful for finding a basis.

↑ Back to top

Left Null Space

The null space of the transpose

A^T

— the set of all vectors

\mathbf{y}

satisfying

A^T\mathbf{y} = \mathbf{0}

\text{Nul}(A^T) = \{\mathbf{y} \in \mathbb{R}^m \mid A^T\mathbf{y} = \mathbf{0}\}

Fundamental Spaces

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

It completes the four fundamental subspaces: column space, null space, row space, and left null space.

↑ Back to top

Coordinates

The unique scalars

c_1, \ldots, c_n

expressing a vector in a chosen basis:

\mathbf{v} = c_1\mathbf{b}_1 + c_2\mathbf{b}_2 + \cdots + c_n\mathbf{b}_n

Coordinates

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

An address for a vector, read against a particular ruler. Change the basis and the same vector gets different numbers — the coordinates belong to the basis, not to the vector. Uniqueness is what makes them an address rather than one description among many.

↑ Back to top

Pivot Column

A column of a matrix that contains a pivot after row reduction

Pivot Columns and Free Columns

See details

↑ Back to top

intuitioncommon errorsrelated concepts

The columns that carry genuinely new directions. Their variables are the leading variables of the system, and the pivot columns of the original matrix — not of the reduced one — form a basis for the column space.

↑ Back to top

Free Variable

A variable whose column carries no pivot after row reduction

Pivot Columns and Free Columns

See details

↑ Back to top

intuitionrelated concepts

A variable the equations do not pin down: it may take any value, and the leading variables then follow. Each free variable adds one parameter to the solution set and one dimension to the null space.

↑ Back to top

Special Solutions

The solutions of

A\mathbf{x} = \mathbf{0}

obtained by setting one free variable to

1

and the others to

0

Finding the Null Space

See details

↑ Back to top

intuitionrelated concepts

One solution per free variable, each switching on a single free variable and solving for the rest. They are independent by construction and every solution of the homogeneous system is a combination of them, so together they form a basis for the null space.

↑ Back to top

Nullity

The dimension of the null space:

\text{nullity}(A) = n - \text{rank}(A)

The Rank-Nullity Theorem

See details

↑ Back to top

intuitionrelated concepts

How many independent directions the matrix crushes to zero — equivalently, how many free variables the system has. Rank counts what survives, nullity counts what is lost, and the two always add up to the number of columns.

↑ Back to top

Rank-Nullity Theorem

For an

m \times n

matrix:

\text{rank}(A) + \text{nullity}(A) = n

The Rank-Nullity Theorem as a Dimension Statement

See details

↑ Back to top

intuitionrelated concepts

Bookkeeping rather than a surprise: every column is either a pivot column, contributing a dimension to the column space, or a free column, contributing a dimension to the null space. No column is both and none is left out.

↑ Back to top

Full Rank

A matrix whose rank is as large as its shape allows:

\text{rank}(A) = \min(m, n)

Properties of Rank

See details

↑ Back to top

intuitionrelated concepts

Nothing is wasted. A square matrix of full rank is invertible; a tall one has independent columns and a wide one has independent rows. Losing full rank is the same event as the columns becoming dependent.

↑ Back to top

Matrices

(25 items)

Matrix

A rectangular array of numbers with

m

rows and

n

columns:

A \in \mathbb{R}^{m \times n}

Matrices

See details

↑ Back to top

intuitionnotationrelated conceptsrelated formulas

A compact way to encode a linear transformation or a system of linear equations. The entry in row

i

, column

j

is denoted

a_{ij}

↑ Back to top

Square Matrix

A matrix with equal numbers of rows and columns:

A \in \mathbb{R}^{n \times n}

Matrix Types

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

Only square matrices can have determinants, eigenvalues, inverses, and a trace. They represent transformations from a space to itself.

↑ Back to top

Identity Matrix

The square matrix with

1

s on the main diagonal and

0

s elsewhere, denoted

I_n

I_n = \begin{pmatrix} 1 & 0 & \cdots & 0 \\ 0 & 1 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 1 \end{pmatrix}

Matrix Types

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

The matrix that leaves every vector unchanged:

I\mathbf{v} = \mathbf{v}

. It is the multiplicative identity for matrices, analogous to the number

1

↑ Back to top

Symmetric Matrix

A square matrix satisfying

A = A^T

Matrix Types

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

A matrix that equals its own transpose — entries are mirrored across the main diagonal. Symmetric matrices arise naturally in distance, covariance, and quadratic form problems.

↑ Back to top

Inverse Matrix

A square matrix

A^{-1}

such that

AA^{-1} = A^{-1}A = I

Matrix Inverse

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

The matrix that "undoes" the transformation applied by

A

. If

A

maps

\mathbf{x}

\mathbf{b}

, then

A^{-1}

maps

\mathbf{b}

back to

\mathbf{x}

↑ Back to top

Singular Matrix

A square matrix

A

with

\det(A) = 0

Determinants

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

A matrix that collapses at least one dimension — it maps some nonzero vector to

\mathbf{0}

. A singular matrix has no inverse and the system

A\mathbf{x} = \mathbf{b}

either has no solution or infinitely many.

↑ Back to top

Rank

The number of linearly independent columns (equivalently, rows) in a matrix:

\text{rank}(A) = \dim(\text{Col}(A)) = \dim(\text{Row}(A))

Matrix Rank

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

It measures the "effective dimensionality" of the transformation — how many independent output directions the matrix produces.

↑ Back to top

Trace

The sum of the main diagonal entries of a square matrix:

\text{tr}(A) = a_{11} + a_{22} + \cdots + a_{nn} = \sum_{i=1}^{n} a_{ii}

Matrix Trace

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

It equals the sum of all eigenvalues (counted with multiplicity), providing a quick invariant of the matrix.

↑ Back to top

Diagonal Matrix

A square matrix where

a_{ij} = 0

for all

i \neq j

Matrix Types

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

A matrix whose only nonzero entries lie on the main diagonal. Diagonal matrices scale each coordinate axis independently, making them the simplest matrices to work with.

↑ Back to top

Positive Definite Matrix

A symmetric matrix

A

satisfying

\mathbf{x}^T A \mathbf{x} > 0

for all nonzero

\mathbf{x}

Cholesky Decomposition

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

A matrix where the associated quadratic form is always positive — it curves upward in every direction, like a bowl. Positive definite matrices generalize the idea of a positive number to matrix algebra.

↑ Back to top

Transpose

The matrix

A^T

obtained by swapping rows and columns:

(A^T)_{ij} = a_{ji}

The Transpose

See details

↑ Back to top

intuitionpropertiesrelated concepts

A reflection of the matrix across its main diagonal. The diagonal itself does not move, which is why transposing twice returns the original and why a matrix equal to its own transpose is symmetric about that line.

↑ Back to top

Main Diagonal

The entries

a_{11}, a_{22}, a_{33}, \ldots

whose row index equals their column index

Square Matrices

See details

↑ Back to top

intuitionrelated concepts

The line running from the top-left corner downward. Everything else is off-diagonal, either above it or below it, and a great many matrix types are defined by what sits where relative to this line: diagonal, triangular, symmetric.

↑ Back to top

Zero Matrix

The matrix

0

with every entry equal to zero

Matrix Equality and the Zero Matrix

See details

↑ Back to top

intuitionrelated concepts

The additive identity for matrices of its shape:

A + 0 = A

. Multiplying by it wipes everything out, so its determinant, rank and every eigenvalue are all

0

↑ Back to top

Scalar Matrix

A diagonal matrix whose diagonal entries are all the same number:

cI

Diagonal Matrices

See details

↑ Back to top

intuitionrelated concepts

A matrix that does nothing but scale: multiplying by

cI

stretches every vector by

c

without turning it. The identity matrix is the case

c = 1

↑ Back to top

Skew-Symmetric Matrix

A square matrix equal to the negative of its transpose:

A^T = -A

Skew-Symmetric Matrices

See details

↑ Back to top

intuitionpropertiesrelated concepts

Mirrored entries carry opposite signs,

a_{ij} = -a_{ji}

. The diagonal is forced to be zero, since each diagonal entry must equal its own negative.

↑ Back to top

Triangular Matrix

A square matrix with zeros on one side of the main diagonal

Triangular Matrices

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

Upper triangular when everything below the diagonal is zero, lower triangular when everything above is, and unit triangular when the diagonal is all ones. Elimination is the business of turning an arbitrary matrix into this shape, because a triangular system can be solved by substitution alone.

↑ Back to top

Permutation Matrix

A square matrix with exactly one entry equal to

1

in each row and each column, and

0

everywhere else — the identity matrix with its rows rearranged:

P = \begin{pmatrix} 0 & 1 & 0 \\ 0 & 0 & 1 \\ 1 & 0 & 0 \end{pmatrix}

Permutation Matrices

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

It moves entries without changing them.

PA

reorders the rows of

A

and

AP

reorders the columns, but nothing is scaled and nothing is combined. There are

n!

permutation matrices of order

n

, one for each permutation of the indices. It is orthogonal, so its inverse is its transpose, and it is how the row swaps of partial pivoting get recorded in

PA = LU

↑ Back to top

Nilpotent Matrix

A square matrix some power of which is the zero matrix:

N^k = 0

Nilpotent and Idempotent Matrices

See details

↑ Back to top

intuitionrelated concepts

A matrix that destroys everything if applied often enough. A strictly upper triangular matrix is the standard example — each multiplication pushes the non-zero band one step further from the diagonal until nothing is left. Every eigenvalue of a nilpotent matrix is

0

↑ Back to top

Idempotent Matrix

A square matrix that equals its own square:

P^2 = P

Nilpotent and Idempotent Matrices

See details

↑ Back to top

intuitionrelated concepts

Applying it twice changes nothing after the first time. Every projection matrix is idempotent — once a vector has been dropped onto a subspace, dropping it again leaves it where it is. The eigenvalues can only be

0

1

↑ Back to top

Involutory Matrix

A square matrix that is its own inverse:

A^2 = I

Involutory Matrices

See details

↑ Back to top

intuitionrelated concepts

Applying it twice returns every vector to where it started, as a reflection does. Its eigenvalues can only be

\pm 1

↑ Back to top

Conformability

The shape condition under which a matrix operation is defined

Matrix Multiplication — Definition

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

Addition and subtraction need identical shapes. The product

AB

needs the inner dimensions to agree —

A

m \times n

and

B

n \times p

— and the result takes the outer dimensions,

m \times p

. Where the requirement fails the operation simply does not exist.

↑ Back to top

Element-wise Operation

An operation applied independently at each position, so the entry at

(i, j)

of the result depends only on the entries at

(i, j)

of the inputs

Matrix Addition

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

The positions never talk to each other. Addition, subtraction, scalar multiplication and the Hadamard product are all element-wise; ordinary matrix multiplication is the important operation that is not.

↑ Back to top

Matrix Square Root

A matrix

B

with

B^2 = A

, or more loosely

BB^T = A

What Cholesky Decomposition Is

See details

↑ Back to top

intuitionrelated concepts

A positive definite matrix has exactly one positive definite square root, and its Cholesky factor

L

, with

LL^T = A

, is a lower-triangular one. Most matrices have no square root at all.

↑ Back to top

Spectral Norm

The largest factor by which a matrix can stretch any vector:

\|A\|_2 = \max_{\mathbf{x} \neq \mathbf{0}} \frac{\|A\mathbf{x}\|}{\|\mathbf{x}\|}

Spectral Norm and Condition Number

See details

↑ Back to top

intuitionrelated concepts

Feed the matrix every possible direction and record the worst-case stretch. That maximum is the largest singular value, which is why the SVD is the natural way to compute it.

↑ Back to top

Condition Number

The ratio of the largest to the smallest singular value:

\kappa(A) = \sigma_{\max} / \sigma_{\min}

Spectral Norm and Condition Number

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

How much a small change in the data of

A\mathbf{x} = \mathbf{b}

can be amplified in the answer. A condition number near

1

means a well-behaved system; a huge one means tiny rounding errors become large errors; an infinite one means the matrix is singular.

↑ Back to top

Determinants

(9 items)

Determinant

A scalar

\det(A) \in \mathbb{R}

assigned to every square matrix, defined recursively via cofactor expansion

Determinants

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

The signed volume scaling factor of the linear transformation represented by the matrix. A determinant of zero means the transformation collapses space into a lower dimension.

↑ Back to top

Minor

The determinant of the submatrix obtained by deleting row

i

and column

j

from a matrix:

M_{ij} = \det(\hat{A}_{ij})

Cofactors

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

Minors are the building blocks for cofactors and, through them, for the full determinant computation.

↑ Back to top

Cofactor

A signed minor, with sign determined by the position

(i,j)

C_{ij} = (-1)^{i+j} M_{ij}

Cofactors

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

The sign alternates in a checkerboard pattern (

+, -, +, \ldots

). Cofactors are used in the expansion formula for determinants and in computing the adjugate.

↑ Back to top

Cofactor Matrix (Adjugate)

The transpose of the matrix of cofactors of

A

\text{adj}(A) = C^T

Cofactors

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

It provides a formula for the inverse:

A^{-1} = \frac{1}{\det(A)}\,\text{adj}(A)

↑ Back to top

Cofactor Expansion

Computing a determinant by expanding along any row or column:

\det A = \sum_{j} a_{ij}\,C_{ij}

Laplace Expansion Along a Row

See details

↑ Back to top

intuitionrelated concepts

Each entry of the chosen row is multiplied by its cofactor — the signed determinant of what is left when its row and column are struck out — and the products are summed. The signs follow the checkerboard

(-1)^{i+j}

. Any row or column gives the same answer, so pick the one with the most zeros. Also called Laplace expansion.

↑ Back to top

Sarrus's Rule

A shortcut for

3 \times 3

determinants: add the three products along the down-right diagonals, subtract the three along the down-left diagonals

The 3×3 Formula

See details

↑ Back to top

intuitionrelated concepts

Copy the first two columns to the right of the matrix and trace six diagonals. It is a mnemonic, not a method — it works for

3 \times 3

only and does not extend to larger matrices, where the number of terms jumps to

24

and beyond.

↑ Back to top

Adjugate

The transpose of the cofactor matrix:

\text{adj}(A) = C^T

The Adjugate

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

The matrix that almost inverts

A

A\,\text{adj}(A) = (\det A)\,I

. Dividing by the determinant finishes the job, which puts the invertibility condition in plain view — the division fails precisely when

\det A = 0

↑ Back to top

Cramer's Rule

For a square system

A\mathbf{x} = \mathbf{b}

with

\det A \neq 0

x_i = \dfrac{\det(A_i)}{\det A}

, where

A_i

A

with its

i

-th column replaced by

\mathbf{b}

Cramer

See details

↑ Back to top

intuitionrelated concepts

Each unknown is a ratio of two determinants. It is exact and elegant for two or three unknowns and hopeless beyond that, since every unknown costs a full determinant — elimination beats it for anything you would actually solve.

↑ Back to top

Orientation

The handedness of an ordered set of vectors, read off the sign of the determinant

Orientation

See details

↑ Back to top

intuitionrelated concepts

A positive determinant keeps the standard orientation; a negative one turns space over, as a reflection does. The absolute value says how much area or volume is scaled, the sign says whether it was flipped.

↑ Back to top

Linear Systems

(19 items)

System of Linear Equations

A collection of equations

A\mathbf{x} = \mathbf{b}

where

A

is an

m \times n

matrix and

\mathbf{b} \in \mathbb{R}^m

Linear Systems

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

A set of linear equations sharing the same unknowns. The system has either no solution, exactly one solution, or infinitely many solutions — there is no other possibility.

↑ Back to top

Augmented Matrix

The matrix formed by appending the right-hand side vector

\mathbf{b}

as an additional column to the coefficient matrix

A

, written

[A \mid \mathbf{b}]

Linear Systems

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

A compact notation that combines the coefficient matrix and the right-hand side into a single matrix, so row operations can be applied to the entire system at once.

↑ Back to top

Row Echelon Form

A matrix where:
• all zero rows are at the bottom
• each leading entry (pivot) is to the right of the pivot in the row above
• all entries below each pivot are zero

Echelon Form

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

A staircase pattern achieved by row operations, making back-substitution possible. The number of pivots equals the rank.

↑ Back to top

Reduced Row Echelon Form

Row echelon form with the additional requirements:
• every pivot is

1

• each pivot is the only nonzero entry in its column

Echelon Form

See details

↑ Back to top

intuitionnotationrelated conceptsrelated formulas

The fully simplified form of a matrix under row operations. Unlike row echelon form, the reduced form is unique — every matrix has exactly one RREF.

↑ Back to top

Pivot

The first nonzero entry in each row of a matrix in row echelon form

Echelon Form

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

Pivots mark the "independent" columns. The number of pivots determines the rank; columns without pivots correspond to free variables in the solution.

↑ Back to top

Homogeneous System

A system of linear equations in which every equation equals zero:

A\mathbf{x} = \mathbf{0}

Homogeneous Systems

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

It always has at least the trivial solution

\mathbf{x} = \mathbf{0}

. Nontrivial solutions exist if and only if

\text{rank}(A) < n

↑ Back to top

Coefficient Matrix

The matrix

A

of coefficients of the unknowns when a system of linear equations is written as

A\mathbf{x} = \mathbf{b}

Writing a System in Matrix Form

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

The numbers multiplying the unknowns, one row per equation and one column per unknown. The vector

\mathbf{b}

of constants is the right-hand side, and appending it as a final column gives the augmented matrix.

↑ Back to top

Elementary Row Operation

One of three moves that leave a system's solution set unchanged: swap two rows, multiply a row by a non-zero constant, or add a multiple of one row to another

The Three Elementary Row Operations

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

Each move is reversible — the same multiple can be subtracted back, the same rows swapped again — and that reversibility is the licence for the whole of elimination: nothing about the solutions can change if every step can be undone.

↑ Back to top

Gaussian Elimination

The algorithm that reduces a matrix to row echelon form using elementary row operations

The Goal

See details

↑ Back to top

intuitionrelated concepts

Work column by column, clearing every entry below each pivot by subtracting multiples of the pivot row. What remains is a staircase that can be finished by back-substitution. Continuing to clear above the pivots as well is Gauss-Jordan elimination. Also called row reduction.

↑ Back to top

Back-Substitution

Solving an upper-triangular system from the bottom row upward

Back Substitution

See details

↑ Back to top

intuitionrelated concepts

The last equation involves one unknown, so it is solved directly. That value is substituted into the row above, which now involves one unknown, and so on up the staircase. It is the second half of Gaussian elimination.

↑ Back to top

Forward Substitution

Solving a lower-triangular system from the top row downward

Solving Systems with LU

See details

↑ Back to top

intuitionrelated concepts

The mirror image of back-substitution: the first equation has one unknown, its value feeds the second, and so on downward. It is the first of the two triangular solves when a system is attacked through its LU decomposition.

↑ Back to top

Partial Pivoting

Swapping rows during elimination so that the largest available entry in the current column becomes the pivot

Partial Pivoting

See details

↑ Back to top

intuitionrelated concepts

Dividing by a zero pivot is impossible and dividing by a tiny one wrecks accuracy, so the row with the biggest entry is moved up first. The swaps are recorded in a permutation matrix, which is why the honest form of LU is

PA = LU

↑ Back to top

Consistent System

A system of linear equations that has at least one solution

The Existence Condition

See details

↑ Back to top

intuitionrelated concepts

Exactly one solution when every column has a pivot, infinitely many when some column is free. An inconsistent system has none, and reduction announces it with a row reading

0 = c

for some

c \neq 0

— inconsistency is detected, not tested for.

↑ Back to top

Overdetermined System

A system with more equations than unknowns

Overdetermined Systems

See details

↑ Back to top

intuitionrelated concepts

Too many constraints for the unknowns to satisfy all at once, so the system is usually inconsistent. It is then solved in the least-squares sense — the answer that misses every equation by as little as possible overall — which is exactly the situation in fitting a line to data.

↑ Back to top

LU Decomposition

Writing a matrix as

A = LU

, with

L

lower triangular and

U

upper triangular; with row swaps,

PA = LU

What LU Decomposition Is

See details

↑ Back to top

intuitionrelated concepts

Gaussian elimination with the multipliers kept instead of thrown away:

U

is the result of the elimination and

L

is its record. Once factored,

A\mathbf{x} = \mathbf{b}

is two cheap triangular solves — forward then back — for as many right-hand sides as you like.

↑ Back to top

Least-Squares Solution

For an inconsistent system

A\mathbf{x} = \mathbf{b}

, the

\hat{\mathbf{x}}

that minimizes

\|\mathbf{b} - A\hat{\mathbf{x}}\|

; it satisfies the normal equations

A^TA\hat{\mathbf{x}} = A^T\mathbf{b}

The Problem

See details

↑ Back to top

intuitionrelated concepts

When

\mathbf{b}

cannot be reached, aim for the nearest point that can: its projection onto the column space. Minimizing the error and making it perpendicular to the column space turn out to be the same requirement, and that perpendicularity is what the normal equations say.

↑ Back to top

Residual

The error vector left by an approximate solution:

\mathbf{r} = \mathbf{b} - A\hat{\mathbf{x}}

The Geometric Interpretation

See details

↑ Back to top

intuitionrelated concepts

What the fit fails to explain. At the least-squares solution the residual is not merely small — it is orthogonal to the column space, which is the geometric fact everything else follows from. Its squared length is the sum of squared errors.

↑ Back to top

Design Matrix

The matrix

A

built from the data in a fitting problem: one row per observation, one column per unknown coefficient

Worked Example: Fitting a Line

See details

↑ Back to top

intuitionrelated concepts

Fitting a line

y = c_0 + c_1 x

to points

(x_i, y_i)

gives rows

(1, x_i)

; fitting a parabola adds a column of

x_i^2

. The fit is the least-squares solution of

A\mathbf{c} = \mathbf{y}

, and changing the model changes only the columns, never the method.

↑ Back to top

Pseudoinverse

The matrix

A^+

that plays the role of an inverse for a matrix that has none; for independent columns,

A^+ = (A^TA)^{-1}A^T

The Pseudoinverse

See details

↑ Back to top

intuitionrelated concepts

It sends

\mathbf{b}

to the least-squares solution of smallest length, and reduces to the ordinary inverse when one exists. In general it is built from the SVD by inverting the non-zero singular values and leaving the zero ones alone.

↑ Back to top

Transformations

(8 items)

Linear Transformation

A function

T: V \to W

between vector spaces that preserves addition and scalar multiplication:

T(\mathbf{u} + \mathbf{v}) = T(\mathbf{u}) + T(\mathbf{v})

T(c\mathbf{u}) = cT(\mathbf{u})

Linear Transformations

See details

↑ Back to top

intuitionexamplesrelated conceptsrelated formulas

Lines map to lines (or to the origin), and the origin stays fixed. Every linear transformation between finite-dimensional spaces can be represented by a matrix.

↑ Back to top

Image (Range)

The set of all output vectors of a linear transformation:

\text{Im}(T) = \{T(\mathbf{v}) \mid \mathbf{v} \in V\}

Image and Kernel

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

For a matrix transformation

T(\mathbf{x}) = A\mathbf{x}

, the image equals the column space of

A

↑ Back to top

Matrix Representation

A matrix

A

such that

T(\mathbf{v}) = A[\mathbf{v}]_{\mathcal{B}}

for a chosen basis

\mathcal{B}

Matrix Representation

See details

↑ Back to top

intuitionrelated concepts

Every linear transformation between finite-dimensional spaces can be encoded as a matrix once bases are chosen. Different bases produce different matrices representing the same transformation.

↑ Back to top

Change of Basis Matrix

A matrix

P

that converts coordinates from one basis to another:

[\mathbf{v}]_{\mathcal{B}'} = P^{-1}[\mathbf{v}]_{\mathcal{B}}

Basis Change

See details

↑ Back to top

intuitionrelated concepts

The same vector can be described using different coordinate systems (bases). The change of basis matrix translates between these coordinate systems.

↑ Back to top

Similar Matrices

Matrices

A

and

B

are similar if

B = P^{-1}AP

for some invertible matrix

P

Basis Change

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

Two matrices are similar when they represent the same linear transformation under different bases. They share all basis-independent properties.

↑ Back to top

Rotation

A linear transformation that turns every vector through the same angle about the origin; in

\mathbb{R}^2

R_\theta = \begin{bmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{bmatrix}

Rotations in R²

See details

↑ Back to top

intuitionrelated concepts

Lengths and angles survive, only headings change. The matrix is orthogonal with determinant

+1

, and a rotation by a non-trivial angle has no real eigenvectors — no direction is left pointing where it started.

↑ Back to top

Reflection

A linear transformation that mirrors every vector across a line through the origin — or a plane, in three dimensions

Reflections in R²

See details

↑ Back to top

intuitionrelated concepts

Points on the mirror stay put while everything else swaps to the far side at equal distance. The matrix is orthogonal with determinant

-1

and is its own inverse: reflecting twice restores everything.

↑ Back to top

Projection Matrix

The matrix

P

sending every vector to its orthogonal projection onto a subspace; for a subspace with basis matrix

A

P = A(A^TA)^{-1}A^T

The Projection Matrix

See details

↑ Back to top

intuitionrelated concepts

Drop every point perpendicularly onto the subspace. Points already there do not move, so applying

P

twice changes nothing after the first time — it is idempotent,

P^2 = P

, and symmetric. Its determinant is

0

whenever the subspace is smaller than the whole space, because information has been lost.

↑ Back to top

Eigen

(15 items)

Eigenvalue

A scalar

\lambda

such that

A\mathbf{v} = \lambda\mathbf{v}

for some nonzero vector

\mathbf{v}

Eigenvalues and Eigenvectors

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

The factor by which the linear transformation stretches or compresses an eigenvector. A negative eigenvalue means the eigenvector's direction is reversed.

↑ Back to top

Eigenvector

A nonzero vector

\mathbf{v}

such that

A\mathbf{v} = \lambda\mathbf{v}

for some scalar

\lambda

Eigenvalues and Eigenvectors

See details

↑ Back to top

intuitioncommon errorsrelated conceptsrelated formulas

A direction that the matrix preserves — the transformation only stretches or compresses along this direction without rotating it. Eigenvectors from distinct eigenvalues are always linearly independent.

↑ Back to top

Eigenspace

The set of all eigenvectors for a given eigenvalue

\lambda

, together with the zero vector — equivalently, the null space of

(A - \lambda I)

E_\lambda = \text{Nul}(A - \lambda I)

Eigenvalues and Eigenvectors

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

It is always a subspace, and its dimension reveals how many independent eigenvector directions exist for that eigenvalue.

↑ Back to top

Characteristic Polynomial

The polynomial whose roots are the eigenvalues of

A

, obtained by computing:

p(\lambda) = \det(A - \lambda I)

Characteristic Equation

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

For an

n \times n

matrix, the characteristic polynomial has degree

n

, so there are at most

n

eigenvalues (counted with multiplicity).

↑ Back to top

Algebraic Multiplicity

The multiplicity of

\lambda

as a root of the characteristic polynomial

Characteristic Equation

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

How many times the eigenvalue appears as a root. If

p(\lambda) = (\lambda - 2)^3(\lambda + 1)

, then

\lambda = 2

has algebraic multiplicity

3

and

\lambda = -1

has algebraic multiplicity

1

↑ Back to top

Geometric Multiplicity

The dimension of the eigenspace associated with an eigenvalue

\lambda

\text{geo. mult.}(\lambda) = \dim(E_\lambda) = \dim(\text{Nul}(A - \lambda I))

Eigenvalue Properties

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

The number of independent eigenvector directions for a given eigenvalue. A matrix is diagonalizable if and only if every eigenvalue has geometric multiplicity equal to its algebraic multiplicity.

↑ Back to top

Singular Value

A nonnegative scalar measuring how much a matrix stretches space along each principal direction, derived from the eigenvalues of

A^TA

\sigma_i = \sqrt{\lambda_i(A^TA)}

SVD

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

Unlike eigenvalues, singular values are always nonnegative and exist for any matrix, not just square ones.

↑ Back to top

Diagonalizable Matrix

A square matrix with a full set of

n

linearly independent eigenvectors — equivalently, one that can be written as

A = PDP^{-1}

with

D

diagonal

When Is a Matrix Diagonalizable?

See details

↑ Back to top

intuitionrelated concepts

A matrix that is secretly just a stretch along some set of directions. Every matrix with

n

distinct eigenvalues qualifies, and every symmetric matrix does; what disqualifies a matrix is never awkward eigenvalues, only running out of independent eigenvectors.

↑ Back to top

Diagonalization

Factoring a matrix as

A = PDP^{-1}

, where the columns of

P

are eigenvectors and

D

carries the matching eigenvalues on its diagonal

What Diagonalization Means

See details

↑ Back to top

intuitioncommon errorsrelated concepts

A change to the basis in which the matrix acts as a plain stretch. The payoff is powers:

A^k = PD^kP^{-1}

, and raising a diagonal matrix to a power is entrywise, so repeated multiplication becomes a handful of scalar powers.

↑ Back to top

Defective Matrix

A matrix with too few independent eigenvectors to be diagonalized: for some eigenvalue the geometric multiplicity is smaller than the algebraic multiplicity

When Diagonalization Fails

See details

↑ Back to top

intuitionrelated concepts

A repeated eigenvalue that failed to bring a full set of directions with it. There are not enough independent eigenvectors to fill the columns of

P

, so

P

cannot be inverted and the factorization never forms. The Jordan form exists to give these matrices a canonical shape instead.

↑ Back to top

Dominant Eigenvalue

The eigenvalue of largest absolute value

The Dominant Eigenvalue and Power Iteration

See details

↑ Back to top

intuitionrelated concepts

Multiply a vector by the matrix again and again and it swings toward the dominant eigenvector, because that component grows fastest. The rate is set by the ratio

|\lambda_2 / \lambda_1|

— close to

1

means slow convergence — and this is exactly what power iteration exploits.

↑ Back to top

Rayleigh Quotient

The scalar

R(\mathbf{x}) = \frac{\mathbf{x}^T A\,\mathbf{x}}{\mathbf{x}^T \mathbf{x}}

The Dominant Eigenvalue and Power Iteration

See details

↑ Back to top

intuitionrelated concepts

The best estimate of an eigenvalue from a vector that is only approximately an eigenvector. It equals the eigenvalue exactly when

\mathbf{x}

is an eigenvector, and for a symmetric matrix it is always trapped between the smallest and largest eigenvalues.

↑ Back to top

Quadratic Form

A function

q(\mathbf{x}) = \mathbf{x}^T A\,\mathbf{x}

built from a symmetric matrix

Quadratic Forms

See details

↑ Back to top

intuitionrelated concepts

Every term is a product of two coordinates —

x_1^2

x_1 x_2

and so on — with the matrix holding the coefficients. Its sign behaviour is read off the eigenvalues of

A

: all positive means positive definite, all negative means negative definite, mixed means indefinite.

↑ Back to top

Spectral Decomposition

The diagonalization of a symmetric matrix by an orthogonal matrix:

A = Q\Lambda Q^T = \sum_{i} \lambda_i\,\mathbf{q}_i \mathbf{q}_i^T

What the Spectral Decomposition Is

See details

↑ Back to top

intuitionrelated concepts

Because the eigenvectors of a symmetric matrix can be chosen orthonormal, the matrix of them is orthogonal and its inverse is just its transpose — no inverse ever has to be computed. The sum form shows the matrix as rank-one pieces weighted by eigenvalues, and dropping the smallest weights is how it becomes an approximation.

↑ Back to top

Singular Value Decomposition

The factorization

A = U\Sigma V^T

with

U

and

V

orthogonal and

\Sigma

a rectangular diagonal matrix of singular values in decreasing order

What the Singular Value Decomposition Is

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

The one factorization every matrix has — square or not, symmetric or not, with or without eigenvectors. The columns of

V

are the right singular vectors, the columns of

U

the left ones, and

\Sigma

says how much each direction is stretched. Truncating it keeps the most important part first.

↑ Back to top

Orthogonality

(9 items)

Inner Product

A function

\langle \cdot, \cdot \rangle: V \times V \to \mathbb{R}

satisfying symmetry, linearity, and positive-definiteness

Inner Product

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

A generalization of the dot product to abstract vector spaces. It defines notions of length, angle, and orthogonality in spaces where the standard dot product may not apply.

↑ Back to top

Orthogonal Vectors

Vectors

\mathbf{u}

and

\mathbf{v}

are orthogonal if

\langle \mathbf{u}, \mathbf{v} \rangle = 0

Orthogonality

See details

↑ Back to top

intuitionnotationrelated conceptsrelated formulas

Perpendicular vectors. Their dot product (or inner product) is zero, meaning they share no component in each other's direction.

↑ Back to top

Orthogonal Set

A set of vectors

\{\mathbf{v}_1, \ldots, \mathbf{v}_k\}

where

\langle \mathbf{v}_i, \mathbf{v}_j \rangle = 0

for all

i \neq j

Orthogonal Sets

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

A collection of mutually perpendicular vectors. Every orthogonal set of nonzero vectors is automatically linearly independent.

↑ Back to top

Orthonormal Set

An orthogonal set where every vector is a unit vector:

\langle \mathbf{v}_i, \mathbf{v}_j \rangle = \delta_{ij}

Orthogonal Sets

See details

↑ Back to top

intuitionnotationrelated conceptsrelated formulas

Mutually perpendicular vectors, each of length

1

. An orthonormal basis gives the simplest coordinate system — coordinates are just inner products with the basis vectors.

↑ Back to top

Orthogonal Complement

The set of all vectors in

V

that are orthogonal to every vector in a subspace

W

W^\perp = \{\mathbf{v} \in V \mid \langle \mathbf{v}, \mathbf{w} \rangle = 0 \text{ for all } \mathbf{w} \in W\}

Orthogonality

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

Together,

W

and

W^\perp

span the entire space, with no overlap except the zero vector.

↑ Back to top

Orthogonal Matrix

A square matrix

Q

satisfying

Q^TQ = QQ^T = I

Matrix Types

See details

↑ Back to top

intuitionpropertiesrelated conceptsrelated formulas

A matrix whose columns (and rows) form an orthonormal set. Orthogonal matrices preserve lengths and angles — they represent rotations and reflections.

↑ Back to top

Gram-Schmidt Process

The procedure that turns any linearly independent set into an orthonormal set spanning the same space

The Goal

See details

↑ Back to top

intuitioncommon errorsrelated concepts

Take each vector in turn, subtract its projections onto the directions already fixed, and normalize what is left. The subtraction guarantees orthogonality by construction; the normalization only fixes the length, which is why the two concerns can be separated.

↑ Back to top

QR Decomposition

Writing a matrix with independent columns as

A = QR

, with

Q

orthogonal and

R

upper triangular

What QR Decomposition Is

See details

↑ Back to top

intuitionrelated conceptsrelated formulas

Gram-Schmidt written as a factorization: the orthonormal vectors it produces become the columns of

Q

, and the coefficients it subtracted along the way become the entries of

R

. The triangularity of

R

is not imposed — each column was only ever corrected against the ones before it.

↑ Back to top

Cholesky Factorization

Writing a positive definite matrix as

A = LL^T

with

L

lower triangular and positive diagonal

What Cholesky Decomposition Is

See details

↑ Back to top

intuitionrelated concepts

A square root of the matrix, computed by alternating square roots on the diagonal with simple divisions below it. It exists exactly when

A

is positive definite, so attempting it doubles as the standard test for that property, and it works on only half the matrix — about twice as fast as LU.

↑ Back to top