Linear Algebra

Introduction to Linear Algebra

Linear algebra is a field of mathematics that focuses on studying vectors, matrices, and the relationships between them, forming the mathematical framework for analyzing structures and transformations in multidimensional spaces. It introduces powerful tools to understand and solve problems where quantities interact linearly, making it fundamental to numerous disciplines.

This field begins with vectors—quantities that have both magnitude and direction—and their operations, such as addition and scaling. It extends to matrices, which are grid-like arrangements of numbers used to represent systems of equations or transformations. Learning how to manipulate matrices and understand their properties is a key part of linear algebra.

Students also explore vector spaces, the environments in which vectors live, and subspaces, which reveal structure and constraints within these spaces. Concepts like linear independence, span, and basis give insight into how vectors relate and interact. The study of linear transformations, which describe how vectors change under operations like rotations or scaling, helps build a deeper understanding of the subject.

To help students navigate these foundational concepts, we created a dedicated Matrix Theory section that provides an in-depth exploration of matrices - from basic definitions and notations to various matrix types and properties. This interactive guide covers matrix structure, indexing, special cases of square matrices, and key matrix properties, with clear mathematical notation and visual examples throughout. The section serves as both a comprehensive learning resource and a quick reference for understanding these essential building blocks of linear algebra.

Eigenvalues and eigenvectors, pivotal concepts in linear algebra, allow students to uncover hidden properties of transformations. Techniques like solving systems of equations, matrix decomposition, and understanding projections or orthogonality are practical outcomes of this study.

Ultimately, linear algebra provides a foundation for solving abstract and applied problems, developing skills to think logically, recognize patterns, and simplify complex systems. It equips students with a versatile toolkit for further studies in mathematics, sciences, engineering, and beyond.

Linear Algebra Formulas

Navigate through an essential collection of linear algebra formulas that power mathematical analysis and transformations. This guide presents key formulas across vector operations, matrix calculations, eigenvalues, and linear transformations - each equipped with clear notation, detailed explanations, and practical examples. You will find precise mathematical representations, component breakdowns, and specific use cases for over 15 fundamental formula categories. The organized structure helps you quickly locate and understand the tools you need, whether for solving equations, analyzing transformations, or applying linear algebra concepts in real-world scenarios. Perfect for students needing formula clarification, researchers requiring quick mathematical reference, or practitioners applying linear algebra in their work.

Vector Component Form

\mathbf{v} = (v_1, v_2, \ldots, v_n) \in \mathbb{R}^n

Standard Basis Decomposition

\mathbf{v} = v_1 \mathbf{e}_1 + v_2 \mathbf{e}_2 + \cdots + v_n \mathbf{e}_n

Direction Cosines

\cos\alpha = \frac{v_1}{\|\mathbf{v}\|}, \quad \cos\beta = \frac{v_2}{\|\mathbf{v}\|}, \quad \cos\gamma = \frac{v_3}{\|\mathbf{v}\|}

Direction Cosines Identity

\cos^2\alpha + \cos^2\beta + \cos^2\gamma = 1

Vector Addition

\mathbf{a} + \mathbf{b} = (a_1 + b_1,\ a_2 + b_2,\ \ldots,\ a_n + b_n)

Vector Subtraction

\mathbf{a} - \mathbf{b} = \mathbf{a} + (-\mathbf{b}) = (a_1 - b_1,\ a_2 - b_2,\ \ldots,\ a_n - b_n)

Scalar Multiplication of Vectors

c\mathbf{a} = (ca_1,\ ca_2,\ \ldots,\ ca_n)

Linear Combination

c_1 \mathbf{v}_1 + c_2 \mathbf{v}_2 + \cdots + c_k \mathbf{v}_k

Span

\text{Span}\{\mathbf{v}_1, \ldots, \mathbf{v}_k\} = \{c_1 \mathbf{v}_1 + \cdots + c_k \mathbf{v}_k \mid c_i \in \mathbb{R}\}

Euclidean Norm

\|\mathbf{v}\| = \sqrt{v_1^2 + v_2^2 + \cdots + v_n^2} = \sqrt{\sum_{i=1}^{n} v_i^2}

Distance Formula

d(\mathbf{a}, \mathbf{b}) = \|\mathbf{a} - \mathbf{b}\| = \sqrt{(a_1 - b_1)^2 + (a_2 - b_2)^2 + \cdots + (a_n - b_n)^2}

Vector Normalization

\hat{\mathbf{v}} = \frac{\mathbf{v}}{\|\mathbf{v}\|}

Norm Scaling Property

\|c\mathbf{v}\| = |c|\,\|\mathbf{v}\|

Triangle Inequality

\|\mathbf{a} + \mathbf{b}\| \leq \|\mathbf{a}\| + \|\mathbf{b}\|

Cauchy-Schwarz Inequality

|\mathbf{a} \cdot \mathbf{b}| \leq \|\mathbf{a}\|\,\|\mathbf{b}\|

Dot Product (Algebraic)

\mathbf{a} \cdot \mathbf{b} = a_1 b_1 + a_2 b_2 + \cdots + a_n b_n = \sum_{i=1}^{n} a_i b_i

Dot Product (Geometric)

\mathbf{a} \cdot \mathbf{b} = \|\mathbf{a}\|\,\|\mathbf{b}\|\cos\theta

Angle Between Vectors

\cos\theta = \frac{\mathbf{a} \cdot \mathbf{b}}{\|\mathbf{a}\|\,\|\mathbf{b}\|}

Self Dot Product

\mathbf{v} \cdot \mathbf{v} = v_1^2 + v_2^2 + \cdots + v_n^2 = \|\mathbf{v}\|^2

Orthogonality Condition

\mathbf{a} \cdot \mathbf{b} = 0 \iff \mathbf{a} \perp \mathbf{b}

Scalar Projection

\text{comp}_{\mathbf{b}}\,\mathbf{a} = \frac{\mathbf{a} \cdot \mathbf{b}}{\|\mathbf{b}\|}

Vector Projection

\text{proj}_{\mathbf{b}}\,\mathbf{a} = \frac{\mathbf{a} \cdot \mathbf{b}}{\|\mathbf{b}\|^2}\,\mathbf{b} = \frac{\mathbf{a} \cdot \mathbf{b}}{\mathbf{b} \cdot \mathbf{b}}\,\mathbf{b}

Orthogonal Decomposition

\mathbf{a} = \text{proj}_{\mathbf{b}}\,\mathbf{a} + \mathbf{a}_{\perp}, \quad \mathbf{a}_{\perp} = \mathbf{a} - \text{proj}_{\mathbf{b}}\,\mathbf{a}

Cross Product (Component Form)

\mathbf{a} \times \mathbf{b} = (a_2 b_3 - a_3 b_2,\ a_3 b_1 - a_1 b_3,\ a_1 b_2 - a_2 b_1)

Cross Product (Determinant Form)

\mathbf{a} \times \mathbf{b} = \begin{vmatrix} \mathbf{i} & \mathbf{j} & \mathbf{k} \\ a_1 & a_2 & a_3 \\ b_1 & b_2 & b_3 \end{vmatrix}

Cross Product Magnitude

\|\mathbf{a} \times \mathbf{b}\| = \|\mathbf{a}\|\,\|\mathbf{b}\|\sin\theta

Standard Basis Cross Products

\mathbf{i} \times \mathbf{j} = \mathbf{k}, \quad \mathbf{j} \times \mathbf{k} = \mathbf{i}, \quad \mathbf{k} \times \mathbf{i} = \mathbf{j}

Cross Product Anti-Commutativity

\mathbf{a} \times \mathbf{b} = -(\mathbf{b} \times \mathbf{a})

Parallelism Test (Cross Product)

\mathbf{a} \times \mathbf{b} = \mathbf{0} \iff \mathbf{a} \parallel \mathbf{b}

Vector Triple Product

\mathbf{a} \times (\mathbf{b} \times \mathbf{c}) = (\mathbf{a} \cdot \mathbf{c})\,\mathbf{b} - (\mathbf{a} \cdot \mathbf{b})\,\mathbf{c}

Lagrange Identity

(\mathbf{a} \times \mathbf{b}) \cdot (\mathbf{c} \times \mathbf{d}) = (\mathbf{a} \cdot \mathbf{c})(\mathbf{b} \cdot \mathbf{d}) - (\mathbf{a} \cdot \mathbf{d})(\mathbf{b} \cdot \mathbf{c})

Scalar Triple Product

\mathbf{a} \cdot (\mathbf{b} \times \mathbf{c}) = \begin{vmatrix} a_1 & a_2 & a_3 \\ b_1 & b_2 & b_3 \\ c_1 & c_2 & c_3 \end{vmatrix}

Parallelogram Area

\text{Area} = \|\mathbf{a} \times \mathbf{b}\|

Parallelepiped Volume

V = |\mathbf{a} \cdot (\mathbf{b} \times \mathbf{c})|

Pyramid Volume

V = \tfrac{1}{6}|\mathbf{a} \cdot (\mathbf{b} \times \mathbf{c})|

Vector Space Axioms

\begin{aligned} &\text{For all } \mathbf{u}, \mathbf{v}, \mathbf{w} \in V \text{ and all } c, d \in \mathbb{F}: \\ &(1)\ \mathbf{u} + \mathbf{v} \in V \quad (2)\ \mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u} \\ &(3)\ (\mathbf{u} + \mathbf{v}) + \mathbf{w} = \mathbf{u} + (\mathbf{v} + \mathbf{w}) \\ &(4)\ \exists\, \mathbf{0} \in V: \mathbf{v} + \mathbf{0} = \mathbf{v} \\ &(5)\ \exists\, -\mathbf{v} \in V: \mathbf{v} + (-\mathbf{v}) = \mathbf{0} \\ &(6)\ c\mathbf{v} \in V \quad (7)\ c(d\mathbf{v}) = (cd)\mathbf{v} \\ &(8)\ c(\mathbf{u} + \mathbf{v}) = c\mathbf{u} + c\mathbf{v} \\ &(9)\ (c + d)\mathbf{v} = c\mathbf{v} + d\mathbf{v} \quad (10)\ 1\mathbf{v} = \mathbf{v} \end{aligned}

Scalar-Zero Property

0\mathbf{v} = \mathbf{0}, \quad c\mathbf{0} = \mathbf{0}

Negative One Scalar

(-1)\mathbf{v} = -\mathbf{v}

Subspace Test

W \subseteq V \text{ is a subspace} \iff \begin{cases} W \neq \emptyset \\ \mathbf{u}, \mathbf{v} \in W \Rightarrow \mathbf{u} + \mathbf{v} \in W \\ \mathbf{v} \in W,\ c \in \mathbb{F} \Rightarrow c\mathbf{v} \in W \end{cases}

Subspace Test Combined

W \subseteq V \text{ is a subspace} \iff W \neq \emptyset \text{ and } c\mathbf{u} + d\mathbf{v} \in W \text{ for all } \mathbf{u}, \mathbf{v} \in W,\ c, d \in \mathbb{F}

Span (Set Definition)

\text{Span}\{\mathbf{v}_1, \ldots, \mathbf{v}_k\} = \left\{c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \cdots + c_k\mathbf{v}_k \mid c_i \in \mathbb{F}\right\}

Span Membership Criterion

\mathbf{b} \in \text{Span}\{\mathbf{v}_1, \ldots, \mathbf{v}_k\} \iff A\mathbf{c} = \mathbf{b} \text{ is consistent}

Span Is Smallest Subspace

\text{Span}(K) = \bigcap_{\substack{W \text{ subspace} \\ K \subseteq W}} W

Linear Independence Equation

\{\mathbf{v}_1, \ldots, \mathbf{v}_k\} \text{ is independent} \iff \bigl(c_1\mathbf{v}_1 + \cdots + c_k\mathbf{v}_k = \mathbf{0} \Rightarrow c_1 = \cdots = c_k = 0\bigr)

Linear Independence Matrix Test

\{\mathbf{v}_1, \ldots, \mathbf{v}_k\} \subset \mathbb{R}^m \text{ is independent} \iff A\mathbf{c} = \mathbf{0} \text{ has only the trivial solution}

Linear Independence Determinant Test

\{\mathbf{v}_1, \ldots, \mathbf{v}_n\} \subset \mathbb{R}^n \text{ is independent} \iff \det[\mathbf{v}_1\ \cdots\ \mathbf{v}_n] \neq 0

Max Independent Set Size

|S| > \dim V \Rightarrow S \text{ is dependent}

Wronskian Test

W(f_1, \ldots, f_n)(x) = \det\begin{pmatrix} f_1(x) & \cdots & f_n(x) \\ f_1'(x) & \cdots & f_n'(x) \\ \vdots & \ddots & \vdots \\ f_1^{(n-1)}(x) & \cdots & f_n^{(n-1)}(x) \end{pmatrix}

Basis Definition

\mathcal{B} = \{\mathbf{v}_1, \ldots, \mathbf{v}_n\} \text{ is a basis for } V \iff \begin{cases} \mathcal{B} \text{ is linearly independent} \\ \text{Span}(\mathcal{B}) = V \end{cases}

Unique Basis Representation

\forall\, \mathbf{v} \in V,\ \exists!\, (c_1, \ldots, c_n): \mathbf{v} = c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \cdots + c_n\mathbf{v}_n

Coordinate Vector

[\mathbf{v}]_\mathcal{B} = \begin{pmatrix} c_1 \\ c_2 \\ \vdots \\ c_n \end{pmatrix} \quad \text{where } \mathbf{v} = c_1\mathbf{v}_1 + \cdots + c_n\mathbf{v}_n

Standard Basis (Rn)

\mathbf{e}_i = (0, \ldots, 0, \underset{i\text{-th}}{1}, 0, \ldots, 0), \quad i = 1, \ldots, n

Change of Basis Formula

[\mathbf{v}]_\mathcal{C} = P_{\mathcal{C} \leftarrow \mathcal{B}}\, [\mathbf{v}]_\mathcal{B}

Change of Basis Inverse

P_{\mathcal{B} \leftarrow \mathcal{C}} = \bigl(P_{\mathcal{C} \leftarrow \mathcal{B}}\bigr)^{-1}

Coordinate Map Linearity

[\mathbf{u} + \mathbf{v}]_\mathcal{B} = [\mathbf{u}]_\mathcal{B} + [\mathbf{v}]_\mathcal{B}, \qquad [c\mathbf{v}]_\mathcal{B} = c\,[\mathbf{v}]_\mathcal{B}

Dimension Definition

\dim(V) = |\mathcal{B}| \quad \text{for any basis } \mathcal{B} \text{ of } V

Subspace Dimension Inequality

W \subseteq V \Rightarrow \dim(W) \leq \dim(V), \quad \text{with equality} \iff W = V

Dimension Sum Formula

\dim(W_1 + W_2) = \dim(W_1) + \dim(W_2) - \dim(W_1 \cap W_2)

Direct Sum Criterion

V = W_1 \oplus W_2 \iff V = W_1 + W_2 \text{ and } W_1 \cap W_2 = \{\mathbf{0}\}

Direct Sum Dimension

V = W_1 \oplus W_2 \Rightarrow \dim(V) = \dim(W_1) + \dim(W_2)

Rank-Nullity Theorem (Matrix Form)

\dim(\text{Col}\,A) + \dim(\text{Null}\,A) = n

Four Fundamental Subspaces Dimensions

\begin{aligned} \dim(\text{Col}\,A) &= r & \dim(\text{Row}\,A) &= r \\ \dim(\text{Null}\,A) &= n - r & \dim(\text{Null}\,A^T) &= m - r \end{aligned}

Row Rank Equals Column Rank

\dim(\text{Row}\,A) = \dim(\text{Col}\,A) = \text{rank}(A)

Matrix Equality

A = B \iff a_{ij} = b_{ij} \text{ for all } i, j

Matrix Addition

(A + B)_{ij} = a_{ij} + b_{ij}

Matrix Subtraction

(A - B)_{ij} = a_{ij} - b_{ij}

Scalar Multiplication of Matrices

(cA)_{ij} = c \cdot a_{ij}

Matrix Multiplication

(AB)_{ij} = \sum_{k=1}^{n} a_{ik}\, b_{kj}

Matrix-Vector Product (Column Form)

A\mathbf{x} = x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 + \cdots + x_n \mathbf{a}_n

Matrix Multiplication Associativity

(AB)C = A(BC)

Matrix Multiplication Distributivity

A(B + C) = AB + AC, \qquad (A + B)C = AC + BC

Matrix Multiplication Non-Commutativity

AB \neq BA \quad \text{in general}

Matrix Power

A^0 = I, \qquad A^k = \underbrace{A \cdot A \cdots A}_{k \text{ factors}}, \qquad A^{-k} = (A^{-1})^k

Transpose Definition

(A^T)_{ij} = a_{ji}

Transpose Involution

(A^T)^T = A

Transpose of Sum

(A + B)^T = A^T + B^T

Transpose of Scalar Multiple

(cA)^T = c\, A^T

Transpose of Product

(AB)^T = B^T A^T

Symmetric Matrix Definition

A = A^T \iff a_{ij} = a_{ji} \text{ for all } i, j

Skew-Symmetric Matrix Definition

A^T = -A \iff a_{ij} = -a_{ji}

Symmetric Skew Decomposition

A = \tfrac{1}{2}(A + A^T) + \tfrac{1}{2}(A - A^T)

Gram Matrix Symmetry

(A^T A)^T = A^T A, \qquad (A A^T)^T = A A^T

Identity Matrix Definition

I_n = [\delta_{ij}], \qquad \delta_{ij} = \begin{cases} 1 & i = j \\ 0 & i \neq j \end{cases}

Identity Matrix Property

AI = IA = A

Diagonal Matrix Definition

D = \operatorname{diag}(d_1, d_2, \ldots, d_n) \iff d_{ij} = 0 \text{ for } i \neq j

Diagonal Matrix Power

D^k = \operatorname{diag}(d_1^k, d_2^k, \ldots, d_n^k)

Diagonal Matrix Determinant

\det\bigl(\operatorname{diag}(d_1, \ldots, d_n)\bigr) = d_1 d_2 \cdots d_n

Triangular Matrix Determinant

\det(T) = t_{11} \, t_{22} \cdots t_{nn}

Orthogonal Matrix Definition

Q^T Q = Q Q^T = I

Orthogonal Matrix Determinant

\det(Q) = \pm 1

Idempotent Matrix Definition

A^2 = A

Idempotent Rank Trace

A^2 = A \implies \operatorname{rank}(A) = \operatorname{tr}(A)

Nilpotent Matrix Definition

A^k = O \quad \text{for some } k \geq 1

Neumann Series Nilpotent

A^k = O \implies (I - A)^{-1} = I + A + A^2 + \cdots + A^{k-1}

Involutory Matrix Definition

A^2 = I

Cross Product Skew Matrix

\mathbf{a} \times \mathbf{b} = [\mathbf{a}]_\times \mathbf{b}, \qquad [\mathbf{a}]_\times = \begin{pmatrix} 0 & -a_3 & a_2 \\ a_3 & 0 & -a_1 \\ -a_2 & a_1 & 0 \end{pmatrix}

Inverse Definition

A A^{-1} = A^{-1} A = I

Inverse 2x2 Formula

\begin{pmatrix} a & b \\ c & d \end{pmatrix}^{-1} = \frac{1}{ad - bc}\begin{pmatrix} d & -b \\ -c & a \end{pmatrix}

Inverse via Adjugate

A^{-1} = \frac{1}{\det(A)}\, \operatorname{adj}(A)

Inverse via Row Reduction

[A \mid I] \;\xrightarrow{\text{row ops}}\; [I \mid A^{-1}]

Inverse Involution

(A^{-1})^{-1} = A

Inverse of Product

(AB)^{-1} = B^{-1} A^{-1}

Inverse of Transpose

(A^T)^{-1} = (A^{-1})^T

Inverse of Scalar Multiple

(cA)^{-1} = \frac{1}{c}\, A^{-1}

Inverse of Power

(A^k)^{-1} = (A^{-1})^k = A^{-k}

Determinant of Inverse

\det(A^{-1}) = \frac{1}{\det(A)}

Diagonal Matrix Inverse

\operatorname{diag}(d_1, \ldots, d_n)^{-1} = \operatorname{diag}\!\left(\frac{1}{d_1}, \ldots, \frac{1}{d_n}\right)

Orthogonal Matrix Inverse

Q^{-1} = Q^T

Solve System via Inverse

A\mathbf{x} = \mathbf{b} \implies \mathbf{x} = A^{-1}\mathbf{b}

Invertible Matrix Theorem

\begin{aligned} \text{For } A \in \mathbb{R}^{n \times n}, \text{ the following are equivalent:} & \\ (1)\ A \text{ is invertible} \quad (2)\ \det(A) \neq 0 \quad (3)\ \operatorname{rank}(A) = n & \\ (4)\ \text{columns of } A \text{ are linearly independent} & \\ (5)\ \text{rows of } A \text{ are linearly independent} & \\ (6)\ \text{columns of } A \text{ span } \mathbb{R}^n \quad (7)\ \text{columns form a basis of } \mathbb{R}^n & \\ (8)\ A\mathbf{x} = \mathbf{0} \text{ has only the trivial solution} & \\ (9)\ A\mathbf{x} = \mathbf{b} \text{ has a unique solution for every } \mathbf{b} & \\ (10)\ \operatorname{Null}(A) = \{\mathbf{0}\} & \\ (11)\ \operatorname{rref}(A) = I \quad (12)\ A \text{ is a product of elementary matrices} & \\ (13)\ 0 \text{ is not an eigenvalue of } A & \end{aligned}

Singular Matrix Definition

A \text{ singular} \iff \det(A) = 0 \iff \operatorname{rank}(A) < n

Rank Bounds

0 \leq \operatorname{rank}(A) \leq \min(m, n)

Rank of Transpose

\operatorname{rank}(A^T) = \operatorname{rank}(A)

Rank Product Inequality

\operatorname{rank}(AB) \leq \min\bigl(\operatorname{rank}(A), \operatorname{rank}(B)\bigr)

Sylvester Rank Inequality

\operatorname{rank}(A) + \operatorname{rank}(B) - n \leq \operatorname{rank}(AB)

Rank Sum Inequality

\operatorname{rank}(A + B) \leq \operatorname{rank}(A) + \operatorname{rank}(B)

Rank Invariance Invertible

\operatorname{rank}(PAQ) = \operatorname{rank}(A)

Gram Rank Identity

\operatorname{rank}(A^T A) = \operatorname{rank}(A A^T) = \operatorname{rank}(A)

Rank-One Outer Product

A = \mathbf{u}\mathbf{v}^T \implies \operatorname{rank}(A) = 1 \quad (\mathbf{u}, \mathbf{v} \neq \mathbf{0})

Trace Definition

\operatorname{tr}(A) = \sum_{i=1}^{n} a_{ii} = a_{11} + a_{22} + \cdots + a_{nn}

Trace Linearity

\operatorname{tr}(cA + dB) = c\operatorname{tr}(A) + d\operatorname{tr}(B)

Trace of Transpose

\operatorname{tr}(A^T) = \operatorname{tr}(A)

Trace Cyclic Property

\operatorname{tr}(AB) = \operatorname{tr}(BA)

Trace Sum of Eigenvalues

\operatorname{tr}(A) = \sum_{i=1}^{n} \lambda_i

Trace Similarity Invariance

\operatorname{tr}(P^{-1}AP) = \operatorname{tr}(A)

Trace of Commutator

\operatorname{tr}(AB - BA) = 0

Trace Symmetric Skew Product

\operatorname{tr}(SK) = 0 \quad (S^T = S, \; K^T = -K)

Trace Orthonormal Basis

\operatorname{tr}(A) = \sum_{i=1}^{n} \mathbf{q}_i^T A \mathbf{q}_i

Frobenius Inner Product

\langle A, B \rangle_F = \operatorname{tr}(A^T B) = \sum_{i,j} a_{ij} b_{ij}

Frobenius Norm

\|A\|_F = \sqrt{\operatorname{tr}(A^T A)} = \sqrt{\sum_{i,j} a_{ij}^2}

Determinant 2x2

\det\begin{pmatrix} a & b \\ c & d \end{pmatrix} = ad - bc

Determinant 3x3

\det(A) = a_{11}(a_{22}a_{33} - a_{23}a_{32}) - a_{12}(a_{21}a_{33} - a_{23}a_{31}) + a_{13}(a_{21}a_{32} - a_{22}a_{31})

Determinant Recursive Definition

\det(A) = \begin{cases} a_{11} & n = 1 \\ \displaystyle\sum_{j=1}^{n} (-1)^{1+j} \, a_{1j} \, M_{1j} & n \geq 2 \end{cases}

Determinant Permutation Formula

\det(A) = \sum_{\sigma \in S_n} \operatorname{sgn}(\sigma) \, a_{\sigma(1),1} \, a_{\sigma(2),2} \cdots a_{\sigma(n),n}

Minor Definition

M_{ij} = \det\!\left(A^{(i,j)}\right)

Cofactor Definition

C_{ij} = (-1)^{i+j} \, M_{ij}

Cofactor Matrix Definition

\operatorname{cof}(A) = \bigl[C_{ij}\bigr]_{n \times n}

Adjugate Definition

\operatorname{adj}(A) = \operatorname{cof}(A)^T

Laplace Row Expansion

\det(A) = \sum_{j=1}^{n} a_{ij} \, C_{ij} \qquad \text{for any fixed row } i

Laplace Column Expansion

\det(A) = \sum_{i=1}^{n} a_{ij} \, C_{ij} \qquad \text{for any fixed column } j

Determinant Row Swap

\det(B) = -\det(A)

Determinant Row Scaling

\det(B) = k \, \det(A)

Determinant Row Addition

\det(B) = \det(A)

Determinant of Transpose

\det(A^T) = \det(A)

Determinant of Product

\det(AB) = \det(A) \, \det(B)

Determinant of Scalar Multiple

\det(kA) = k^n \, \det(A)

Determinant of Power

\det(A^k) = \bigl(\det(A)\bigr)^k

Determinant of Identity

\det(I_n) = 1

Block Triangular Determinant

\det\begin{pmatrix} A_{11} & A_{12} \\ 0 & A_{22} \end{pmatrix} = \det(A_{11}) \, \det(A_{22})

Vandermonde Determinant

\det(V) = \prod_{1 \leq i < j \leq n} (x_j - x_i)

Adjugate Identity

A \cdot \operatorname{adj}(A) = \operatorname{adj}(A) \cdot A = \det(A) \, I

Cramers Rule

x_i = \frac{\det(A_i)}{\det(A)}

Determinant Signed Area 2D

\text{signed area}(\mathbf{u}, \mathbf{v}) = \det\begin{pmatrix} \mathbf{u} & \mathbf{v} \end{pmatrix} = ad - bc

Determinant Signed Volume 3D

\text{signed volume}(\mathbf{a}, \mathbf{b}, \mathbf{c}) = \det\begin{pmatrix} \mathbf{a} & \mathbf{b} & \mathbf{c} \end{pmatrix} = \mathbf{a} \cdot (\mathbf{b} \times \mathbf{c})

Determinant Volume Scaling Factor

\operatorname{vol}\bigl(A(S)\bigr) = |\det(A)| \cdot \operatorname{vol}(S)

Triangle Area via Determinant

\text{Area} = \frac{1}{2} \left| \det\begin{pmatrix} x_2 - x_1 & x_3 - x_1 \\ y_2 - y_1 & y_3 - y_1 \end{pmatrix} \right|

Tetrahedron Volume via Determinant

V = \frac{1}{6} \left| \det\begin{pmatrix} \mathbf{e}_1 & \mathbf{e}_2 & \mathbf{e}_3 \end{pmatrix} \right|

Determinant Product of Eigenvalues

\det(A) = \lambda_1 \, \lambda_2 \cdots \lambda_n

Linear Equation Standard Form

a_1 x_1 + a_2 x_2 + \cdots + a_n x_n = b

Linear System Matrix Form

A\mathbf{x} = \mathbf{b}

Vector Equation Form

x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 + \cdots + x_n \mathbf{a}_n = \mathbf{b}

Augmented Matrix Construction

[A \mid \mathbf{b}] = \left(\begin{array}{cccc|c} a_{11} & a_{12} & \cdots & a_{1n} & b_1 \\ a_{21} & a_{22} & \cdots & a_{2n} & b_2 \\ \vdots & \vdots & \ddots & \vdots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} & b_m \end{array}\right)

Row Echelon Form Definition

\text{REF: } \begin{pmatrix} \boxed{\ast} & \bullet & \bullet & \bullet & \bullet \\ 0 & \boxed{\ast} & \bullet & \bullet & \bullet \\ 0 & 0 & 0 & \boxed{\ast} & \bullet \\ 0 & 0 & 0 & 0 & 0 \end{pmatrix}

Reduced Row Echelon Form Definition

\text{RREF: } \begin{pmatrix} \boxed{1} & 0 & \bullet & 0 & \bullet \\ 0 & \boxed{1} & \bullet & 0 & \bullet \\ 0 & 0 & 0 & \boxed{1} & \bullet \\ 0 & 0 & 0 & 0 & 0 \end{pmatrix}

RREF Uniqueness

\text{RREF}(A) \text{ is unique}

Pivot Definition

\text{pivot} = \text{leading nonzero entry of a row in echelon form}

Elementary Row Operations

\begin{aligned} R_i &\leftrightarrow R_j \quad \text{(swap)} \\ kR_i &\to R_i, \quad k \neq 0 \quad \text{(scaling)} \\ R_i + cR_j &\to R_i \quad \text{(addition)} \end{aligned}

Row Equivalence Preserves Solutions

[A \mid \mathbf{b}] \sim [A' \mid \mathbf{b}'] \;\Longrightarrow\; \text{Sol}(A\mathbf{x} = \mathbf{b}) = \text{Sol}(A'\mathbf{x} = \mathbf{b}')

Free Variables Count

\text{(number of free variables)} = n - \text{rank}(A)

Solvability Rank Criterion

A\mathbf{x} = \mathbf{b} \text{ is consistent} \iff \text{rank}(A) = \text{rank}([A \mid \mathbf{b}])

Solution Structure Decomposition

\mathbf{x} = \mathbf{x}_p + \mathbf{x}_h, \quad \mathbf{x}_h \in \text{Null}(A)

Homogeneous Solution Space Dimension

\dim \text{Null}(A) = n - \text{rank}(A)

Underdetermined Homogeneous Has Nontrivial

n > m \;\Longrightarrow\; A\mathbf{x} = \mathbf{0} \text{ has a nontrivial solution}

Linear Transformation Definition

T(c\mathbf{u} + d\mathbf{v}) = cT(\mathbf{u}) + dT(\mathbf{v})

Zero Vector Preservation

T(\mathbf{0}_V) = \mathbf{0}_W

Composition Is Linear

(S \circ T)(\mathbf{u}) = S(T(\mathbf{u})), \qquad [S \circ T] = [S]\,[T]

Standard Matrix

A = \bigl[\,T(\mathbf{e}_1) \;\; T(\mathbf{e}_2) \;\; \cdots \;\; T(\mathbf{e}_n)\,\bigr]

Linear Map as Matrix Multiplication

T(\mathbf{x}) = A\mathbf{x} \quad \text{for every } \mathbf{x} \in \mathbb{R}^n

Matrix Representation Abstract Bases

[T]_{\mathcal{C} \leftarrow \mathcal{B}} = \bigl[\,[T(\mathbf{v}_1)]_{\mathcal{C}} \;\; \cdots \;\; [T(\mathbf{v}_n)]_{\mathcal{C}}\,\bigr]

Image Definition

\text{Im}(T) = \{T(\mathbf{v}) : \mathbf{v} \in V\} \subseteq W

Kernel Definition

\ker(T) = \{\mathbf{v} \in V : T(\mathbf{v}) = \mathbf{0}\} \subseteq V

Injectivity Kernel Criterion

T \text{ injective} \iff \ker(T) = \{\mathbf{0}\}

Rank-Nullity for Maps

\dim\text{Im}(T) + \dim\ker(T) = \dim V

Bijectivity Equal Dim Case

\dim V = \dim W \Rightarrow \bigl(T \text{ injective} \iff T \text{ surjective} \iff T \text{ bijective}\bigr)

Similarity Relation

A' = P^{-1} A P

Similarity Invariants

A' = P^{-1}AP \Rightarrow \begin{cases} \det(A') = \det(A) \\ \text{tr}(A') = \text{tr}(A) \\ \text{rank}(A') = \text{rank}(A) \\ \text{eigenvalues}(A') = \text{eigenvalues}(A) \end{cases}

Diagonalization Formula

A = P D P^{-1}, \quad D = \operatorname{diag}(\lambda_1, \ldots, \lambda_n), \quad P = [\mathbf{v}_1 \;\cdots\; \mathbf{v}_n]

Rotation Matrix 2D

R_\theta = \begin{pmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{pmatrix}

Rotation Matrices 3D

R_x(\theta) = \begin{pmatrix} 1 & 0 & 0 \\ 0 & \cos\theta & -\sin\theta \\ 0 & \sin\theta & \cos\theta \end{pmatrix}, \;\; R_y(\theta) = \begin{pmatrix} \cos\theta & 0 & \sin\theta \\ 0 & 1 & 0 \\ -\sin\theta & 0 & \cos\theta \end{pmatrix}, \;\; R_z(\theta) = \begin{pmatrix} \cos\theta & -\sin\theta & 0 \\ \sin\theta & \cos\theta & 0 \\ 0 & 0 & 1 \end{pmatrix}

Reflection Across Line 2D

H_\alpha = \begin{pmatrix} \cos 2\alpha & \sin 2\alpha \\ \sin 2\alpha & -\cos 2\alpha \end{pmatrix}

Householder Reflection

H = I - 2\,\mathbf{n}\mathbf{n}^T

Projection onto Line

P = \frac{\mathbf{u}\mathbf{u}^T}{\mathbf{u}^T\mathbf{u}}

Projection onto Plane

P = I - \frac{\mathbf{n}\mathbf{n}^T}{\mathbf{n}^T\mathbf{n}}

Shear Matrix

\text{Shear}_x = \begin{pmatrix} 1 & k \\ 0 & 1 \end{pmatrix}, \qquad \text{Shear}_y = \begin{pmatrix} 1 & 0 \\ k & 1 \end{pmatrix}

Eigenvalue Definition

A\mathbf{v} = \lambda\mathbf{v}, \quad \mathbf{v} \neq \mathbf{0}

Characteristic Equation

\det(A - \lambda I) = 0

Eigenspace

E_\lambda = \text{Null}(A - \lambda I) = \{\mathbf{v} : A\mathbf{v} = \lambda\mathbf{v}\}

Characteristic Polynomial

p(\lambda) = \det(A - \lambda I)

Characteristic Polynomial 2x2

p(\lambda) = \lambda^2 - \text{tr}(A)\,\lambda + \det(A)

Cayley-Hamilton

p(A) = O

Multiplicity Inequality

1 \leq m_g(\lambda) \leq m_a(\lambda)

Independence of Distinct Eigenvectors

\lambda_1, \ldots, \lambda_k \text{ distinct} \Rightarrow \{\mathbf{v}_1, \ldots, \mathbf{v}_k\} \text{ linearly independent}

Eigenvalue of Power

A\mathbf{v} = \lambda\mathbf{v} \Rightarrow A^k\mathbf{v} = \lambda^k\mathbf{v}

Eigenvalue of Inverse

A\mathbf{v} = \lambda\mathbf{v} \Rightarrow A^{-1}\mathbf{v} = \frac{1}{\lambda}\mathbf{v}

Eigenvalue of Polynomial

q(A)\mathbf{v} = q(\lambda)\mathbf{v}

Eigenvalue Shift

A\mathbf{v} = \lambda\mathbf{v} \Rightarrow (A + cI)\mathbf{v} = (\lambda + c)\mathbf{v}

Diagonalizability Condition

A \text{ diagonalizable} \iff m_g(\lambda) = m_a(\lambda) \text{ for every eigenvalue } \lambda

Distinct Eigenvalues Imply Diagonalizable

A \text{ has } n \text{ distinct eigenvalues} \Rightarrow A \text{ is diagonalizable}

Special Matrix Eigenvalue Restrictions

\begin{aligned} \text{symmetric (real)}: \;& \lambda \in \mathbb{R} \\ \text{skew-symmetric (real)}: \;& \lambda = 0 \text{ or } \lambda \in i\mathbb{R} \\ \text{orthogonal}: \;& |\lambda| = 1 \\ \text{idempotent}: \;& \lambda \in \{0, 1\} \\ \text{nilpotent}: \;& \lambda = 0 \\ \text{involutory}: \;& \lambda \in \{-1, +1\} \\ \text{positive definite}: \;& \lambda > 0 \end{aligned}

Spectral Theorem

A = A^T \Rightarrow A = Q D Q^T, \quad Q^T Q = I, \quad D = \text{diag}(\lambda_1, \ldots, \lambda_n) \in \mathbb{R}^{n \times n}

Spectral Decomposition

A = \sum_{i=1}^{n} \lambda_i \, \mathbf{q}_i \mathbf{q}_i^T

Matrix Exponential

e^{At} = P\, e^{Dt}\, P^{-1} = P \,\text{diag}(e^{\lambda_1 t}, \ldots, e^{\lambda_n t})\, P^{-1}

Complex Conjugate Pairs

A \in \mathbb{R}^{n \times n}, \;\; A\mathbf{v} = \lambda\mathbf{v} \Rightarrow A\bar{\mathbf{v}} = \bar{\lambda}\bar{\mathbf{v}}

Discriminant Classification 2x2

\Delta = \text{tr}(A)^2 - 4\det(A) \quad \begin{cases} \Delta > 0: & \text{two distinct real eigenvalues} \\ \Delta = 0: & \text{one repeated real eigenvalue} \\ \Delta < 0: & \text{complex conjugate pair} \end{cases}

Real Canonical Form 2x2

\lambda = a \pm bi \Rightarrow P^{-1}AP = \begin{pmatrix} a & -b \\ b & a \end{pmatrix} = r\begin{pmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{pmatrix}

Cauchy-Schwarz Inequality (General)

|\mathbf{u} \cdot \mathbf{v}| \leq \|\mathbf{u}\| \, \|\mathbf{v}\|

Triangle Inequality (Inner Product)

\|\mathbf{u} + \mathbf{v}\| \leq \|\mathbf{u}\| + \|\mathbf{v}\|

Pythagorean Theorem

\mathbf{u} \cdot \mathbf{v} = 0 \Rightarrow \|\mathbf{u} + \mathbf{v}\|^2 = \|\mathbf{u}\|^2 + \|\mathbf{v}\|^2

Inner Product Axioms

\begin{aligned} \text{Symmetry:} \;& \langle\mathbf{u},\mathbf{v}\rangle = \langle\mathbf{v},\mathbf{u}\rangle \\ \text{Linearity:} \;& \langle c\mathbf{u}+d\mathbf{w},\mathbf{v}\rangle = c\langle\mathbf{u},\mathbf{v}\rangle + d\langle\mathbf{w},\mathbf{v}\rangle \\ \text{Positive definite:} \;& \langle\mathbf{v},\mathbf{v}\rangle > 0 \text{ for } \mathbf{v} \neq \mathbf{0} \end{aligned}

Distance Formula (Inner Product)

d(\mathbf{u}, \mathbf{v}) = \|\mathbf{u} - \mathbf{v}\| = \sqrt{\sum_{i=1}^{n} (u_i - v_i)^2}

Orthogonal Complement Definition

W^\perp = \{\mathbf{v} \in \mathbb{R}^n : \mathbf{v} \cdot \mathbf{w} = 0 \text{ for all } \mathbf{w} \in W\}

Complement Dimension Sum

\dim(W) + \dim(W^\perp) = n

Orthogonal Decomposition (Subspace)

\mathbf{v} = \hat{\mathbf{v}} + \mathbf{z}, \quad \hat{\mathbf{v}} \in W, \quad \mathbf{z} \in W^\perp

Orthogonal Set Independence

\mathbf{v}_i \cdot \mathbf{v}_j = 0 \;\; (i \neq j), \;\; \mathbf{v}_i \neq \mathbf{0} \;\Rightarrow\; \{\mathbf{v}_1, \ldots, \mathbf{v}_k\} \text{ linearly independent}

Orthonormal Set

\mathbf{q}_i \cdot \mathbf{q}_j = \delta_{ij} = \begin{cases} 1 & i = j \\ 0 & i \neq j \end{cases}

Coordinates via Orthonormal Basis

\mathbf{x} = \sum_{i=1}^{n} (\mathbf{x} \cdot \mathbf{q}_i)\,\mathbf{q}_i, \qquad c_i = \mathbf{x} \cdot \mathbf{q}_i

Parseval Identity

\|\mathbf{x}\|^2 = \sum_{i=1}^{n} (\mathbf{x} \cdot \mathbf{q}_i)^2 = \sum_{i=1}^{n} c_i^2

Projection onto Subspace

\hat{\mathbf{b}} = A(A^TA)^{-1}A^T\,\mathbf{b}, \qquad P = A(A^TA)^{-1}A^T

Projection onto Orthonormal Basis

\text{proj}_W\,\mathbf{b} = \sum_{i=1}^{k} (\mathbf{q}_i \cdot \mathbf{b})\,\mathbf{q}_i

Orthonormal Columns Projection

P = QQ^T, \qquad Q^TQ = I_k

Projection Matrix Properties

P^2 = P, \qquad P^T = P

Complementary Projection

P_{W^\perp} = I - P_W, \qquad P_W\mathbf{b} + (I - P_W)\mathbf{b} = \mathbf{b}

Gram-Schmidt Process

\mathbf{u}_1 = \mathbf{v}_1, \qquad \mathbf{u}_j = \mathbf{v}_j - \sum_{i=1}^{j-1} \frac{\mathbf{u}_i \cdot \mathbf{v}_j}{\mathbf{u}_i \cdot \mathbf{u}_i}\,\mathbf{u}_i, \qquad \mathbf{q}_j = \frac{\mathbf{u}_j}{\|\mathbf{u}_j\|}

Normal Equations

A^TA\,\hat{\mathbf{x}} = A^T\mathbf{b}

Least-Squares Solution

\hat{\mathbf{x}} = (A^TA)^{-1} A^T \mathbf{b}, \qquad A^+ = (A^TA)^{-1}A^T

Least-Squares via QR

A = QR \;\Rightarrow\; R\,\hat{\mathbf{x}} = Q^T\mathbf{b}

LU Decomposition

A = LU

PA LU Partial Pivoting

PA = LU

Determinant via LU

\det(A) = (-1)^s \prod_{i=1}^{n} u_{ii}

LU Solve Steps

A\mathbf{x} = \mathbf{b} \;\Longleftrightarrow\; \begin{cases} L\mathbf{y} = P\mathbf{b} & (\text{forward sub}) \\ U\mathbf{x} = \mathbf{y} & (\text{back sub}) \end{cases}

Cholesky Decomposition

A = L L^T

Cholesky Diagonal Formula

l_{jj} = \sqrt{a_{jj} - \sum_{k=1}^{j-1} l_{jk}^2}

Cholesky Off-Diagonal Formula

l_{ij} = \frac{1}{l_{jj}}\left( a_{ij} - \sum_{k=1}^{j-1} l_{ik} l_{jk} \right), \qquad i > j

QR Decomposition

A = Q R

QR Gram-Schmidt R Entries

R_{ij} = \mathbf{q}_i \cdot \mathbf{a}_j \;\; (i \leq j), \qquad R_{ij} = 0 \;\; (i > j), \qquad R_{jj} = \|\mathbf{u}_j\|

QR Algorithm for Eigenvalues

A_k = Q_k R_k, \qquad A_{k+1} = R_k Q_k

SVD

A = U \Sigma V^T

Singular Values

\sigma_i = \sqrt{\lambda_i(A^TA)} = \sqrt{\lambda_i(AA^T)}

SVD Rank

\text{rank}(A) = \#\{i : \sigma_i > 0\}

SVD Outer Product Form

A = \sum_{i=1}^{r} \sigma_i \, \mathbf{u}_i \mathbf{v}_i^T

Moore-Penrose Pseudoinverse

A^+ = V \Sigma^+ U^T

Eckart-Young Low-Rank Approximation

A_k = \sum_{i=1}^{k} \sigma_i \mathbf{u}_i \mathbf{v}_i^T, \qquad \|A - A_k\|_2 = \sigma_{k+1}, \quad \|A - A_k\|_F = \sqrt{\sum_{i=k+1}^{r}\sigma_i^2}

Condition Number

\kappa(A) = \frac{\sigma_1}{\sigma_r}

Operator Norm

\|A\|_2 = \sigma_1 = \max_{\|\mathbf{x}\|=1} \|A\mathbf{x}\|

Frobenius Norm via Singular Values

\|A\|_F = \sqrt{\sum_{i=1}^{r} \sigma_i^2}

SVD Four Fundamental Subspaces

\begin{aligned} \text{Col}(A) &= \text{Span}\{\mathbf{u}_1, \ldots, \mathbf{u}_r\} \\ \text{Null}(A^T) &= \text{Span}\{\mathbf{u}_{r+1}, \ldots, \mathbf{u}_m\} \\ \text{Row}(A) &= \text{Span}\{\mathbf{v}_1, \ldots, \mathbf{v}_r\} \\ \text{Null}(A) &= \text{Span}\{\mathbf{v}_{r+1}, \ldots, \mathbf{v}_n\} \end{aligned}

Quadratic Form Diagonalization

\mathbf{x}^T A \mathbf{x} = \mathbf{y}^T D \mathbf{y} = \sum_{i=1}^{n} \lambda_i y_i^2, \qquad \mathbf{x} = Q\mathbf{y}

Vector Component Form

\mathbf{v} = (v_1, v_2, \ldots, v_n) \in \mathbb{R}^n

Standard Basis Decomposition

\mathbf{v} = v_1 \mathbf{e}_1 + v_2 \mathbf{e}_2 + \cdots + v_n \mathbf{e}_n

Direction Cosines

\cos\alpha = \frac{v_1}{\|\mathbf{v}\|}, \quad \cos\beta = \frac{v_2}{\|\mathbf{v}\|}, \quad \cos\gamma = \frac{v_3}{\|\mathbf{v}\|}

Direction Cosines Identity

\cos^2\alpha + \cos^2\beta + \cos^2\gamma = 1

Vector Addition

\mathbf{a} + \mathbf{b} = (a_1 + b_1,\ a_2 + b_2,\ \ldots,\ a_n + b_n)

Vector Subtraction

\mathbf{a} - \mathbf{b} = \mathbf{a} + (-\mathbf{b}) = (a_1 - b_1,\ a_2 - b_2,\ \ldots,\ a_n - b_n)

Scalar Multiplication of Vectors

c\mathbf{a} = (ca_1,\ ca_2,\ \ldots,\ ca_n)

Linear Combination

c_1 \mathbf{v}_1 + c_2 \mathbf{v}_2 + \cdots + c_k \mathbf{v}_k

Span

\text{Span}\{\mathbf{v}_1, \ldots, \mathbf{v}_k\} = \{c_1 \mathbf{v}_1 + \cdots + c_k \mathbf{v}_k \mid c_i \in \mathbb{R}\}

Euclidean Norm

\|\mathbf{v}\| = \sqrt{v_1^2 + v_2^2 + \cdots + v_n^2} = \sqrt{\sum_{i=1}^{n} v_i^2}

Distance Formula

d(\mathbf{a}, \mathbf{b}) = \|\mathbf{a} - \mathbf{b}\| = \sqrt{(a_1 - b_1)^2 + (a_2 - b_2)^2 + \cdots + (a_n - b_n)^2}

Vector Normalization

\hat{\mathbf{v}} = \frac{\mathbf{v}}{\|\mathbf{v}\|}

Norm Scaling Property

\|c\mathbf{v}\| = |c|\,\|\mathbf{v}\|

Triangle Inequality

\|\mathbf{a} + \mathbf{b}\| \leq \|\mathbf{a}\| + \|\mathbf{b}\|

Cauchy-Schwarz Inequality

|\mathbf{a} \cdot \mathbf{b}| \leq \|\mathbf{a}\|\,\|\mathbf{b}\|

Dot Product (Algebraic)

\mathbf{a} \cdot \mathbf{b} = a_1 b_1 + a_2 b_2 + \cdots + a_n b_n = \sum_{i=1}^{n} a_i b_i

Dot Product (Geometric)

\mathbf{a} \cdot \mathbf{b} = \|\mathbf{a}\|\,\|\mathbf{b}\|\cos\theta

Angle Between Vectors

\cos\theta = \frac{\mathbf{a} \cdot \mathbf{b}}{\|\mathbf{a}\|\,\|\mathbf{b}\|}

Self Dot Product

\mathbf{v} \cdot \mathbf{v} = v_1^2 + v_2^2 + \cdots + v_n^2 = \|\mathbf{v}\|^2

Orthogonality Condition

\mathbf{a} \cdot \mathbf{b} = 0 \iff \mathbf{a} \perp \mathbf{b}

Scalar Projection

\text{comp}_{\mathbf{b}}\,\mathbf{a} = \frac{\mathbf{a} \cdot \mathbf{b}}{\|\mathbf{b}\|}

Vector Projection

\text{proj}_{\mathbf{b}}\,\mathbf{a} = \frac{\mathbf{a} \cdot \mathbf{b}}{\|\mathbf{b}\|^2}\,\mathbf{b} = \frac{\mathbf{a} \cdot \mathbf{b}}{\mathbf{b} \cdot \mathbf{b}}\,\mathbf{b}

Orthogonal Decomposition

\mathbf{a} = \text{proj}_{\mathbf{b}}\,\mathbf{a} + \mathbf{a}_{\perp}, \quad \mathbf{a}_{\perp} = \mathbf{a} - \text{proj}_{\mathbf{b}}\,\mathbf{a}

Cross Product (Component Form)

\mathbf{a} \times \mathbf{b} = (a_2 b_3 - a_3 b_2,\ a_3 b_1 - a_1 b_3,\ a_1 b_2 - a_2 b_1)

Cross Product (Determinant Form)

\mathbf{a} \times \mathbf{b} = \begin{vmatrix} \mathbf{i} & \mathbf{j} & \mathbf{k} \\ a_1 & a_2 & a_3 \\ b_1 & b_2 & b_3 \end{vmatrix}

Cross Product Magnitude

\|\mathbf{a} \times \mathbf{b}\| = \|\mathbf{a}\|\,\|\mathbf{b}\|\sin\theta

Standard Basis Cross Products

\mathbf{i} \times \mathbf{j} = \mathbf{k}, \quad \mathbf{j} \times \mathbf{k} = \mathbf{i}, \quad \mathbf{k} \times \mathbf{i} = \mathbf{j}

Cross Product Anti-Commutativity

\mathbf{a} \times \mathbf{b} = -(\mathbf{b} \times \mathbf{a})

Parallelism Test (Cross Product)

\mathbf{a} \times \mathbf{b} = \mathbf{0} \iff \mathbf{a} \parallel \mathbf{b}

Vector Triple Product

\mathbf{a} \times (\mathbf{b} \times \mathbf{c}) = (\mathbf{a} \cdot \mathbf{c})\,\mathbf{b} - (\mathbf{a} \cdot \mathbf{b})\,\mathbf{c}

Lagrange Identity

(\mathbf{a} \times \mathbf{b}) \cdot (\mathbf{c} \times \mathbf{d}) = (\mathbf{a} \cdot \mathbf{c})(\mathbf{b} \cdot \mathbf{d}) - (\mathbf{a} \cdot \mathbf{d})(\mathbf{b} \cdot \mathbf{c})

Scalar Triple Product

\mathbf{a} \cdot (\mathbf{b} \times \mathbf{c}) = \begin{vmatrix} a_1 & a_2 & a_3 \\ b_1 & b_2 & b_3 \\ c_1 & c_2 & c_3 \end{vmatrix}

Parallelogram Area

\text{Area} = \|\mathbf{a} \times \mathbf{b}\|

Parallelepiped Volume

V = |\mathbf{a} \cdot (\mathbf{b} \times \mathbf{c})|

Pyramid Volume

V = \tfrac{1}{6}|\mathbf{a} \cdot (\mathbf{b} \times \mathbf{c})|

Vector Space Axioms

\begin{aligned} &\text{For all } \mathbf{u}, \mathbf{v}, \mathbf{w} \in V \text{ and all } c, d \in \mathbb{F}: \\ &(1)\ \mathbf{u} + \mathbf{v} \in V \quad (2)\ \mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u} \\ &(3)\ (\mathbf{u} + \mathbf{v}) + \mathbf{w} = \mathbf{u} + (\mathbf{v} + \mathbf{w}) \\ &(4)\ \exists\, \mathbf{0} \in V: \mathbf{v} + \mathbf{0} = \mathbf{v} \\ &(5)\ \exists\, -\mathbf{v} \in V: \mathbf{v} + (-\mathbf{v}) = \mathbf{0} \\ &(6)\ c\mathbf{v} \in V \quad (7)\ c(d\mathbf{v}) = (cd)\mathbf{v} \\ &(8)\ c(\mathbf{u} + \mathbf{v}) = c\mathbf{u} + c\mathbf{v} \\ &(9)\ (c + d)\mathbf{v} = c\mathbf{v} + d\mathbf{v} \quad (10)\ 1\mathbf{v} = \mathbf{v} \end{aligned}

Scalar-Zero Property

0\mathbf{v} = \mathbf{0}, \quad c\mathbf{0} = \mathbf{0}

Negative One Scalar

(-1)\mathbf{v} = -\mathbf{v}

Subspace Test

W \subseteq V \text{ is a subspace} \iff \begin{cases} W \neq \emptyset \\ \mathbf{u}, \mathbf{v} \in W \Rightarrow \mathbf{u} + \mathbf{v} \in W \\ \mathbf{v} \in W,\ c \in \mathbb{F} \Rightarrow c\mathbf{v} \in W \end{cases}

Subspace Test Combined

W \subseteq V \text{ is a subspace} \iff W \neq \emptyset \text{ and } c\mathbf{u} + d\mathbf{v} \in W \text{ for all } \mathbf{u}, \mathbf{v} \in W,\ c, d \in \mathbb{F}

Span (Set Definition)

\text{Span}\{\mathbf{v}_1, \ldots, \mathbf{v}_k\} = \left\{c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \cdots + c_k\mathbf{v}_k \mid c_i \in \mathbb{F}\right\}

Span Membership Criterion

\mathbf{b} \in \text{Span}\{\mathbf{v}_1, \ldots, \mathbf{v}_k\} \iff A\mathbf{c} = \mathbf{b} \text{ is consistent}

Span Is Smallest Subspace

\text{Span}(K) = \bigcap_{\substack{W \text{ subspace} \\ K \subseteq W}} W

Linear Independence Equation

\{\mathbf{v}_1, \ldots, \mathbf{v}_k\} \text{ is independent} \iff \bigl(c_1\mathbf{v}_1 + \cdots + c_k\mathbf{v}_k = \mathbf{0} \Rightarrow c_1 = \cdots = c_k = 0\bigr)

Linear Independence Matrix Test

\{\mathbf{v}_1, \ldots, \mathbf{v}_k\} \subset \mathbb{R}^m \text{ is independent} \iff A\mathbf{c} = \mathbf{0} \text{ has only the trivial solution}

Linear Independence Determinant Test

\{\mathbf{v}_1, \ldots, \mathbf{v}_n\} \subset \mathbb{R}^n \text{ is independent} \iff \det[\mathbf{v}_1\ \cdots\ \mathbf{v}_n] \neq 0

Max Independent Set Size

|S| > \dim V \Rightarrow S \text{ is dependent}

Wronskian Test

W(f_1, \ldots, f_n)(x) = \det\begin{pmatrix} f_1(x) & \cdots & f_n(x) \\ f_1'(x) & \cdots & f_n'(x) \\ \vdots & \ddots & \vdots \\ f_1^{(n-1)}(x) & \cdots & f_n^{(n-1)}(x) \end{pmatrix}

Basis Definition

\mathcal{B} = \{\mathbf{v}_1, \ldots, \mathbf{v}_n\} \text{ is a basis for } V \iff \begin{cases} \mathcal{B} \text{ is linearly independent} \\ \text{Span}(\mathcal{B}) = V \end{cases}

Unique Basis Representation

\forall\, \mathbf{v} \in V,\ \exists!\, (c_1, \ldots, c_n): \mathbf{v} = c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \cdots + c_n\mathbf{v}_n

Coordinate Vector

[\mathbf{v}]_\mathcal{B} = \begin{pmatrix} c_1 \\ c_2 \\ \vdots \\ c_n \end{pmatrix} \quad \text{where } \mathbf{v} = c_1\mathbf{v}_1 + \cdots + c_n\mathbf{v}_n

Standard Basis (Rn)

\mathbf{e}_i = (0, \ldots, 0, \underset{i\text{-th}}{1}, 0, \ldots, 0), \quad i = 1, \ldots, n

Change of Basis Formula

[\mathbf{v}]_\mathcal{C} = P_{\mathcal{C} \leftarrow \mathcal{B}}\, [\mathbf{v}]_\mathcal{B}

Change of Basis Inverse

P_{\mathcal{B} \leftarrow \mathcal{C}} = \bigl(P_{\mathcal{C} \leftarrow \mathcal{B}}\bigr)^{-1}

Coordinate Map Linearity

[\mathbf{u} + \mathbf{v}]_\mathcal{B} = [\mathbf{u}]_\mathcal{B} + [\mathbf{v}]_\mathcal{B}, \qquad [c\mathbf{v}]_\mathcal{B} = c\,[\mathbf{v}]_\mathcal{B}

Dimension Definition

\dim(V) = |\mathcal{B}| \quad \text{for any basis } \mathcal{B} \text{ of } V

Subspace Dimension Inequality

W \subseteq V \Rightarrow \dim(W) \leq \dim(V), \quad \text{with equality} \iff W = V

Dimension Sum Formula

\dim(W_1 + W_2) = \dim(W_1) + \dim(W_2) - \dim(W_1 \cap W_2)

Direct Sum Criterion

V = W_1 \oplus W_2 \iff V = W_1 + W_2 \text{ and } W_1 \cap W_2 = \{\mathbf{0}\}

Direct Sum Dimension

V = W_1 \oplus W_2 \Rightarrow \dim(V) = \dim(W_1) + \dim(W_2)

Rank-Nullity Theorem (Matrix Form)

\dim(\text{Col}\,A) + \dim(\text{Null}\,A) = n

Four Fundamental Subspaces Dimensions

\begin{aligned} \dim(\text{Col}\,A) &= r & \dim(\text{Row}\,A) &= r \\ \dim(\text{Null}\,A) &= n - r & \dim(\text{Null}\,A^T) &= m - r \end{aligned}

Row Rank Equals Column Rank

\dim(\text{Row}\,A) = \dim(\text{Col}\,A) = \text{rank}(A)

Matrix Equality

A = B \iff a_{ij} = b_{ij} \text{ for all } i, j

Matrix Addition

(A + B)_{ij} = a_{ij} + b_{ij}

Matrix Subtraction

(A - B)_{ij} = a_{ij} - b_{ij}

Scalar Multiplication of Matrices

(cA)_{ij} = c \cdot a_{ij}

Matrix Multiplication

(AB)_{ij} = \sum_{k=1}^{n} a_{ik}\, b_{kj}

Matrix-Vector Product (Column Form)

A\mathbf{x} = x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 + \cdots + x_n \mathbf{a}_n

Matrix Multiplication Associativity

(AB)C = A(BC)

Matrix Multiplication Distributivity

A(B + C) = AB + AC, \qquad (A + B)C = AC + BC

Matrix Multiplication Non-Commutativity

AB \neq BA \quad \text{in general}

Matrix Power

A^0 = I, \qquad A^k = \underbrace{A \cdot A \cdots A}_{k \text{ factors}}, \qquad A^{-k} = (A^{-1})^k

Transpose Definition

(A^T)_{ij} = a_{ji}

Transpose Involution

(A^T)^T = A

Transpose of Sum

(A + B)^T = A^T + B^T

Transpose of Scalar Multiple

(cA)^T = c\, A^T

Transpose of Product

(AB)^T = B^T A^T

Symmetric Matrix Definition

A = A^T \iff a_{ij} = a_{ji} \text{ for all } i, j

Skew-Symmetric Matrix Definition

A^T = -A \iff a_{ij} = -a_{ji}

Symmetric Skew Decomposition

A = \tfrac{1}{2}(A + A^T) + \tfrac{1}{2}(A - A^T)

Gram Matrix Symmetry

(A^T A)^T = A^T A, \qquad (A A^T)^T = A A^T

Identity Matrix Definition

I_n = [\delta_{ij}], \qquad \delta_{ij} = \begin{cases} 1 & i = j \\ 0 & i \neq j \end{cases}

Identity Matrix Property

AI = IA = A

Diagonal Matrix Definition

D = \operatorname{diag}(d_1, d_2, \ldots, d_n) \iff d_{ij} = 0 \text{ for } i \neq j

Diagonal Matrix Power

D^k = \operatorname{diag}(d_1^k, d_2^k, \ldots, d_n^k)

Diagonal Matrix Determinant

\det\bigl(\operatorname{diag}(d_1, \ldots, d_n)\bigr) = d_1 d_2 \cdots d_n

Triangular Matrix Determinant

\det(T) = t_{11} \, t_{22} \cdots t_{nn}

Orthogonal Matrix Definition

Q^T Q = Q Q^T = I

Orthogonal Matrix Determinant

\det(Q) = \pm 1

Idempotent Matrix Definition

A^2 = A

Idempotent Rank Trace

A^2 = A \implies \operatorname{rank}(A) = \operatorname{tr}(A)

Nilpotent Matrix Definition

A^k = O \quad \text{for some } k \geq 1

Neumann Series Nilpotent

A^k = O \implies (I - A)^{-1} = I + A + A^2 + \cdots + A^{k-1}

Involutory Matrix Definition

A^2 = I

Cross Product Skew Matrix

\mathbf{a} \times \mathbf{b} = [\mathbf{a}]_\times \mathbf{b}, \qquad [\mathbf{a}]_\times = \begin{pmatrix} 0 & -a_3 & a_2 \\ a_3 & 0 & -a_1 \\ -a_2 & a_1 & 0 \end{pmatrix}

Inverse Definition

A A^{-1} = A^{-1} A = I

Inverse 2x2 Formula

\begin{pmatrix} a & b \\ c & d \end{pmatrix}^{-1} = \frac{1}{ad - bc}\begin{pmatrix} d & -b \\ -c & a \end{pmatrix}

Inverse via Adjugate

A^{-1} = \frac{1}{\det(A)}\, \operatorname{adj}(A)

Inverse via Row Reduction

[A \mid I] \;\xrightarrow{\text{row ops}}\; [I \mid A^{-1}]

Inverse Involution

(A^{-1})^{-1} = A

Inverse of Product

(AB)^{-1} = B^{-1} A^{-1}

Inverse of Transpose

(A^T)^{-1} = (A^{-1})^T

Inverse of Scalar Multiple

(cA)^{-1} = \frac{1}{c}\, A^{-1}

Inverse of Power

(A^k)^{-1} = (A^{-1})^k = A^{-k}

Determinant of Inverse

\det(A^{-1}) = \frac{1}{\det(A)}

Diagonal Matrix Inverse

\operatorname{diag}(d_1, \ldots, d_n)^{-1} = \operatorname{diag}\!\left(\frac{1}{d_1}, \ldots, \frac{1}{d_n}\right)

Orthogonal Matrix Inverse

Q^{-1} = Q^T

Solve System via Inverse

A\mathbf{x} = \mathbf{b} \implies \mathbf{x} = A^{-1}\mathbf{b}

Invertible Matrix Theorem

\begin{aligned} \text{For } A \in \mathbb{R}^{n \times n}, \text{ the following are equivalent:} & \\ (1)\ A \text{ is invertible} \quad (2)\ \det(A) \neq 0 \quad (3)\ \operatorname{rank}(A) = n & \\ (4)\ \text{columns of } A \text{ are linearly independent} & \\ (5)\ \text{rows of } A \text{ are linearly independent} & \\ (6)\ \text{columns of } A \text{ span } \mathbb{R}^n \quad (7)\ \text{columns form a basis of } \mathbb{R}^n & \\ (8)\ A\mathbf{x} = \mathbf{0} \text{ has only the trivial solution} & \\ (9)\ A\mathbf{x} = \mathbf{b} \text{ has a unique solution for every } \mathbf{b} & \\ (10)\ \operatorname{Null}(A) = \{\mathbf{0}\} & \\ (11)\ \operatorname{rref}(A) = I \quad (12)\ A \text{ is a product of elementary matrices} & \\ (13)\ 0 \text{ is not an eigenvalue of } A & \end{aligned}

Singular Matrix Definition

A \text{ singular} \iff \det(A) = 0 \iff \operatorname{rank}(A) < n

Rank Bounds

0 \leq \operatorname{rank}(A) \leq \min(m, n)

Rank of Transpose

\operatorname{rank}(A^T) = \operatorname{rank}(A)

Rank Product Inequality

\operatorname{rank}(AB) \leq \min\bigl(\operatorname{rank}(A), \operatorname{rank}(B)\bigr)

Sylvester Rank Inequality

\operatorname{rank}(A) + \operatorname{rank}(B) - n \leq \operatorname{rank}(AB)

Rank Sum Inequality

\operatorname{rank}(A + B) \leq \operatorname{rank}(A) + \operatorname{rank}(B)

Rank Invariance Invertible

\operatorname{rank}(PAQ) = \operatorname{rank}(A)

Gram Rank Identity

\operatorname{rank}(A^T A) = \operatorname{rank}(A A^T) = \operatorname{rank}(A)

Rank-One Outer Product

A = \mathbf{u}\mathbf{v}^T \implies \operatorname{rank}(A) = 1 \quad (\mathbf{u}, \mathbf{v} \neq \mathbf{0})

Trace Definition

\operatorname{tr}(A) = \sum_{i=1}^{n} a_{ii} = a_{11} + a_{22} + \cdots + a_{nn}

Trace Linearity

\operatorname{tr}(cA + dB) = c\operatorname{tr}(A) + d\operatorname{tr}(B)

Trace of Transpose

\operatorname{tr}(A^T) = \operatorname{tr}(A)

Trace Cyclic Property

\operatorname{tr}(AB) = \operatorname{tr}(BA)

Trace Sum of Eigenvalues

\operatorname{tr}(A) = \sum_{i=1}^{n} \lambda_i

Trace Similarity Invariance

\operatorname{tr}(P^{-1}AP) = \operatorname{tr}(A)

Trace of Commutator

\operatorname{tr}(AB - BA) = 0

Trace Symmetric Skew Product

\operatorname{tr}(SK) = 0 \quad (S^T = S, \; K^T = -K)

Trace Orthonormal Basis

\operatorname{tr}(A) = \sum_{i=1}^{n} \mathbf{q}_i^T A \mathbf{q}_i

Frobenius Inner Product

\langle A, B \rangle_F = \operatorname{tr}(A^T B) = \sum_{i,j} a_{ij} b_{ij}

Frobenius Norm

\|A\|_F = \sqrt{\operatorname{tr}(A^T A)} = \sqrt{\sum_{i,j} a_{ij}^2}

Determinant 2x2

\det\begin{pmatrix} a & b \\ c & d \end{pmatrix} = ad - bc

Determinant 3x3

\det(A) = a_{11}(a_{22}a_{33} - a_{23}a_{32}) - a_{12}(a_{21}a_{33} - a_{23}a_{31}) + a_{13}(a_{21}a_{32} - a_{22}a_{31})

Determinant Recursive Definition

\det(A) = \begin{cases} a_{11} & n = 1 \\ \displaystyle\sum_{j=1}^{n} (-1)^{1+j} \, a_{1j} \, M_{1j} & n \geq 2 \end{cases}

Determinant Permutation Formula

\det(A) = \sum_{\sigma \in S_n} \operatorname{sgn}(\sigma) \, a_{\sigma(1),1} \, a_{\sigma(2),2} \cdots a_{\sigma(n),n}

Minor Definition

M_{ij} = \det\!\left(A^{(i,j)}\right)

Cofactor Definition

C_{ij} = (-1)^{i+j} \, M_{ij}

Cofactor Matrix Definition

\operatorname{cof}(A) = \bigl[C_{ij}\bigr]_{n \times n}

Adjugate Definition

\operatorname{adj}(A) = \operatorname{cof}(A)^T

Laplace Row Expansion

\det(A) = \sum_{j=1}^{n} a_{ij} \, C_{ij} \qquad \text{for any fixed row } i

Laplace Column Expansion

\det(A) = \sum_{i=1}^{n} a_{ij} \, C_{ij} \qquad \text{for any fixed column } j

Determinant Row Swap

\det(B) = -\det(A)

Determinant Row Scaling

\det(B) = k \, \det(A)

Determinant Row Addition

\det(B) = \det(A)

Determinant of Transpose

\det(A^T) = \det(A)

Determinant of Product

\det(AB) = \det(A) \, \det(B)

Determinant of Scalar Multiple

\det(kA) = k^n \, \det(A)

Determinant of Power

\det(A^k) = \bigl(\det(A)\bigr)^k

Determinant of Identity

\det(I_n) = 1

Block Triangular Determinant

\det\begin{pmatrix} A_{11} & A_{12} \\ 0 & A_{22} \end{pmatrix} = \det(A_{11}) \, \det(A_{22})

Vandermonde Determinant

\det(V) = \prod_{1 \leq i < j \leq n} (x_j - x_i)

Adjugate Identity

A \cdot \operatorname{adj}(A) = \operatorname{adj}(A) \cdot A = \det(A) \, I

Cramers Rule

x_i = \frac{\det(A_i)}{\det(A)}

Determinant Signed Area 2D

\text{signed area}(\mathbf{u}, \mathbf{v}) = \det\begin{pmatrix} \mathbf{u} & \mathbf{v} \end{pmatrix} = ad - bc

Determinant Signed Volume 3D

\text{signed volume}(\mathbf{a}, \mathbf{b}, \mathbf{c}) = \det\begin{pmatrix} \mathbf{a} & \mathbf{b} & \mathbf{c} \end{pmatrix} = \mathbf{a} \cdot (\mathbf{b} \times \mathbf{c})

Determinant Volume Scaling Factor

\operatorname{vol}\bigl(A(S)\bigr) = |\det(A)| \cdot \operatorname{vol}(S)

Triangle Area via Determinant

\text{Area} = \frac{1}{2} \left| \det\begin{pmatrix} x_2 - x_1 & x_3 - x_1 \\ y_2 - y_1 & y_3 - y_1 \end{pmatrix} \right|

Tetrahedron Volume via Determinant

V = \frac{1}{6} \left| \det\begin{pmatrix} \mathbf{e}_1 & \mathbf{e}_2 & \mathbf{e}_3 \end{pmatrix} \right|

Determinant Product of Eigenvalues

\det(A) = \lambda_1 \, \lambda_2 \cdots \lambda_n

Linear Equation Standard Form

a_1 x_1 + a_2 x_2 + \cdots + a_n x_n = b

Linear System Matrix Form

A\mathbf{x} = \mathbf{b}

Vector Equation Form

x_1 \mathbf{a}_1 + x_2 \mathbf{a}_2 + \cdots + x_n \mathbf{a}_n = \mathbf{b}

Augmented Matrix Construction

[A \mid \mathbf{b}] = \left(\begin{array}{cccc|c} a_{11} & a_{12} & \cdots & a_{1n} & b_1 \\ a_{21} & a_{22} & \cdots & a_{2n} & b_2 \\ \vdots & \vdots & \ddots & \vdots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} & b_m \end{array}\right)

Row Echelon Form Definition

\text{REF: } \begin{pmatrix} \boxed{\ast} & \bullet & \bullet & \bullet & \bullet \\ 0 & \boxed{\ast} & \bullet & \bullet & \bullet \\ 0 & 0 & 0 & \boxed{\ast} & \bullet \\ 0 & 0 & 0 & 0 & 0 \end{pmatrix}

Reduced Row Echelon Form Definition

\text{RREF: } \begin{pmatrix} \boxed{1} & 0 & \bullet & 0 & \bullet \\ 0 & \boxed{1} & \bullet & 0 & \bullet \\ 0 & 0 & 0 & \boxed{1} & \bullet \\ 0 & 0 & 0 & 0 & 0 \end{pmatrix}

RREF Uniqueness

\text{RREF}(A) \text{ is unique}

Pivot Definition

\text{pivot} = \text{leading nonzero entry of a row in echelon form}

Elementary Row Operations

\begin{aligned} R_i &\leftrightarrow R_j \quad \text{(swap)} \\ kR_i &\to R_i, \quad k \neq 0 \quad \text{(scaling)} \\ R_i + cR_j &\to R_i \quad \text{(addition)} \end{aligned}

Row Equivalence Preserves Solutions

[A \mid \mathbf{b}] \sim [A' \mid \mathbf{b}'] \;\Longrightarrow\; \text{Sol}(A\mathbf{x} = \mathbf{b}) = \text{Sol}(A'\mathbf{x} = \mathbf{b}')

Free Variables Count

\text{(number of free variables)} = n - \text{rank}(A)

Solvability Rank Criterion

A\mathbf{x} = \mathbf{b} \text{ is consistent} \iff \text{rank}(A) = \text{rank}([A \mid \mathbf{b}])

Solution Structure Decomposition

\mathbf{x} = \mathbf{x}_p + \mathbf{x}_h, \quad \mathbf{x}_h \in \text{Null}(A)

Homogeneous Solution Space Dimension

\dim \text{Null}(A) = n - \text{rank}(A)

Underdetermined Homogeneous Has Nontrivial

n > m \;\Longrightarrow\; A\mathbf{x} = \mathbf{0} \text{ has a nontrivial solution}

Linear Transformation Definition

T(c\mathbf{u} + d\mathbf{v}) = cT(\mathbf{u}) + dT(\mathbf{v})

Zero Vector Preservation

T(\mathbf{0}_V) = \mathbf{0}_W

Composition Is Linear

(S \circ T)(\mathbf{u}) = S(T(\mathbf{u})), \qquad [S \circ T] = [S]\,[T]

Standard Matrix

A = \bigl[\,T(\mathbf{e}_1) \;\; T(\mathbf{e}_2) \;\; \cdots \;\; T(\mathbf{e}_n)\,\bigr]

Linear Map as Matrix Multiplication

T(\mathbf{x}) = A\mathbf{x} \quad \text{for every } \mathbf{x} \in \mathbb{R}^n

Matrix Representation Abstract Bases

[T]_{\mathcal{C} \leftarrow \mathcal{B}} = \bigl[\,[T(\mathbf{v}_1)]_{\mathcal{C}} \;\; \cdots \;\; [T(\mathbf{v}_n)]_{\mathcal{C}}\,\bigr]

Image Definition

\text{Im}(T) = \{T(\mathbf{v}) : \mathbf{v} \in V\} \subseteq W

Kernel Definition

\ker(T) = \{\mathbf{v} \in V : T(\mathbf{v}) = \mathbf{0}\} \subseteq V

Injectivity Kernel Criterion

T \text{ injective} \iff \ker(T) = \{\mathbf{0}\}

Rank-Nullity for Maps

\dim\text{Im}(T) + \dim\ker(T) = \dim V

Bijectivity Equal Dim Case

\dim V = \dim W \Rightarrow \bigl(T \text{ injective} \iff T \text{ surjective} \iff T \text{ bijective}\bigr)

Similarity Relation

A' = P^{-1} A P

Similarity Invariants

A' = P^{-1}AP \Rightarrow \begin{cases} \det(A') = \det(A) \\ \text{tr}(A') = \text{tr}(A) \\ \text{rank}(A') = \text{rank}(A) \\ \text{eigenvalues}(A') = \text{eigenvalues}(A) \end{cases}

Diagonalization Formula

A = P D P^{-1}, \quad D = \operatorname{diag}(\lambda_1, \ldots, \lambda_n), \quad P = [\mathbf{v}_1 \;\cdots\; \mathbf{v}_n]

Rotation Matrix 2D

R_\theta = \begin{pmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{pmatrix}

Rotation Matrices 3D

R_x(\theta) = \begin{pmatrix} 1 & 0 & 0 \\ 0 & \cos\theta & -\sin\theta \\ 0 & \sin\theta & \cos\theta \end{pmatrix}, \;\; R_y(\theta) = \begin{pmatrix} \cos\theta & 0 & \sin\theta \\ 0 & 1 & 0 \\ -\sin\theta & 0 & \cos\theta \end{pmatrix}, \;\; R_z(\theta) = \begin{pmatrix} \cos\theta & -\sin\theta & 0 \\ \sin\theta & \cos\theta & 0 \\ 0 & 0 & 1 \end{pmatrix}

Reflection Across Line 2D

H_\alpha = \begin{pmatrix} \cos 2\alpha & \sin 2\alpha \\ \sin 2\alpha & -\cos 2\alpha \end{pmatrix}

Householder Reflection

H = I - 2\,\mathbf{n}\mathbf{n}^T

Projection onto Line

P = \frac{\mathbf{u}\mathbf{u}^T}{\mathbf{u}^T\mathbf{u}}

Projection onto Plane

P = I - \frac{\mathbf{n}\mathbf{n}^T}{\mathbf{n}^T\mathbf{n}}

Shear Matrix

\text{Shear}_x = \begin{pmatrix} 1 & k \\ 0 & 1 \end{pmatrix}, \qquad \text{Shear}_y = \begin{pmatrix} 1 & 0 \\ k & 1 \end{pmatrix}

Eigenvalue Definition

A\mathbf{v} = \lambda\mathbf{v}, \quad \mathbf{v} \neq \mathbf{0}

Characteristic Equation

\det(A - \lambda I) = 0

Eigenspace

E_\lambda = \text{Null}(A - \lambda I) = \{\mathbf{v} : A\mathbf{v} = \lambda\mathbf{v}\}

Characteristic Polynomial

p(\lambda) = \det(A - \lambda I)

Characteristic Polynomial 2x2

p(\lambda) = \lambda^2 - \text{tr}(A)\,\lambda + \det(A)

Cayley-Hamilton

p(A) = O

Multiplicity Inequality

1 \leq m_g(\lambda) \leq m_a(\lambda)

Independence of Distinct Eigenvectors

\lambda_1, \ldots, \lambda_k \text{ distinct} \Rightarrow \{\mathbf{v}_1, \ldots, \mathbf{v}_k\} \text{ linearly independent}

Eigenvalue of Power

A\mathbf{v} = \lambda\mathbf{v} \Rightarrow A^k\mathbf{v} = \lambda^k\mathbf{v}

Eigenvalue of Inverse

A\mathbf{v} = \lambda\mathbf{v} \Rightarrow A^{-1}\mathbf{v} = \frac{1}{\lambda}\mathbf{v}

Eigenvalue of Polynomial

q(A)\mathbf{v} = q(\lambda)\mathbf{v}

Eigenvalue Shift

A\mathbf{v} = \lambda\mathbf{v} \Rightarrow (A + cI)\mathbf{v} = (\lambda + c)\mathbf{v}

Diagonalizability Condition

A \text{ diagonalizable} \iff m_g(\lambda) = m_a(\lambda) \text{ for every eigenvalue } \lambda

Distinct Eigenvalues Imply Diagonalizable

A \text{ has } n \text{ distinct eigenvalues} \Rightarrow A \text{ is diagonalizable}

Special Matrix Eigenvalue Restrictions

\begin{aligned} \text{symmetric (real)}: \;& \lambda \in \mathbb{R} \\ \text{skew-symmetric (real)}: \;& \lambda = 0 \text{ or } \lambda \in i\mathbb{R} \\ \text{orthogonal}: \;& |\lambda| = 1 \\ \text{idempotent}: \;& \lambda \in \{0, 1\} \\ \text{nilpotent}: \;& \lambda = 0 \\ \text{involutory}: \;& \lambda \in \{-1, +1\} \\ \text{positive definite}: \;& \lambda > 0 \end{aligned}

Spectral Theorem

A = A^T \Rightarrow A = Q D Q^T, \quad Q^T Q = I, \quad D = \text{diag}(\lambda_1, \ldots, \lambda_n) \in \mathbb{R}^{n \times n}

Spectral Decomposition

A = \sum_{i=1}^{n} \lambda_i \, \mathbf{q}_i \mathbf{q}_i^T

Matrix Exponential

e^{At} = P\, e^{Dt}\, P^{-1} = P \,\text{diag}(e^{\lambda_1 t}, \ldots, e^{\lambda_n t})\, P^{-1}

Complex Conjugate Pairs

A \in \mathbb{R}^{n \times n}, \;\; A\mathbf{v} = \lambda\mathbf{v} \Rightarrow A\bar{\mathbf{v}} = \bar{\lambda}\bar{\mathbf{v}}

Discriminant Classification 2x2

\Delta = \text{tr}(A)^2 - 4\det(A) \quad \begin{cases} \Delta > 0: & \text{two distinct real eigenvalues} \\ \Delta = 0: & \text{one repeated real eigenvalue} \\ \Delta < 0: & \text{complex conjugate pair} \end{cases}

Real Canonical Form 2x2

\lambda = a \pm bi \Rightarrow P^{-1}AP = \begin{pmatrix} a & -b \\ b & a \end{pmatrix} = r\begin{pmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{pmatrix}

Cauchy-Schwarz Inequality (General)

|\mathbf{u} \cdot \mathbf{v}| \leq \|\mathbf{u}\| \, \|\mathbf{v}\|

Triangle Inequality (Inner Product)

\|\mathbf{u} + \mathbf{v}\| \leq \|\mathbf{u}\| + \|\mathbf{v}\|

Pythagorean Theorem

\mathbf{u} \cdot \mathbf{v} = 0 \Rightarrow \|\mathbf{u} + \mathbf{v}\|^2 = \|\mathbf{u}\|^2 + \|\mathbf{v}\|^2

Inner Product Axioms

\begin{aligned} \text{Symmetry:} \;& \langle\mathbf{u},\mathbf{v}\rangle = \langle\mathbf{v},\mathbf{u}\rangle \\ \text{Linearity:} \;& \langle c\mathbf{u}+d\mathbf{w},\mathbf{v}\rangle = c\langle\mathbf{u},\mathbf{v}\rangle + d\langle\mathbf{w},\mathbf{v}\rangle \\ \text{Positive definite:} \;& \langle\mathbf{v},\mathbf{v}\rangle > 0 \text{ for } \mathbf{v} \neq \mathbf{0} \end{aligned}

Distance Formula (Inner Product)

d(\mathbf{u}, \mathbf{v}) = \|\mathbf{u} - \mathbf{v}\| = \sqrt{\sum_{i=1}^{n} (u_i - v_i)^2}

Orthogonal Complement Definition

W^\perp = \{\mathbf{v} \in \mathbb{R}^n : \mathbf{v} \cdot \mathbf{w} = 0 \text{ for all } \mathbf{w} \in W\}

Complement Dimension Sum

\dim(W) + \dim(W^\perp) = n

Orthogonal Decomposition (Subspace)

\mathbf{v} = \hat{\mathbf{v}} + \mathbf{z}, \quad \hat{\mathbf{v}} \in W, \quad \mathbf{z} \in W^\perp

Orthogonal Set Independence

\mathbf{v}_i \cdot \mathbf{v}_j = 0 \;\; (i \neq j), \;\; \mathbf{v}_i \neq \mathbf{0} \;\Rightarrow\; \{\mathbf{v}_1, \ldots, \mathbf{v}_k\} \text{ linearly independent}

Orthonormal Set

\mathbf{q}_i \cdot \mathbf{q}_j = \delta_{ij} = \begin{cases} 1 & i = j \\ 0 & i \neq j \end{cases}

Coordinates via Orthonormal Basis

\mathbf{x} = \sum_{i=1}^{n} (\mathbf{x} \cdot \mathbf{q}_i)\,\mathbf{q}_i, \qquad c_i = \mathbf{x} \cdot \mathbf{q}_i

Parseval Identity

\|\mathbf{x}\|^2 = \sum_{i=1}^{n} (\mathbf{x} \cdot \mathbf{q}_i)^2 = \sum_{i=1}^{n} c_i^2

Projection onto Subspace

\hat{\mathbf{b}} = A(A^TA)^{-1}A^T\,\mathbf{b}, \qquad P = A(A^TA)^{-1}A^T

Projection onto Orthonormal Basis

\text{proj}_W\,\mathbf{b} = \sum_{i=1}^{k} (\mathbf{q}_i \cdot \mathbf{b})\,\mathbf{q}_i

Orthonormal Columns Projection

P = QQ^T, \qquad Q^TQ = I_k

Projection Matrix Properties

P^2 = P, \qquad P^T = P

Complementary Projection

P_{W^\perp} = I - P_W, \qquad P_W\mathbf{b} + (I - P_W)\mathbf{b} = \mathbf{b}

Gram-Schmidt Process

\mathbf{u}_1 = \mathbf{v}_1, \qquad \mathbf{u}_j = \mathbf{v}_j - \sum_{i=1}^{j-1} \frac{\mathbf{u}_i \cdot \mathbf{v}_j}{\mathbf{u}_i \cdot \mathbf{u}_i}\,\mathbf{u}_i, \qquad \mathbf{q}_j = \frac{\mathbf{u}_j}{\|\mathbf{u}_j\|}

Normal Equations

A^TA\,\hat{\mathbf{x}} = A^T\mathbf{b}

Least-Squares Solution

\hat{\mathbf{x}} = (A^TA)^{-1} A^T \mathbf{b}, \qquad A^+ = (A^TA)^{-1}A^T

Least-Squares via QR

A = QR \;\Rightarrow\; R\,\hat{\mathbf{x}} = Q^T\mathbf{b}

LU Decomposition

A = LU

PA LU Partial Pivoting

PA = LU

Determinant via LU

\det(A) = (-1)^s \prod_{i=1}^{n} u_{ii}

LU Solve Steps

A\mathbf{x} = \mathbf{b} \;\Longleftrightarrow\; \begin{cases} L\mathbf{y} = P\mathbf{b} & (\text{forward sub}) \\ U\mathbf{x} = \mathbf{y} & (\text{back sub}) \end{cases}

Cholesky Decomposition

A = L L^T

Cholesky Diagonal Formula

l_{jj} = \sqrt{a_{jj} - \sum_{k=1}^{j-1} l_{jk}^2}

Cholesky Off-Diagonal Formula

l_{ij} = \frac{1}{l_{jj}}\left( a_{ij} - \sum_{k=1}^{j-1} l_{ik} l_{jk} \right), \qquad i > j

QR Decomposition

A = Q R

QR Gram-Schmidt R Entries

R_{ij} = \mathbf{q}_i \cdot \mathbf{a}_j \;\; (i \leq j), \qquad R_{ij} = 0 \;\; (i > j), \qquad R_{jj} = \|\mathbf{u}_j\|

QR Algorithm for Eigenvalues

A_k = Q_k R_k, \qquad A_{k+1} = R_k Q_k

SVD

A = U \Sigma V^T

Singular Values

\sigma_i = \sqrt{\lambda_i(A^TA)} = \sqrt{\lambda_i(AA^T)}

SVD Rank

\text{rank}(A) = \#\{i : \sigma_i > 0\}

SVD Outer Product Form

A = \sum_{i=1}^{r} \sigma_i \, \mathbf{u}_i \mathbf{v}_i^T

Moore-Penrose Pseudoinverse

A^+ = V \Sigma^+ U^T

Eckart-Young Low-Rank Approximation

A_k = \sum_{i=1}^{k} \sigma_i \mathbf{u}_i \mathbf{v}_i^T, \qquad \|A - A_k\|_2 = \sigma_{k+1}, \quad \|A - A_k\|_F = \sqrt{\sum_{i=k+1}^{r}\sigma_i^2}

Condition Number

\kappa(A) = \frac{\sigma_1}{\sigma_r}

Operator Norm

\|A\|_2 = \sigma_1 = \max_{\|\mathbf{x}\|=1} \|A\mathbf{x}\|

Frobenius Norm via Singular Values

\|A\|_F = \sqrt{\sum_{i=1}^{r} \sigma_i^2}

SVD Four Fundamental Subspaces

\begin{aligned} \text{Col}(A) &= \text{Span}\{\mathbf{u}_1, \ldots, \mathbf{u}_r\} \\ \text{Null}(A^T) &= \text{Span}\{\mathbf{u}_{r+1}, \ldots, \mathbf{u}_m\} \\ \text{Row}(A) &= \text{Span}\{\mathbf{v}_1, \ldots, \mathbf{v}_r\} \\ \text{Null}(A) &= \text{Span}\{\mathbf{v}_{r+1}, \ldots, \mathbf{v}_n\} \end{aligned}

Quadratic Form Diagonalization

\mathbf{x}^T A \mathbf{x} = \mathbf{y}^T D \mathbf{y} = \sum_{i=1}^{n} \lambda_i y_i^2, \qquad \mathbf{x} = Q\mathbf{y}

See All Formulas

Learn More

Linear Algebra Terms and Definitions

Discover essential linear algebra definitions that form the mathematical foundation for understanding vectors, matrices, and their relationships. This guide breaks down key terms from basic vector concepts like magnitude and direction to advanced matrix classifications and properties. Each definition includes precise mathematical notation, clear explanations, and visual examples to help grasp the concept. Whether you're learning about vector spaces, exploring matrix types like triangular and symmetric matrices, or studying transformations, this organized reference makes complex linear algebra terminology accessible. The page serves as both a learning tool and a quick reference for students and practitioners, featuring interactive mathematical notation and practical examples throughout.

Vector

An ordered list of $n$ real numbers: $\mathbf{v} = (v_1, v_2, \ldots, v_n) \in \mathbb{R}^n$

Scalar

An element of the underlying field — in standard linear algebra, a real number $c \in \mathbb{R}$

Magnitude (Norm)

The length of a [vector](!/linear-algebra/definitions#vector), measured as its distance from the origin: $$\|\mathbf{v}\| = \sqrt{v_1^2 + v_2^2 + \cdots + v_n^2}$$

Unit Vector

A vector $\hat{\mathbf{u}}$ with $\|\hat{\mathbf{u}}\| = 1$

Dot Product

An operation that takes two [vectors](!/linear-algebra/definitions#vector) and returns a [scalar](!/linear-algebra/definitions#scalar), computed by summing the products of corresponding components: $$\mathbf{u} \cdot \mathbf{v} = u_1 v_1 + u_2 v_2 + \cdots + u_n v_n = \|\mathbf{u}\|\,\|\mathbf{v}\|\cos\theta$$

Cross Product

A binary operation on two vectors in $\mathbb{R}^3$ that produces a vector perpendicular to both inputs: $$\mathbf{u} \times \mathbf{v} = \begin{pmatrix} u_2 v_3 - u_3 v_2 \\ u_3 v_1 - u_1 v_3 \\ u_1 v_2 - u_2 v_1 \end{pmatrix}$$

Linear Combination

A sum of [vectors](!/linear-algebra/definitions#vector), each multiplied by a [scalar](!/linear-algebra/definitions#scalar) coefficient: $$c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \cdots + c_k\mathbf{v}_k$$

Vector Space

A set $V$ equipped with vector addition and scalar multiplication satisfying the [vector space axioms](!/linear-algebra/vector-spaces/axioms)

Subspace

A nonempty subset $W \subseteq V$ that is itself a [vector space](!/linear-algebra/definitions#vector_space) under the same operations

Span

The set of all [linear combinations](!/linear-algebra/definitions#linear_combination) of a given collection of vectors: $$\text{Span}\{\mathbf{v}_1, \ldots, \mathbf{v}_k\} = \{c_1\mathbf{v}_1 + \cdots + c_k\mathbf{v}_k \mid c_i \in \mathbb{R}\}$$

Linear Independence

Vectors $\mathbf{v}_1, \ldots, \mathbf{v}_k$ are linearly independent if the only solution to $$c_1\mathbf{v}_1 + \cdots + c_k\mathbf{v}_k = \mathbf{0}$$ is $c_1 = c_2 = \cdots = c_k = 0$

Basis

A set $\{\mathbf{v}_1, \ldots, \mathbf{v}_n\}$ that is [linearly independent](!/linear-algebra/definitions#linear_independence) and [spans](!/linear-algebra/definitions#span) the entire [vector space](!/linear-algebra/definitions#vector_space)

Dimension

The number of vectors in any [basis](!/linear-algebra/definitions#basis) of a [vector space](!/linear-algebra/definitions#vector_space) $V$, denoted $\dim(V)$

Column Space

The set of all vectors expressible as $A\mathbf{x}$ — equivalently, the [span](!/linear-algebra/definitions#span) of the columns of $A$: $$\text{Col}(A) = \{A\mathbf{x} \mid \mathbf{x} \in \mathbb{R}^n\}$$

Null Space (Kernel)

The set of all solutions to the [homogeneous system](!/linear-algebra/definitions#homogeneous_system) $A\mathbf{x} = \mathbf{0}$: $$\text{Nul}(A) = \{\mathbf{x} \in \mathbb{R}^n \mid A\mathbf{x} = \mathbf{0}\}$$

Row Space

The [span](!/linear-algebra/definitions#span) of the rows of a [matrix](!/linear-algebra/definitions#matrix), equivalently the [column space](!/linear-algebra/definitions#column_space) of its transpose: $$\text{Row}(A) = \text{Col}(A^T)$$

Left Null Space

The [null space](!/linear-algebra/definitions#null_space) of the transpose $A^T$ — the set of all vectors $\mathbf{y}$ satisfying $A^T\mathbf{y} = \mathbf{0}$: $$\text{Nul}(A^T) = \{\mathbf{y} \in \mathbb{R}^m \mid A^T\mathbf{y} = \mathbf{0}\}$$

Matrix

A rectangular array of numbers with $m$ rows and $n$ columns: $A \in \mathbb{R}^{m \times n}$

Square Matrix

A [matrix](!/linear-algebra/definitions#matrix) with equal numbers of rows and columns: $A \in \mathbb{R}^{n \times n}$

Identity Matrix

The [square matrix](!/linear-algebra/definitions#square_matrix) with $1$s on the main diagonal and $0$s elsewhere, denoted $I_n$: $$I_n = \begin{pmatrix} 1 & 0 & \cdots & 0 \\ 0 & 1 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 1 \end{pmatrix}$$

Symmetric Matrix

A [square matrix](!/linear-algebra/definitions#square_matrix) satisfying $A = A^T$

Inverse Matrix

A [square matrix](!/linear-algebra/definitions#square_matrix) $A^{-1}$ such that $AA^{-1} = A^{-1}A = I$

Singular Matrix

A [square matrix](!/linear-algebra/definitions#square_matrix) $A$ with $\det(A) = 0$

Rank

The number of [linearly independent](!/linear-algebra/definitions#linear_independence) columns (equivalently, rows) in a [matrix](!/linear-algebra/definitions#matrix): $$\text{rank}(A) = \dim(\text{Col}(A)) = \dim(\text{Row}(A))$$

Trace

The sum of the main diagonal entries of a [square matrix](!/linear-algebra/definitions#square_matrix): $$\text{tr}(A) = a_{11} + a_{22} + \cdots + a_{nn} = \sum_{i=1}^{n} a_{ii}$$

Diagonal Matrix

A [square matrix](!/linear-algebra/definitions#square_matrix) where $a_{ij} = 0$ for all $i \neq j$

Positive Definite Matrix

A [symmetric matrix](!/linear-algebra/definitions#symmetric_matrix) $A$ satisfying $\mathbf{x}^T A \mathbf{x} > 0$ for all nonzero $\mathbf{x}$

Determinant

A scalar $\det(A) \in \mathbb{R}$ assigned to every [square matrix](!/linear-algebra/definitions#square_matrix), defined recursively via [cofactor](!/linear-algebra/definitions#cofactor) expansion

Minor

The [determinant](!/linear-algebra/definitions#determinant) of the submatrix obtained by deleting row $i$ and column $j$ from a [matrix](!/linear-algebra/definitions#matrix): $$M_{ij} = \det(\hat{A}_{ij})$$

Cofactor

A signed [minor](!/linear-algebra/definitions#minor), with sign determined by the position $(i,j)$: $$C_{ij} = (-1)^{i+j} M_{ij}$$

Cofactor Matrix (Adjugate)

The transpose of the matrix of [cofactors](!/linear-algebra/definitions#cofactor) of $A$: $$\text{adj}(A) = C^T$$

System of Linear Equations

A collection of equations $A\mathbf{x} = \mathbf{b}$ where $A$ is an $m \times n$ [matrix](!/linear-algebra/definitions#matrix) and $\mathbf{b} \in \mathbb{R}^m$

Augmented Matrix

The [matrix](!/linear-algebra/definitions#matrix) formed by appending the right-hand side vector $\mathbf{b}$ as an additional column to the coefficient matrix $A$, written $[A \mid \mathbf{b}]$

Row Echelon Form

A matrix where: • all zero rows are at the bottom • each leading entry ([pivot](!/linear-algebra/definitions#pivot)) is to the right of the pivot in the row above • all entries below each pivot are zero

Reduced Row Echelon Form

[Row echelon form](!/linear-algebra/definitions#row_echelon_form) with the additional requirements: • every [pivot](!/linear-algebra/definitions#pivot) is $1$ • each pivot is the only nonzero entry in its column

Pivot

The first nonzero entry in each row of a matrix in [row echelon form](!/linear-algebra/definitions#row_echelon_form)

Homogeneous System

A [system of linear equations](!/linear-algebra/definitions#system_of_linear_equations) in which every equation equals zero: $A\mathbf{x} = \mathbf{0}$

Linear Transformation

A function $T: V \to W$ between [vector spaces](!/linear-algebra/definitions#vector_space) that preserves addition and scalar multiplication: $$T(\mathbf{u} + \mathbf{v}) = T(\mathbf{u}) + T(\mathbf{v})$$ $$T(c\mathbf{u}) = cT(\mathbf{u})$$

Image (Range)

The set of all output vectors of a [linear transformation](!/linear-algebra/definitions#linear_transformation): $$\text{Im}(T) = \{T(\mathbf{v}) \mid \mathbf{v} \in V\}$$

Matrix Representation

A [matrix](!/linear-algebra/definitions#matrix) $A$ such that $T(\mathbf{v}) = A[\mathbf{v}]_{\mathcal{B}}$ for a chosen [basis](!/linear-algebra/definitions#basis) $\mathcal{B}$

Change of Basis Matrix

A matrix $P$ that converts coordinates from one [basis](!/linear-algebra/definitions#basis) to another: $[\mathbf{v}]_{\mathcal{B}'} = P^{-1}[\mathbf{v}]_{\mathcal{B}}$

Similar Matrices

Matrices $A$ and $B$ are similar if $B = P^{-1}AP$ for some invertible matrix $P$

Eigenvalue

A scalar $\lambda$ such that $A\mathbf{v} = \lambda\mathbf{v}$ for some nonzero [vector](!/linear-algebra/definitions#vector) $\mathbf{v}$

Eigenvector

A nonzero vector $\mathbf{v}$ such that $A\mathbf{v} = \lambda\mathbf{v}$ for some scalar $\lambda$

Eigenspace

The set of all [eigenvectors](!/linear-algebra/definitions#eigenvector) for a given [eigenvalue](!/linear-algebra/definitions#eigenvalue) $\lambda$, together with the zero vector — equivalently, the [null space](!/linear-algebra/definitions#null_space) of $(A - \lambda I)$: $$E_\lambda = \text{Nul}(A - \lambda I)$$

Characteristic Polynomial

The polynomial whose roots are the [eigenvalues](!/linear-algebra/definitions#eigenvalue) of $A$, obtained by computing: $$p(\lambda) = \det(A - \lambda I)$$

Algebraic Multiplicity

The multiplicity of $\lambda$ as a root of the [characteristic polynomial](!/linear-algebra/definitions#characteristic_polynomial)

Geometric Multiplicity

The [dimension](!/linear-algebra/definitions#dimension) of the [eigenspace](!/linear-algebra/definitions#eigenspace) associated with an [eigenvalue](!/linear-algebra/definitions#eigenvalue) $\lambda$: $$\text{geo. mult.}(\lambda) = \dim(E_\lambda) = \dim(\text{Nul}(A - \lambda I))$$

Singular Value

A nonnegative scalar measuring how much a matrix stretches space along each principal direction, derived from the [eigenvalues](!/linear-algebra/definitions#eigenvalue) of $A^TA$: $$\sigma_i = \sqrt{\lambda_i(A^TA)}$$

Inner Product

A function $\langle \cdot, \cdot \rangle: V \times V \to \mathbb{R}$ satisfying symmetry, linearity, and positive-definiteness

Orthogonal Vectors

Vectors $\mathbf{u}$ and $\mathbf{v}$ are orthogonal if $\langle \mathbf{u}, \mathbf{v} \rangle = 0$

Orthogonal Set

A set of vectors $\{\mathbf{v}_1, \ldots, \mathbf{v}_k\}$ where $\langle \mathbf{v}_i, \mathbf{v}_j \rangle = 0$ for all $i \neq j$

Orthonormal Set

An [orthogonal set](!/linear-algebra/definitions#orthogonal_set) where every vector is a [unit vector](!/linear-algebra/definitions#unit_vector): $\langle \mathbf{v}_i, \mathbf{v}_j \rangle = \delta_{ij}$

Orthogonal Complement

The set of all vectors in $V$ that are [orthogonal](!/linear-algebra/definitions#orthogonal_vectors) to every vector in a [subspace](!/linear-algebra/definitions#subspace) $W$: $$W^\perp = \{\mathbf{v} \in V \mid \langle \mathbf{v}, \mathbf{w} \rangle = 0 \text{ for all } \mathbf{w} \in W\}$$

Orthogonal Matrix

A [square matrix](!/linear-algebra/definitions#square_matrix) $Q$ satisfying $Q^TQ = QQ^T = I$

Vector

An ordered list of $n$ real numbers: $\mathbf{v} = (v_1, v_2, \ldots, v_n) \in \mathbb{R}^n$

Scalar

An element of the underlying field — in standard linear algebra, a real number $c \in \mathbb{R}$

Magnitude (Norm)

The length of a [vector](!/linear-algebra/definitions#vector), measured as its distance from the origin: $$\|\mathbf{v}\| = \sqrt{v_1^2 + v_2^2 + \cdots + v_n^2}$$

Unit Vector

A vector $\hat{\mathbf{u}}$ with $\|\hat{\mathbf{u}}\| = 1$

Dot Product

Cross Product

Linear Combination

Vector Space

A set $V$ equipped with vector addition and scalar multiplication satisfying the [vector space axioms](!/linear-algebra/vector-spaces/axioms)

Subspace

A nonempty subset $W \subseteq V$ that is itself a [vector space](!/linear-algebra/definitions#vector_space) under the same operations

Span

Linear Independence

Vectors $\mathbf{v}_1, \ldots, \mathbf{v}_k$ are linearly independent if the only solution to $$c_1\mathbf{v}_1 + \cdots + c_k\mathbf{v}_k = \mathbf{0}$$ is $c_1 = c_2 = \cdots = c_k = 0$

Basis

Dimension

The number of vectors in any [basis](!/linear-algebra/definitions#basis) of a [vector space](!/linear-algebra/definitions#vector_space) $V$, denoted $\dim(V)$

Column Space

Null Space (Kernel)

Row Space

Left Null Space

Matrix

A rectangular array of numbers with $m$ rows and $n$ columns: $A \in \mathbb{R}^{m \times n}$

Square Matrix

A [matrix](!/linear-algebra/definitions#matrix) with equal numbers of rows and columns: $A \in \mathbb{R}^{n \times n}$

Identity Matrix

Symmetric Matrix

A [square matrix](!/linear-algebra/definitions#square_matrix) satisfying $A = A^T$

Inverse Matrix

A [square matrix](!/linear-algebra/definitions#square_matrix) $A^{-1}$ such that $AA^{-1} = A^{-1}A = I$

Singular Matrix

A [square matrix](!/linear-algebra/definitions#square_matrix) $A$ with $\det(A) = 0$

Rank

Trace

The sum of the main diagonal entries of a [square matrix](!/linear-algebra/definitions#square_matrix): $$\text{tr}(A) = a_{11} + a_{22} + \cdots + a_{nn} = \sum_{i=1}^{n} a_{ii}$$

Diagonal Matrix

A [square matrix](!/linear-algebra/definitions#square_matrix) where $a_{ij} = 0$ for all $i \neq j$

Positive Definite Matrix

A [symmetric matrix](!/linear-algebra/definitions#symmetric_matrix) $A$ satisfying $\mathbf{x}^T A \mathbf{x} > 0$ for all nonzero $\mathbf{x}$

Determinant

A scalar $\det(A) \in \mathbb{R}$ assigned to every [square matrix](!/linear-algebra/definitions#square_matrix), defined recursively via [cofactor](!/linear-algebra/definitions#cofactor) expansion

Minor

Cofactor

A signed [minor](!/linear-algebra/definitions#minor), with sign determined by the position $(i,j)$: $$C_{ij} = (-1)^{i+j} M_{ij}$$

Cofactor Matrix (Adjugate)

The transpose of the matrix of [cofactors](!/linear-algebra/definitions#cofactor) of $A$: $$\text{adj}(A) = C^T$$

System of Linear Equations

A collection of equations $A\mathbf{x} = \mathbf{b}$ where $A$ is an $m \times n$ [matrix](!/linear-algebra/definitions#matrix) and $\mathbf{b} \in \mathbb{R}^m$

Augmented Matrix

The [matrix](!/linear-algebra/definitions#matrix) formed by appending the right-hand side vector $\mathbf{b}$ as an additional column to the coefficient matrix $A$, written $[A \mid \mathbf{b}]$

Row Echelon Form

Reduced Row Echelon Form

Pivot

The first nonzero entry in each row of a matrix in [row echelon form](!/linear-algebra/definitions#row_echelon_form)

Homogeneous System

A [system of linear equations](!/linear-algebra/definitions#system_of_linear_equations) in which every equation equals zero: $A\mathbf{x} = \mathbf{0}$

Linear Transformation

Image (Range)

The set of all output vectors of a [linear transformation](!/linear-algebra/definitions#linear_transformation): $$\text{Im}(T) = \{T(\mathbf{v}) \mid \mathbf{v} \in V\}$$

Matrix Representation

A [matrix](!/linear-algebra/definitions#matrix) $A$ such that $T(\mathbf{v}) = A[\mathbf{v}]_{\mathcal{B}}$ for a chosen [basis](!/linear-algebra/definitions#basis) $\mathcal{B}$

Change of Basis Matrix

A matrix $P$ that converts coordinates from one [basis](!/linear-algebra/definitions#basis) to another: $[\mathbf{v}]_{\mathcal{B}'} = P^{-1}[\mathbf{v}]_{\mathcal{B}}$

Similar Matrices

Matrices $A$ and $B$ are similar if $B = P^{-1}AP$ for some invertible matrix $P$

Eigenvalue

A scalar $\lambda$ such that $A\mathbf{v} = \lambda\mathbf{v}$ for some nonzero [vector](!/linear-algebra/definitions#vector) $\mathbf{v}$

Eigenvector

A nonzero vector $\mathbf{v}$ such that $A\mathbf{v} = \lambda\mathbf{v}$ for some scalar $\lambda$

Eigenspace

Characteristic Polynomial

The polynomial whose roots are the [eigenvalues](!/linear-algebra/definitions#eigenvalue) of $A$, obtained by computing: $$p(\lambda) = \det(A - \lambda I)$$

Algebraic Multiplicity

The multiplicity of $\lambda$ as a root of the [characteristic polynomial](!/linear-algebra/definitions#characteristic_polynomial)

Geometric Multiplicity

Singular Value

Inner Product

A function $\langle \cdot, \cdot \rangle: V \times V \to \mathbb{R}$ satisfying symmetry, linearity, and positive-definiteness

Orthogonal Vectors

Vectors $\mathbf{u}$ and $\mathbf{v}$ are orthogonal if $\langle \mathbf{u}, \mathbf{v} \rangle = 0$

Orthogonal Set

A set of vectors $\{\mathbf{v}_1, \ldots, \mathbf{v}_k\}$ where $\langle \mathbf{v}_i, \mathbf{v}_j \rangle = 0$ for all $i \neq j$

Orthonormal Set

Orthogonal Complement

Orthogonal Matrix

A [square matrix](!/linear-algebra/definitions#square_matrix) $Q$ satisfying $Q^TQ = QQ^T = I$

See All Definitions

Learn More

Matrices

Explore matrices in linear algebra through our detailed guide.Starting with matrix definitions and notations, the page explains matrix structure, elements, and indexing. You will learn to distinguish between different matrix types - from basic row and column matrices to more complex square matrices. The guide also covers essential matrix properties and dives into special cases of square matrices like diagonal and triangular forms. Each topic features clear mathematical notation and visual examples to reinforce your understanding of these fundamental concepts.

Definitions and Notations - Explains how matrices are written using different types of brackets (square, parentheses, vertical bars) and introduces basic matrix notation conventions.
Elements, Structure and Indexing - Covers how matrix elements are organized in rows and columns, explains the 1-based indexing system, and demonstrates how to reference specific elements using row and column indices.
Types of Matrices - Describes different classifications of matrices based on their shape (column, row, rectangular, and square matrices) and content type (numeric, variable/symbolic, mixed, and zero matrices).
Matrix Properties - Introduces essential characteristics like size/dimension, rank, determinant, eigenvalues/eigenvectors, and trace, explaining their importance in matrix operations and transformations.
Square Matrices and Special Cases - Focuses on unique types of square matrices, including those with special diagonal patterns (diagonal, upper triangular, lower triangular) and element relationships (symmetric, skew-symmetric, identity, scalar).

Learn More

Linear Algebra Symbols Reference

Our Linear Algebra Symbols page presents a well-organized collection of notation fundamental to matrix theory and vector spaces. This reference serves as a valuable resource for students and practitioners working with linear systems.
The page features symbols categorized by their mathematical applications, including matrix operations (A⊤, det(A), tr(A)), vector spaces (ℝⁿ, ⟨v,w⟩, ∥v∥), and eigenvalue concepts (Av=λv). It covers advanced topics like matrix decompositions (LU, QR, SVD) and linear transformations, alongside practical notation for representing matrices and vectors in LaTeX.
Every symbol includes its proper mathematical notation, corresponding LaTeX code for typesetting, and a brief explanation of its mathematical significance—making this an indispensable reference for anyone working with linear algebra in academic research or applications.RetryClaude can make mistakes. Please double-check responses.

Learn More

Tools

Determinant Visual Calculator with Steps

Use visualinteractive Determinant Calculator with step-by-step explanations

Try it