Visual Tools
Calculators
Tables
Mathematical Keyboard
Converters
Other Tools


Orthogonality



What Orthogonality Means
Orthogonality in R² and R³
Orthogonal Complements
The Four Fundamental Subspaces Revisited
Why Orthogonality Matters
Inner Products
Orthogonal and Orthonormal Sets
Projections
Gram-Schmidt and Least Squares
Summary: Orthogonality at a Glance



Perpendicularity and Its Consequences

Orthogonality — the condition that two vectors are perpendicular — is the geometric idea that makes linear algebra computationally clean. Orthogonal bases turn coordinate-finding into dot products. Projections onto subspaces become explicit formulas. Least-squares approximation reduces to a single matrix equation. Every simplification traces back to the same root: when vectors are perpendicular, their interactions vanish and problems decouple.



What Orthogonality Means

Two vectors u\mathbf{u} and v\mathbf{v} in Rn\mathbb{R}^n are orthogonal if their dot product is zero:

u⋅v=u1v1+u2v2+⋯+unvn=0\mathbf{u} \cdot \mathbf{v} = u_1 v_1 + u_2 v_2 + \cdots + u_n v_n = 0


Geometrically, this means the angle between the two vectors is 90°90°. The vectors are perpendicular — pointing in completely independent directions with no component of one lying along the other.

The zero vector is orthogonal to every vector, since 0⋅v=0\mathbf{0} \cdot \mathbf{v} = 0 for all v\mathbf{v}. This is a convention that keeps the theory clean, not a geometric statement — the zero vector has no direction. Orthogonality is defined relative to an inner product, and on this site the standard dot product is used unless stated otherwise.

Orthogonality in R² and R³

In R2\mathbb{R}^2, the vectors (a,b)(a, b) and (c,d)(c, d) are orthogonal if and only if ac+bd=0ac + bd = 0. The pair (1,2)(1, 2) and (−2,1)(-2, 1) satisfies 1(−2)+2(1)=01(-2) + 2(1) = 0, so these vectors are perpendicular. Rotating any vector by 90°90° produces an orthogonal partner: (a,b)(a, b) is orthogonal to (−b,a)(-b, a).

In R3\mathbb{R}^3, the standard basis vectors e1=(1,0,0)\mathbf{e}_1 = (1, 0, 0), e2=(0,1,0)\mathbf{e}_2 = (0, 1, 0), e3=(0,0,1)\mathbf{e}_3 = (0, 0, 1) are mutually orthogonal — every pair has dot product zero. The cross product a×b\mathbf{a} \times \mathbf{b} produces a vector orthogonal to both a\mathbf{a} and b\mathbf{b}, constructing perpendicularity from any two non-parallel inputs.

Orthogonality is the foundation of coordinate systems. Axes that are perpendicular allow each coordinate to be read independently — changing one coordinate does not affect any other. This independence is what makes orthogonal bases so powerful.

Orthogonal Complements

For a subspace WW of Rn\mathbb{R}^n, the orthogonal complement W⊥W^\perp is the set of all vectors perpendicular to everything in WW:

W⊥={v∈Rn:v⋅w=0 for all w∈W}W^\perp = \{\mathbf{v} \in \mathbb{R}^n : \mathbf{v} \cdot \mathbf{w} = 0 \text{ for all } \mathbf{w} \in W\}


The orthogonal complement is itself a subspace. Its dimension satisfies dim⁡(W)+dim⁡(W⊥)=n\dim(W) + \dim(W^\perp) = n, and taking the complement twice returns to the original: (W⊥)⊥=W(W^\perp)^\perp = W.

The most important structural consequence is the orthogonal decomposition. Every vector v∈Rn\mathbf{v} \in \mathbb{R}^n can be written uniquely as

v=w+w⊥\mathbf{v} = \mathbf{w} + \mathbf{w}^\perp


where w∈W\mathbf{w} \in W and w⊥∈W⊥\mathbf{w}^\perp \in W^\perp. The two components are perpendicular to each other: w⋅w⊥=0\mathbf{w} \cdot \mathbf{w}^\perp = 0. This decomposition is the geometric heart of projection: w\mathbf{w} is the projection of v\mathbf{v} onto WW, and w⊥\mathbf{w}^\perp is the residual.

The Four Fundamental Subspaces Revisited

The orthogonal complement structure appears naturally in the four fundamental subspaces of any m×nm \times n matrix AA.

In Rn\mathbb{R}^n, the row space and the null space are orthogonal complements:

Row(A)⊥=Null(A)\text{Row}(A)^\perp = \text{Null}(A)


Every vector in the null space is perpendicular to every row of AA, because Ax=0A\mathbf{x} = \mathbf{0} means the dot product of x\mathbf{x} with each row is zero.

In Rm\mathbb{R}^m, the column space and the left null space are orthogonal complements:

Col(A)⊥=Null(AT)\text{Col}(A)^\perp = \text{Null}(A^T)


These two pairs of complements are the structural backbone of projection and least squares. Projecting a vector b\mathbf{b} onto the column space means decomposing b\mathbf{b} into a column-space component (the best approximation Ax^A\hat{\mathbf{x}}) and a left-null-space component (the residual b−Ax^\mathbf{b} - A\hat{\mathbf{x}}).
Ambient space Subspace Its orthogonal complement Dimension sum
ℝⁿ (the domain) Row(A) — row space of A Null(A) — null space of A rank(A) + nullity(A) = n
ℝᵐ (the codomain) Col(A) — column space of A Null(Aᵀ) — left null space of A rank(A) + (m − rank(A)) = m

Why Orthogonality Matters

Orthogonality is the single property that converts hard linear algebra problems into easy ones.

Orthogonal bases make coordinate computation trivial: the coefficient of each basis vector is a single dot product, not the solution of a system. For a general basis, finding coordinates requires solving nn equations; for an orthonormal basis, it requires nn dot products.

Projections onto subspaces have explicit formulas when the basis is orthogonal. The projection of b\mathbf{b} onto a subspace splits into independent projections onto each basis vector, with no cross-talk between components.

Least-squares approximation — the best approximate solution when Ax=bA\mathbf{x} = \mathbf{b} has no exact solution — reduces to projecting b\mathbf{b} onto the column space. The normal equations ATAx^=ATbA^TA\hat{\mathbf{x}} = A^T\mathbf{b} are a direct consequence of the orthogonality condition on the residual.

Orthogonal matrices preserve lengths and angles, making them numerically stable in computation. The Gram-Schmidt process converts any basis into an orthogonal one, ensuring these benefits are always available.

Inner Products

The dot product is the standard way to measure angles and lengths in Rn\mathbb{R}^n, but it is not the only one. An inner product is any function ⟨⋅,⋅⟩\langle \cdot, \cdot \rangle that satisfies symmetry, linearity, and positive definiteness. Different inner products define different notions of perpendicularity and distance.

A weighted inner product ⟨u,v⟩=uTWv\langle \mathbf{u}, \mathbf{v} \rangle = \mathbf{u}^T W \mathbf{v} (with WW symmetric positive definite) distorts the geometry — circles become ellipses, and "perpendicular" means something different than in the standard dot product. On function spaces, the integral ⟨f,g⟩=∫abf(x)g(x) dx\langle f, g \rangle = \int_a^b f(x)g(x)\,dx defines orthogonality for functions, leading to Fourier series and orthogonal polynomials.

Every inner product induces a norm (∥v∥=⟨v,v⟩\|\mathbf{v}\| = \sqrt{\langle \mathbf{v}, \mathbf{v} \rangle}), a distance (d(u,v)=∥u−v∥d(\mathbf{u}, \mathbf{v}) = \|\mathbf{u} - \mathbf{v}\|), and the Cauchy-Schwarz inequality (∣⟨u,v⟩∣≤∥u∥∥v∥|\langle \mathbf{u}, \mathbf{v} \rangle| \leq \|\mathbf{u}\|\|\mathbf{v}\|). The entire orthogonality framework — projections, Gram-Schmidt, least squares — works in any inner product space.
Inner product Definition Setting What it enables
Standard dot product u · v = Σ ui vi ℝⁿ with Euclidean geometry standard lengths, angles, and perpendicularity
Weighted inner product ⟨u, v⟩ = uᵀ W v (W symmetric positive definite) ℝⁿ with distorted metric ellipsoidal geometry; statistics (e.g. Mahalanobis distance)
Function-space integral ⟨f, g⟩ = ∫ab f(x) g(x) dx spaces of square-integrable functions Fourier series, orthogonal polynomials, signal decomposition

Orthogonal and Orthonormal Sets

An orthogonal set is a collection of vectors that are pairwise perpendicular: vi⋅vj=0\mathbf{v}_i \cdot \mathbf{v}_j = 0 whenever i≠ji \neq j. An orthonormal set adds the requirement that each vector has unit length: ∥vi∥=1\|\mathbf{v}_i\| = 1.

Orthogonal sets of nonzero vectors are automatically linearly independent — no independence check is needed. The proof is one line: if ∑civi=0\sum c_i \mathbf{v}_i = \mathbf{0}, dotting both sides with vj\mathbf{v}_j gives cj∥vj∥2=0c_j \|\mathbf{v}_j\|^2 = 0, forcing cj=0c_j = 0.

The computational advantage of an orthonormal basis {q1,…,qn}\{\mathbf{q}_1, \dots, \mathbf{q}_n\} is that coordinates are free: ci=qi⋅vc_i = \mathbf{q}_i \cdot \mathbf{v}. No system of equations, no row reduction, no matrix inversion — just nn dot products.

Projections

The orthogonal projection of a vector b\mathbf{b} onto a subspace WW is the closest point in WW to b\mathbf{b}. It is the component of b\mathbf{b} that lies in WW, with the perpendicular remainder discarded.

For projection onto a single vector a\mathbf{a}: projab=a⋅ba⋅aa\text{proj}_{\mathbf{a}}\mathbf{b} = \frac{\mathbf{a} \cdot \mathbf{b}}{\mathbf{a} \cdot \mathbf{a}}\mathbf{a}. For projection onto a subspace with basis matrix AA: b^=A(ATA)−1ATb\hat{\mathbf{b}} = A(A^TA)^{-1}A^T\mathbf{b}.

The projection matrix P=A(ATA)−1ATP = A(A^TA)^{-1}A^T is symmetric and idempotent: PT=PP^T = P and P2=PP^2 = P. The residual b−Pb\mathbf{b} - P\mathbf{b} is orthogonal to WW — this is the defining geometric property. And I−PI - P projects onto the orthogonal complement W⊥W^\perp.

Gram-Schmidt and Least Squares

The Gram-Schmidt process converts any linearly independent set into an orthogonal (or orthonormal) set spanning the same subspace. It does this by sequentially subtracting projections: each new vector has its components along all previously computed directions removed, leaving only the perpendicular remainder.

Gram-Schmidt applied to the columns of a matrix AA produces the QR decomposition A=QRA = QR, where QQ has orthonormal columns and RR is upper triangular. This decomposition is numerically superior to forming ATAA^TA directly and is the standard method for least-squares computation.

Least squares addresses the case where Ax=bA\mathbf{x} = \mathbf{b} has no exact solution. The best approximation x^\hat{\mathbf{x}} minimizes ∥Ax−b∥2\|A\mathbf{x} - \mathbf{b}\|^2 and satisfies the normal equations ATAx^=ATbA^TA\hat{\mathbf{x}} = A^T\mathbf{b}. Geometrically, Ax^A\hat{\mathbf{x}} is the projection of b\mathbf{b} onto the column space of AA — the closest reachable point to the unreachable target.

Summary: Orthogonality at a Glance

The orthogonality subtree branches into several specialized topics — inner products, orthogonal and orthonormal sets, projections, Gram–Schmidt, and least squares — each developed in its own page. The table below collects each topic alongside its key statement and the main formula or result, providing a roadmap from this hub into the deeper material and a single reference for the central facts.
Topic Key statement Main formula or result
Orthogonal vectors u · v = 0 means u and v are perpendicular the dot-product-zero test
Orthogonal complement W⊥ vectors perpendicular to every vector in W form a subspace dim W + dim W⊥ = n; every v = w + w⊥ uniquely
Inner product generalizes the dot product to arbitrary vector spaces symmetry, linearity, positive definiteness; induces norm, distance, Cauchy–Schwarz
Orthogonal & orthonormal sets pairwise-perpendicular vectors are automatically independent in an orthonormal basis, coordinates are free: ci = qi · v
Projection onto a subspace the closest point in W to a given b projection matrix P = A(AᵀA)⁻¹Aᵀ; idempotent and symmetric
Gram–Schmidt process converts any basis into an orthonormal one sequential projection subtraction; produces A = QR (QR decomposition)
Least squares best approximate solution when A x = b has none normal equations AᵀA x̂ = Aᵀb; A x̂ is the projection of b onto Col(A)