Orthogonal & Orthonormal Sets and Bases

Orthogonal Sets

Six sets of vectors. In which of them does every vector meet every other at a right angle?

Some are lined up with the axes, some are tilted, one contains a suspicious vector. From the numbers alone it is hard to tell.

set	vectors
A	(1, 0, 0), (0, 2, 0), (0, 0, −3)
B	(1, 1, 1), (1, −2, 1), (1, 0, −1)
C	(1, 1, 0), (1, −1, 0), (1, 0, 1)
D	(1, 2), (2, −1)
E	(1, 1), (2, 2)
F	(1, 0), (0, 0)

Taking the dot product of every pair settles it: A, B and D pass; C, E and F fail.

set	dot products of the pairs	orthogonal set?
A	0, 0, 0	yes
B	0, 0, 0	yes
C	0, 1, 1 — two pairs fail	no
D	2 − 2 = 0	yes
E	2 + 2 = 4 — the vectors are parallel	no
F	0, but (0, 0) is the zero vector	no

C fails on two pairs, E is parallel, and F only passes the dot-product test because of the zero vector.

In C the first two vectors are perpendicular, but the third is not perpendicular to them, and one bad pair is enough to fail. B looks just as tilted as C, yet all three of its pairs give

0

.

The vectors in an orthogonal set must all be nonzero — the zero vector is excluded because including it would trivialize the structure (every vector is orthogonal to

\mathbf{0}

, so

\mathbf{0}

carries no directional information).

The rule: every pair perpendicular, and no zero vector allowed.

Think of the edges meeting at the corner of a box: each one is at a right angle to every other. An orthogonal set is a collection of directions arranged like that, possibly with different lengths.

The definition

An orthogonal set is a set of nonzero vectors in which every pair of distinct vectors has dot product zero.

A set of vectors

\{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\}

is orthogonal if every pair has dot product zero:

Orthogonal Sets

\mathbf{v}_i \cdot \mathbf{v}_j = 0 \quad \text{for all } i \neq j

So sets A, B and D are orthogonal; C, E and F are not.

Being orthogonal is a property of the whole set — every pair has to pass.

For example,

\{(1, 0, 0), (0, 2, 0), (0, 0, -3)\}

is orthogonal in

\mathbb{R}^3

: every pair of distinct vectors has dot product zero. The vectors need not have the same length, and their lengths can be anything nonzero.

Example

A less obvious example:

\{(1, 1, 1), (1, -2, 1), (1, 0, -1)\}

. Checking:

(1)(1) + (1)(-2) + (1)(1) = 0

,

(1)(1) + (1)(0) + (1)(-1) = 0

,

(1)(1) + (-2)(0) + (1)(-1) = 0

. All three pairwise products vanish — the set is orthogonal despite none of the vectors being aligned with the coordinate axes.

Fact: All-sign bases need a multiple of 4

n

mutually orthogonal vectors in

\mathbb{R}^n

with every entry

\pm 1

exist for

n = 1

and

n = 2

, and otherwise only when

n

is a multiple of

4

: any three of them sort the

n

positions into four equal groups by which signs agree. Whether every multiple of

4

works is still unproved.

A set with every pair at right angles

Every pair in this set has a dot product of zero. That single condition forces independence — no vector can be a combination of the others without picking up a non-zero dot product with one of them. Independence therefore comes free with orthogonality rather than needing a separate check. Build such a set from arbitrary vectors on the Gram-Schmidt visualizer.

The zero vector has to be excluded by hand, since it is orthogonal to everything and would wreck the independence it otherwise guarantees.

A rejected set can often be repaired: tilting its vectors into right angles keeps the same span.

Set C spans the same space as an orthogonal set, which the Gram-Schmidt process produces step by step. What orthogonality buys in return is shown in the next section: orthogonal sets are independent.

The rest of the page covers notation, why orthogonal sets are independent, orthonormal sets and bases, coordinates via dot products, orthogonal matrices, and matrices with orthonormal columns.

Orthogonal Set Notation

Notation

Orthogonal Set Notation

Two conditions — mutual perpendicularity and unit length — that most of this subject states in prose, compressed by one symbol into a single equation. Around it sit a hat that means a specific normalisation and a matrix letter whose name does not match its defining property. All catalogued among the linear algebra symbols.

The dot and

\perp

come from dot product and inner product notation;

\|\mathbf{v}\|

and the hat from norm notation;

A^{T}

,

I

and

A^{-1}

from matrix operation and inverse notation; subscripted index pairs from matrix entry notation.

\mathbf{v}_i \cdot \mathbf{v}_j = \delta_{ij}

the dot product is one when the indices agree and zero otherwise

The Kronecker delta is a switch written as a symbol:

\delta_{ij} = 1

when

i = j

and

0

otherwise. It compresses the whole definition of an orthonormal set into one equation — perpendicularity is the

i \neq j

case, unit length the

i = j

case, as Orthonormal Sets below states.

CasesThe same symbol is the identity matrix entrywise,

I = [\delta_{ij}]

, which is why

Q^{T}Q = I

and "the columns are orthonormal" are the same statement written two ways — one in matrix marks, one in index marks.

Do not confuseThe

\Delta

and

\delta

of calculus — an increment or a small quantity. Here

\delta

takes two subscripts and returns one of two values; it is a function of a pair of indices, not a magnitude, and nothing about it is small.

\hat{\mathbf{u}} = \dfrac{\mathbf{v}}{\|\mathbf{v}\|}

u-hat — the unit vector in the direction of v

The hat is a claim about length, not about identity: it says this vector has been divided by its own norm and now measures

1

. Normalising is the single step that turns an orthogonal set into an orthonormal one, so the hat marks exactly what the extra word "normal" contributes.

CasesThe standard basis carries hats for the same reason —

\hat{\imath}, \hat{\jmath}, \hat{k}

in physics for what this site writes

\mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3

— and the hat is dropped once every vector in play is known to be a unit vector.

Same glyph elsewhereIn statistics the same hat marks an estimate rather than a length —

\hat{\mu}

is an estimated mean, with no implication that anything equals one. The two-hats convention keeps the jobs apart.

Q^{T}Q = I

, so

Q^{-1} = Q^{T}

Q is orthogonal: its transpose is its inverse

Q

is the reserved letter for a matrix with orthonormal columns, and the defining equation is the payoff — Orthogonal Matrices below — since it replaces inversion, normally expensive, with transposition, which is free.

CasesThe equation is a statement about columns, and for a square

Q

it forces the same of rows, so

QQ^{T} = I

holds too — one of the few places where a matrix identity is symmetric in that way.

Do not confuseThe name itself. An "orthogonal matrix" has orthonormal columns, not merely orthogonal ones — unit length is part of the definition, and a matrix whose columns are perpendicular but not normalised satisfies no such identity. The terminology is a historical accident the notation does not repeat.

All linear algebra symbols →↑ Orthogonality

DownloadPNG2×PDF

Notation · Cheat sheet

Orthogonality Notation at a Glance

The marks from the notation cards across the orthogonality pages: how each is read and what it means. Every group links to the full card on its page.

Inner product

\langle \mathbf{u}, \mathbf{v} \rangle

the inner product of u and v

Two vectors in, one number out.

\langle \cdot, \cdot \rangle

the inner product as a function

The dots are empty slots.

\mathbf{u} \perp \mathbf{v}, \; W^{\perp}

u orthogonal to v; W perp

\langle \mathbf{u}, \mathbf{v} \rangle = 0

;

W^{\perp}

holds all vectors orthogonal to

W

.

Orthogonal setsThis page

\delta_{ij}

the Kronecker delta

1

when

i = j

,

0

otherwise.

\hat{\mathbf{u}} = \mathbf{v} / \|\mathbf{v}\|

u hat

The unit vector in the direction of

\mathbf{v}

.

Q^{T}Q = I

Q is orthogonal

So

Q^{-1} = Q^{T}

.

Projections

\text{proj}_{W}\mathbf{b}

projection of b onto W

The subscript is the target.

\hat{\mathbf{b}}

b hat

The closest point to

\mathbf{b}

in

W

.

P^2 = P

the projection matrix

Projecting twice changes nothing.

Gram–Schmidt

\mathbf{v}_j \to \mathbf{u}_j \to \mathbf{e}_j

input, orthogonalised, normalised

Three letters for three stages of one vector.

\sum_{i=1}^{j-1} \text{proj}_{\mathbf{u}_i}

all projections so far

Subtracted from

\mathbf{v}_j

; the sum grows with

j

.

A = QR

Q R factorisation

Orthonormal

Q

, upper-triangular

R

.

Least squares

\hat{\mathbf{x}}

x hat

The best fit: minimises

\|A\mathbf{x} - \mathbf{b}\|

.

A^{T}A\hat{\mathbf{x}} = A^{T}\mathbf{b}

the normal equations

Normal means perpendicular: the residual is orthogonal to the columns.

A^{+}

A plus, the pseudoinverse

Stands in for an inverse that does not exist:

\hat{\mathbf{x}} = A^{+}\mathbf{b}

.

Same marks, different job

the hat

\hat{\mathbf{u}}

is a unit vector;

\hat{\mathbf{b}}

and

\hat{\mathbf{x}}

are best approximations.

P

Projection here; elsewhere, change of basis or a permutation.

\perp

Between vectors, orthogonal; as a superscript, the orthogonal complement.

Inner product symbols table Orthogonality

Orthogonality Notation at a Glance·/linear-algebra/orthogonality/inner-productLearn Math Class

Orthogonal Sets Are Independent

Every orthogonal set of nonzero vectors is linearly independent:

Orthogonal Set Independence

\{\mathbf{v}_1, \dots, \mathbf{v}_k\} \text{ orthogonal, all nonzero} \implies \text{linearly independent}

Orthogonal set independence →

The proof is short and reveals exactly why orthogonality is so powerful.

Suppose

c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \cdots + c_k\mathbf{v}_k = \mathbf{0}

. Dot both sides with

\mathbf{v}_j

:

Orthogonal Sets Are Independent

c_1(\mathbf{v}_1 \cdot \mathbf{v}_j) + c_2(\mathbf{v}_2 \cdot \mathbf{v}_j) + \cdots + c_k(\mathbf{v}_k \cdot \mathbf{v}_j) = 0

Every term with

i \neq j

vanishes because

\mathbf{v}_i \cdot \mathbf{v}_j = 0

. Only the

j

-th term survives:

c_j \|\mathbf{v}_j\|^2 = 0

. Since

\mathbf{v}_j \neq \mathbf{0}

,

\|\mathbf{v}_j\|^2 > 0

, so

c_j = 0

. This works for every

j

, so all coefficients are zero.

The key mechanism is that orthogonality isolates each coefficient. Dotting with

\mathbf{v}_j

kills every other term, leaving

c_j

alone. This is why orthogonal bases make coordinates computable by individual dot products — the same isolation principle that proves independence also extracts coordinates.

Orthonormal Sets

An orthonormal set is an orthogonal set where every vector additionally has unit length:

\|\mathbf{v}_i\| = 1

for all

i

. The two conditions together can be written compactly using the Kronecker delta:

Orthonormal Set

\mathbf{v}_i \cdot \mathbf{v}_j = \delta_{ij} = \begin{cases} 1 & \text{if } i = j \\ 0 & \text{if } i \neq j \end{cases}

Orthonormal set →

Any orthogonal set can be made orthonormal by normalizing each vector:

\hat{\mathbf{v}}_i = \mathbf{v}_i / \|\mathbf{v}_i\|

. The directions are preserved, only the lengths change to

1

.

Pitfall: Dividing by the squared length

Dividing by

\mathbf{v} \cdot \mathbf{v} = 6

instead of

\|\mathbf{v}\| = \sqrt{6}

turns

(1, -2, 1)

into

\tfrac{1}{6}(1, -2, 1)

, whose length is

\tfrac{1}{\sqrt{6}} \approx 0.41

. The unit vector is

\tfrac{1}{\sqrt{6}}(1, -2, 1)

.

Aspect	Orthogonal set	Orthonormal set
Pairwise dot products	v_i · v_j = 0 for i ≠ j	v_i · v_j = 0 for i ≠ j
Vector lengths	any nonzero value	each ‖v_i‖ = 1
Compact condition	v_i · v_j = 0 (i ≠ j) plus each v_i ≠ 0	v_i · v_j = δ_ij (the Kronecker delta)
Coordinate of x along v_i	c_i = (v_i · x) / (v_i · v_i)	c_i = q_i · x
Linear independence	automatic (nonzero vectors)	automatic (unit vectors are nonzero)
Going between the two	divide each v_i by ‖v_i‖ → orthonormal	already orthogonal as a special case

Orthogonal and Orthonormal Bases

An orthogonal basis is an orthogonal set that spans the space. An orthonormal basis is an orthonormal set that spans the space.

COROLLARY

In

\mathbb{R}^n

, an orthogonal set of

n

nonzero vectors is automatically a basis — independence is guaranteed by orthogonality, and

n

independent vectors in an

n

-dimensional space automatically span.

So the only check needed is: do I have

n

pairwise-orthogonal nonzero vectors? If yes, they form a basis.

Orthonormal bases exist for every finite-dimensional inner product space. The Gram-Schmidt process constructs one from any given basis. This means the computational advantages of orthonormal bases are always available — any space that has a basis at all has an orthonormal one.

Coordinates via Dot Products

The defining computational advantage of orthogonal bases is that coordinates are extracted by individual dot products.

For an orthogonal basis

\{\mathbf{v}_1, \dots, \mathbf{v}_n\}

, the coordinate of

\mathbf{x}

along

\mathbf{v}_i

is

Coordinates via Dot Products

c_i = \frac{\mathbf{x} \cdot \mathbf{v}_i}{\mathbf{v}_i \cdot \mathbf{v}_i}

For an orthonormal basis

\{\mathbf{q}_1, \dots, \mathbf{q}_n\}

, the denominator is

1

, and the formula simplifies to

Coordinates via Orthonormal Basis

c_i = \mathbf{x} \cdot \mathbf{q}_i

Coordinates via orthonormal basis →

No linear system needs to be solved. No matrix needs to be inverted. Each coordinate is computed independently by a single dot product.

Worked Example

Let

\{\mathbf{q}_1, \mathbf{q}_2, \mathbf{q}_3\}

be an orthonormal basis for

\mathbb{R}^3

with

\mathbf{q}_1 = \frac{1}{\sqrt{2}}(1, 1, 0)

,

\mathbf{q}_2 = \frac{1}{\sqrt{6}}(1, -1, 2)

,

\mathbf{q}_3 = \frac{1}{\sqrt{3}}(-1, 1, 1)

.

For

\mathbf{x} = (3, 1, 2)

:

c_1 = \mathbf{x} \cdot \mathbf{q}_1 = \frac{1}{\sqrt{2}}(3 + 1 + 0) = \frac{4}{\sqrt{2}} = 2\sqrt{2}

,

c_2 = \mathbf{x} \cdot \mathbf{q}_2 = \frac{1}{\sqrt{6}}(3 - 1 + 4) = \frac{6}{\sqrt{6}} = \sqrt{6}

,

c_3 = \mathbf{x} \cdot \mathbf{q}_3 = \frac{1}{\sqrt{3}}(-3 + 1 + 2) = 0

.

So

\mathbf{x} = 2\sqrt{2}\,\mathbf{q}_1 + \sqrt{6}\,\mathbf{q}_2 + 0 \cdot \mathbf{q}_3

. The zero third coordinate means

\mathbf{x}

has no component in the

\mathbf{q}_3

direction — it is orthogonal to

\mathbf{q}_3

.

Pitfall: Dropping the denominator

Using

c_i = \mathbf{x} \cdot \mathbf{v}_i

for the set

(1, 1, 1)

,

(1, -2, 1)

,

(1, 0, -1)

and

\mathbf{x} = (3, 1, 2)

gives

(6, 3, 1)

, which rebuilds

(10, 0, 8)

, not

\mathbf{x}

. Dividing by

3

,

6

and

2

gives

(2, \tfrac{1}{2}, \tfrac{1}{2})

.

Orthogonal Matrices

An

n \times n

matrix

Q

is orthogonal if its columns form an orthonormal set. This is equivalent to

Q^TQ = QQ^T = I

, which is equivalent to

Q^{-1} = Q^T

.

THEOREM

The rows of an orthogonal matrix also form an orthonormal set — orthogonality of columns and rows go together.

The determinant of an orthogonal matrix is

\pm 1

, since

1 = \det(I) = \det(Q^TQ) = \det(Q)^2

. When

\det(Q) = +1

, the matrix represents a rotation. When

\det(Q) = -1

, it represents a rotation composed with a reflection.

Pitfall: Stopping at orthogonal columns

Taking

M^{-1} = M^T

for

M = \begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix}

, whose columns are perpendicular but of length

\sqrt{2}

, fails:

M^TM = 2I

, so

M^{-1} = \tfrac{1}{2}M^T

, and

\det M = -2

. Dividing each column by

\sqrt{2}

gives an orthogonal matrix.

Matrices with Orthonormal Columns

An

m \times n

matrix

Q

with

m > n

can have orthonormal columns without being square. Such a matrix satisfies

Q^TQ = I_n

but

QQ^T \neq I_m

(the product

QQ^T

is

m \times m

and has rank

n < m

).

The matrix

QQ^T

is the projection matrix onto the column space of

Q

. For any

\mathbf{b} \in \mathbb{R}^m

, the vector

QQ^T\mathbf{b}

is the orthogonal projection of

\mathbf{b}

onto the

n

-dimensional subspace spanned by the columns of

Q

.

These rectangular matrices with orthonormal columns are the natural output of the Gram-Schmidt process applied to the columns of a matrix. If

A

is

m \times n

with independent columns, Gram-Schmidt produces an

m \times n

matrix

Q

with orthonormal columns and an

n \times n

upper triangular matrix

R

such that

A = QR

. This is the thin QR decomposition.

Property	Square orthogonal matrix Q (n × n)	Rectangular Q with orthonormal columns (m × n, m > n)
Qᵀ Q	I_n	I_n
Q Qᵀ	I_n	NOT the identity; equals the projection matrix onto Col(Q)
Inverse	Q⁻¹ = Qᵀ	no two-sided inverse (Q is not square)
det(Q)	±1	undefined (Q is not square)
Rows	also form an orthonormal set	rows are not orthonormal in general
Preserves dot product?	yes for all u, v ∈ ℝⁿ — full isometry of ℝⁿ	yes from source to image: (Q u) · (Q v) = u · v for u, v ∈ ℝⁿ
Geometric role	rotation (det +1) or rotation composed with reflection (det −1)	embedding ℝⁿ into ℝᵐ as an n-dimensional subspace
Typical source	stored basis change, rotation matrix, Householder/Givens factor	output of Gram–Schmidt; the Q in thin QR decomposition A = Q R

A tall matrix with orthonormal columns inverts on one side only:

Q^TQ = I

, while

QQ^T

merely projects.

Parseval's Identity and Bessel's Inequality

For an orthonormal basis

\{\mathbf{q}_1, \dots, \mathbf{q}_n\}

of

\mathbb{R}^n

and any vector

\mathbf{x}

, the coordinates

c_i = \mathbf{x} \cdot \mathbf{q}_i

satisfy Parseval's identity:

Parseval Identity

\|\mathbf{x}\|^2 = \sum_{i=1}^{n} (\mathbf{x} \cdot \mathbf{q}_i)^2

Parseval identity →

The squared length of

\mathbf{x}

equals the sum of the squares of its coordinates. This is the Pythagorean theorem applied to the orthonormal decomposition

\mathbf{x} = c_1\mathbf{q}_1 + \cdots + c_n\mathbf{q}_n

.

When the orthonormal set does not span — when

k < n

— the sum accounts for only part of the length:

Parseval's Identity and Bessel's Inequality

\sum_{i=1}^{k} (\mathbf{x} \cdot \mathbf{q}_i)^2 \leq \|\mathbf{x}\|^2

This is Bessel's inequality. The left side is the squared length of the projection of

\mathbf{x}

onto

\text{Span}\{\mathbf{q}_1, \dots, \mathbf{q}_k\}

. The deficit

\|\mathbf{x}\|^2 - \sum(\mathbf{x} \cdot \mathbf{q}_i)^2

is the squared length of the component orthogonal to the span. Equality holds if and only if

\mathbf{x}

is already in the span, leaving no perpendicular remainder.

In an orthonormal basis, squared length splits exactly among the coordinates, and whatever a partial set misses lies perpendicular to it.

Isometries: Maps That Preserve Length

An isometry is a linear map that preserves length:

\|T\mathbf{x}\| = \|\mathbf{x}\|

for every

\mathbf{x}

. Because the dot product can be recovered from lengths alone through the polarization identity

Isometries: Maps That Preserve Length

\mathbf{x} \cdot \mathbf{y} = \tfrac{1}{2}\left(\|\mathbf{x} + \mathbf{y}\|^2 - \|\mathbf{x}\|^2 - \|\mathbf{y}\|^2\right)

preserving every length forces

T

to preserve every dot product, and with it every angle and every distance between points. Nothing is stretched, squashed or sheared.

For a matrix

Q

this is exactly the orthogonality condition of the earlier sections. If

Q^TQ = I

then

\|Q\mathbf{x}\|^2 = \mathbf{x}^TQ^TQ\mathbf{x} = \|\mathbf{x}\|^2

. Conversely, a

Q

that preserves all dot products satisfies

(Q\mathbf{e}_i) \cdot (Q\mathbf{e}_j) = \mathbf{e}_i \cdot \mathbf{e}_j

, so its columns are orthonormal and

Q^TQ = I

.

THEOREM

The linear isometries of

\mathbb{R}^n

are precisely the orthogonal matrices.

They split into two families by the determinant. Those with

\det Q = 1

are rotations, which preserve orientation; those with

\det Q = -1

reverse it, and in the plane these are exactly the reflections in a line through the origin. In space an isometry is a rotation about an axis, a reflection in a plane, or a rotation combined with a reflection. Translations also preserve distance but move the origin, so they are not linear and fall outside this classification.

The reflection tool shows the orientation-reversing family in the plane, and the orthogonal matrices tool shows the length-preservation check

Q^TQ = I

directly.

Length is enough: a linear map that preserves every length preserves all of the geometry, angles and distances included.

Summary: Operations in an Orthonormal Basis

An orthonormal basis converts every routine vector operation into something that uses only dot products — no system solving, no matrix inversion, no (AᵀA)⁻¹ factor. The table below collects each operation alongside its formula in an orthonormal basis and a brief note on why orthonormality makes it simple, including Parseval's identity for length and Bessel's inequality for partial projections.

Every row below is an operation that becomes simpler in an orthonormal basis, and the simplification is always the same one: a linear system turns into a dot product. Reading them together shows that these are not six separate conveniences but six consequences of a single fact about the basis vectors.

DownloadPNG2×PDF

Orthogonality · orthonormal bases

Operations in an orthonormal basis

Each row is an operation that costs a linear system in a general basis and costs a dot product here. The saving is not incremental — the system disappears entirely, because orthogonality makes the coordinates independent of each other.

6operations

Coordinates2

1

Finding a coordinate§ 5

one dot product per coordinate

c_i = \mathbf{x} \cdot \mathbf{q}_i

In a general basis this means solving

B\mathbf{c} = \mathbf{x}

— all coordinates at once, none available before the rest. Here each is an independent dot product, so any single coordinate can be computed without the others. This is the entry the rest depend on.

2

Reconstruction§ 5

over a full orthonormal basis

\mathbf{x} = \sum (\mathbf{x} \cdot \mathbf{q}_i)\,\mathbf{q}_i

The expansion written out. Each term is the component of

\mathbf{x}

along one basis direction, so the sum is a decomposition into perpendicular pieces rather than an arbitrary combination that happens to work.

Lengths and angles2

3

Squared length§ 8

Parseval’s identity

\|\mathbf{x}\|^2 = \sum c_i^2

Pythagoras in

n

dimensions. The length of a vector is recoverable from its coordinates alone — which fails in a general basis, where the coordinates say nothing about length without the Gram matrix to weight them.

4

Inner product§ 8

coordinates in the same basis

\mathbf{x} \cdot \mathbf{y} = \sum c_i d_i

The inner product of the vectors is the ordinary dot product of their coordinate vectors. Geometry is preserved exactly under the change of coordinates, which is what makes an orthonormal basis a rigid motion rather than a distortion.

Projection onto a subspace2

5

Projection§ 4

\{\mathbf{q}_i\}

orthonormal basis for

W

\operatorname{proj}_W \mathbf{x} = \sum_{i=1}^{k} (\mathbf{x} \cdot \mathbf{q}_i)\,\mathbf{q}_i

No

(A^{\mathsf{T}}A)^{-1}

anywhere — compare the general projection formula, which needs a matrix inverse. Orthonormality makes

A^{\mathsf{T}}A = I

, so the inverse is the identity and the formula collapses to a sum of independent terms.

6

Bessel’s inequality§ 8

equality iff

\mathbf{x} \in W

\sum_{i=1}^{k} c_i^2 \leq \|\mathbf{x}\|^2

A truncated sum can only lose length. The deficit is exactly the squared distance from

\mathbf{x}

to

W

, so the inequality measures how much of the vector the subspace fails to capture — which is the least-squares error before any least-squares problem is posed.

One fact produces all of them:

\mathbf{q}_i \cdot \mathbf{q}_j

is

1

when

i = j

and

0

otherwise, so every cross term in every expansion vanishes. That is why an orthonormal basis is worth the Gram–Schmidt cost of constructing one — the expense is paid once and every operation afterwards is cheaper.

Operations in an orthonormal basis·/linear-algebra/orthogonality/orthogonal-setsLearn Math Class

What changes is that the coordinates stop depending on each other. In a general basis, finding one coordinate of

\mathbf{x}

means solving for all of them at once — the system

B\mathbf{c} = \mathbf{x}

couples every unknown to every other. With an orthonormal basis each coordinate is an independent dot product, so a single one can be computed alone, and adding a new basis vector does not change the coordinates already found.

The projection row is where this pays off most visibly. The general formula requires

(A^{\mathsf{T}}A)^{-1}

, a matrix inverse computed before anything can be projected. When the columns are orthonormal,

A^{\mathsf{T}}A

is the identity, the inverse disappears, and the projection is a sum of independent terms — which is precisely why Gram–Schmidt is worth running before a projection-heavy computation rather than after.

Common Pitfalls

Three mistakes cause many wrong answers with orthogonal sets. Each is marked with a box where it happens in the sections above. Here each is worked through in full, with the wrong and the right working side by side.

Dividing by the squared length

Normalizing a vector means dividing it by its length

\|\mathbf{v}\| = \sqrt{\mathbf{v} \cdot \mathbf{v}}

. The square root is left out, and the vector is divided by

\mathbf{v} \cdot \mathbf{v}

instead.

Wrong working:

\mathbf{v} = (1, -2, 1), \quad \mathbf{v} \cdot \mathbf{v} = 6, \quad \frac{\mathbf{v}}{6} = \left(\tfrac{1}{6}, -\tfrac{1}{3}, \tfrac{1}{6}\right)

Its length is

\sqrt{\tfrac{1}{36} + \tfrac{1}{9} + \tfrac{1}{36}} = \sqrt{\tfrac{1}{6}} \approx 0.41

, so the result is not a unit vector.

Right working:

\|\mathbf{v}\| = \sqrt{6}, \quad \hat{\mathbf{v}} = \frac{1}{\sqrt{6}}(1, -2, 1), \quad \hat{\mathbf{v}} \cdot \hat{\mathbf{v}} = \frac{1 + 4 + 1}{6} = 1

Why it happens: the squared length

\mathbf{v} \cdot \mathbf{v}

is the number already at hand. It is the denominator of the coordinate formula

c_i = \frac{\mathbf{x} \cdot \mathbf{v}_i}{\mathbf{v}_i \cdot \mathbf{v}_i}

and of projection formulas, while normalizing needs the length itself.

How to check: take the dot product of the result with itself; it must be

1

. After dividing by

\mathbf{v} \cdot \mathbf{v}

it is

\frac{1}{\mathbf{v} \cdot \mathbf{v}}

instead, here

\tfrac{1}{6}

.

Dropping the denominator

The coordinate formula for an orthonormal basis,

c_i = \mathbf{x} \cdot \mathbf{q}_i

, is applied to an orthogonal basis whose vectors are not unit vectors. The denominator

\mathbf{v}_i \cdot \mathbf{v}_i

equals

1

only when

\|\mathbf{v}_i\| = 1

.

Wrong working: with

\mathbf{v}_1 = (1, 1, 1)

,

\mathbf{v}_2 = (1, -2, 1)

,

\mathbf{v}_3 = (1, 0, -1)

and

\mathbf{x} = (3, 1, 2)

,

\mathbf{x} \cdot \mathbf{v}_1 = 6, \quad \mathbf{x} \cdot \mathbf{v}_2 = 3, \quad \mathbf{x} \cdot \mathbf{v}_3 = 1

Taking these as the coordinates gives

6\mathbf{v}_1 + 3\mathbf{v}_2 + 1\mathbf{v}_3 = (10, 0, 8)

, not

\mathbf{x}

.

Right working: divide each dot product by

\mathbf{v}_i \cdot \mathbf{v}_i

, which is

3

,

6

and

2

.

c_1 = \frac{6}{3} = 2, \quad c_2 = \frac{3}{6} = \frac{1}{2}, \quad c_3 = \frac{1}{2}

Then

2\mathbf{v}_1 + \tfrac{1}{2}\mathbf{v}_2 + \tfrac{1}{2}\mathbf{v}_3 = (2 + \tfrac{1}{2} + \tfrac{1}{2}, \; 2 - 1 + 0, \; 2 + \tfrac{1}{2} - \tfrac{1}{2}) = (3, 1, 2)

.

Why it happens: the orthonormal formula is the one that gets remembered, because it is shorter and worked examples tend to use unit vectors. Orthogonal sets with whole-number entries, such as this one, are rarely normalized.

How to check: rebuild

\mathbf{x}

from the coordinates. With the denominator missing, each coordinate is too large by the factor

\mathbf{v}_i \cdot \mathbf{v}_i

.

Stopping at orthogonal columns

A square matrix whose columns are pairwise perpendicular is treated as an orthogonal matrix, and its transpose is used as its inverse. The definition also requires every column to have length

1

.

Wrong working:

M = \begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix}, \quad M^{-1} = M^T = \begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix}

But

M^TM = \begin{pmatrix} 2 & 0 \\ 0 & 2 \end{pmatrix} = 2I

, not

I

, so

M^T

is not the inverse.

Right working: the columns have length

\sqrt{2}

, so

M^{-1} = \tfrac{1}{2}M^T = \begin{pmatrix} \tfrac{1}{2} & \tfrac{1}{2} \\ \tfrac{1}{2} & -\tfrac{1}{2} \end{pmatrix}

. Dividing each column by

\sqrt{2}

gives the orthogonal matrix

Q = \frac{1}{\sqrt{2}}\begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix}

, with

Q^TQ = I

and

Q^{-1} = Q^T

.

Why it happens: the name says orthogonal, and the columns are orthogonal. Unit length is part of the definition but not of the name.

How to check: compute

M^TM

. A diagonal result with entries other than

1

means the columns are orthogonal but not unit length. The determinant is a quick test too: an orthogonal matrix has

\det = \pm 1

, and here

\det M = -2

.

Orthogonal Sets FAQ

Does an "orthogonal matrix" just have orthogonal columns?

+

No, they must be orthonormal, so unit length is part of the definition despite the name. A matrix whose columns are merely perpendicular satisfies no identity of this kind. The terminology is a historical accident that the notation refuses to repeat, which is why the condition is always written

Q^{T}Q = I

rather than described in words.Read more →

What is $\delta_{ij}$ ?

+

The Kronecker delta, a function of two indices returning

1

when they agree and

0

otherwise. It is the identity matrix written entrywise,

I = [\delta_{ij}]

. Do not read it as the

\delta

of calculus: nothing here is small or an increment, and this symbol takes two subscripts rather than measuring a magnitude.Read more →

If $Q^{T}Q = I$ , does $QQ^{T} = I$ as well?

+

Yes, for square

Q

, and this is one of the few places where such a swap is safe. The equation is a statement about the columns being orthonormal, and it forces the rows to be orthonormal too. Most matrix identities are not symmetric in that way, so the fact is worth noting rather than assuming.Read more →

Can an orthogonal set contain the zero vector?

+

No, the definition requires every vector to be nonzero. The set

\{(1, 0), (0, 0)\}

passes the dot-product test only because

\mathbf{0}

is orthogonal to everything, and

\mathbf{0}

carries no direction. Allowing it would also break the guarantee that orthogonal sets are independent:

0 \cdot (1, 0) + 1 \cdot (0, 0) = \mathbf{0}

is a combination with a nonzero coefficient that still gives zero.Read more →

For which $n$ is there an orthogonal basis of $\mathbb{R}^n$ with every entry $\pm 1$ ?

+

For

n = 1

,

n = 2

, and otherwise only when

n

is a multiple of

4

. In

\mathbb{R}^4

, for example,

(1, 1, 1, 1)

,

(1, -1, 1, -1)

,

(1, 1, -1, -1)

and

(1, -1, -1, 1)

are pairwise orthogonal. For

n \geq 3

, any three such vectors sort the

n

positions into four equal groups by which signs agree, which forces the multiple of

4

. Whether every multiple of

4

actually works is still unproved.Read more →

Does adding a vector to an orthonormal set change the coefficients already found?

+

No. Each coefficient

c_i = \mathbf{x} \cdot \mathbf{q}_i

uses only

\mathbf{x}

and

\mathbf{q}_i

, so extending the set from

k

to

k + 1

vectors adds one new term and leaves the first

k

untouched. With a set that is not orthogonal, the coefficients come from a linear system that couples every unknown to every other, so one new vector can change all of them.Read more →

Orthogonal Sets

Bases Where Coordinates Come Free

Orthogonal Sets

Orthogonal Set Notation

Orthogonal Set Notation

Orthogonal Sets Are Independent

Orthonormal Sets

Orthogonal and Orthonormal Bases

Coordinates via Dot Products

Orthogonal Matrices

Matrices with Orthonormal Columns

Parseval's Identity and Bessel's Inequality

Isometries: Maps That Preserve Length

Summary: Operations in an Orthonormal Basis

Operations in an orthonormal basis

Common Pitfalls

Orthogonal Sets FAQ

Does an "orthogonal matrix" just have orthogonal columns?

What is δij\delta_{ij}δij​?

If QTQ=IQ^{T}Q = IQTQ=I, does QQT=IQQ^{T} = IQQT=I as well?

Can an orthogonal set contain the zero vector?

For which nnn is there an orthogonal basis of Rn\mathbb{R}^nRn with every entry ±1\pm 1±1?

Does adding a vector to an orthonormal set change the coefficients already found?

What is $\delta_{ij}$ ?

If $Q^{T}Q = I$ , does $QQ^{T} = I$ as well?

For which $n$ is there an orthogonal basis of $\mathbb{R}^n$ with every entry $\pm 1$ ?