Orthogonal & Orthonormal Sets and Bases

Orthogonal Sets

A set of vectors

\{\mathbf{v}_1, \mathbf{v}_2, \dots, \mathbf{v}_k\}

is orthogonal if every pair has dot product zero:

\mathbf{v}_i \cdot \mathbf{v}_j = 0 \quad \text{for all } i \neq j

The vectors in an orthogonal set must all be nonzero — the zero vector is excluded because including it would trivialize the structure (every vector is orthogonal to

\mathbf{0}

, so

\mathbf{0}

carries no directional information).

For example,

\{(1, 0, 0), (0, 2, 0), (0, 0, -3)\}

is orthogonal in

\mathbb{R}^3

: every pair of distinct vectors has dot product zero. The vectors need not have the same length, and their lengths can be anything nonzero.

A less obvious example:

\{(1, 1, 1), (1, -2, 1), (1, 0, -1)\}

. Checking:

(1)(1) + (1)(-2) + (1)(1) = 0

,

(1)(1) + (1)(0) + (1)(-1) = 0

,

(1)(1) + (-2)(0) + (1)(-1) = 0

. All three pairwise products vanish — the set is orthogonal despite none of the vectors being aligned with the coordinate axes.

A set with every pair at right angles

Every pair in this set has a dot product of zero. That single condition forces independence — no vector can be a combination of the others without picking up a non-zero dot product with one of them. Independence therefore comes free with orthogonality rather than needing a separate check. Build such a set from arbitrary vectors on the Gram-Schmidt visualizer.

The zero vector has to be excluded by hand, since it is orthogonal to everything and would wreck the independence it otherwise guarantees.

Orthogonal Set Notation

Notation

Orthogonal Set Notation

Two conditions — mutual perpendicularity and unit length — that most of this subject states in prose, compressed by one symbol into a single equation. Around it sit a hat that means a specific normalisation and a matrix letter whose name does not match its defining property. All catalogued among the linear algebra symbols.

The dot and

\perp

come from dot product and inner product notation;

\|\mathbf{v}\|

and the hat from norm notation;

A^{T}

,

I

and

A^{-1}

from matrix operation and inverse notation; subscripted index pairs from matrix entry notation.

\mathbf{v}_i \cdot \mathbf{v}_j = \delta_{ij}

the dot product is one when the indices agree and zero otherwise

The Kronecker delta is a switch written as a symbol:

\delta_{ij} = 1

when

i = j

and

0

otherwise. It compresses the whole definition of an orthonormal set into one equation — perpendicularity is the

i \neq j

case, unit length the

i = j

case, as Orthonormal Sets below states.

CasesThe same symbol is the identity matrix entrywise,

I = [\delta_{ij}]

, which is why

Q^{T}Q = I

and "the columns are orthonormal" are the same statement written two ways — one in matrix marks, one in index marks.

Do not confuseThe

\Delta

and

\delta

of calculus — an increment or a small quantity. Here

\delta

takes two subscripts and returns one of two values; it is a function of a pair of indices, not a magnitude, and nothing about it is small.

\hat{\mathbf{u}} = \dfrac{\mathbf{v}}{\|\mathbf{v}\|}

u-hat — the unit vector in the direction of v

The hat is a claim about length, not about identity: it says this vector has been divided by its own norm and now measures

1

. Normalising is the single step that turns an orthogonal set into an orthonormal one, so the hat marks exactly what the extra word "normal" contributes.

CasesThe standard basis carries hats for the same reason —

\hat{\imath}, \hat{\jmath}, \hat{k}

in physics for what this site writes

\mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3

— and the hat is dropped once every vector in play is known to be a unit vector.

Same glyph elsewhereIn statistics the same hat marks an estimate rather than a length —

\hat{\mu}

is an estimated mean, with no implication that anything equals one. The two-hats convention keeps the jobs apart.

Q^{T}Q = I

, so

Q^{-1} = Q^{T}

Q is orthogonal: its transpose is its inverse

Q

is the reserved letter for a matrix with orthonormal columns, and the defining equation is the payoff — Orthogonal Matrices below — since it replaces inversion, normally expensive, with transposition, which is free.

CasesThe equation is a statement about columns, and it forces the same of rows, so

QQ^{T} = I

holds too — one of the few places where a matrix identity is symmetric in that way.

Do not confuseThe name itself. An "orthogonal matrix" has orthonormal columns, not merely orthogonal ones — unit length is part of the definition, and a matrix whose columns are perpendicular but not normalised satisfies no such identity. The terminology is a historical accident the notation does not repeat.

All linear algebra symbols →↑ Orthogonality

Orthogonal Sets Are Independent

Every orthogonal set of nonzero vectors is linearly independent:

Orthogonal Set Independence

\{\mathbf{v}_1, \dots, \mathbf{v}_k\} \text{ orthogonal, all nonzero} \implies \text{linearly independent}

Orthogonal set independence →

The proof is short and reveals exactly why orthogonality is so powerful.

Suppose

c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \cdots + c_k\mathbf{v}_k = \mathbf{0}

. Dot both sides with

\mathbf{v}_j

:

c_1(\mathbf{v}_1 \cdot \mathbf{v}_j) + c_2(\mathbf{v}_2 \cdot \mathbf{v}_j) + \cdots + c_k(\mathbf{v}_k \cdot \mathbf{v}_j) = 0

Every term with

i \neq j

vanishes because

\mathbf{v}_i \cdot \mathbf{v}_j = 0

. Only the

j

-th term survives:

c_j \|\mathbf{v}_j\|^2 = 0

. Since

\mathbf{v}_j \neq \mathbf{0}

,

\|\mathbf{v}_j\|^2 > 0

, so

c_j = 0

. This works for every

j

, so all coefficients are zero.

The key mechanism is that orthogonality isolates each coefficient. Dotting with

\mathbf{v}_j

kills every other term, leaving

c_j

alone. This is why orthogonal bases make coordinates computable by individual dot products — the same isolation principle that proves independence also extracts coordinates.

Orthonormal Sets

An orthonormal set is an orthogonal set where every vector additionally has unit length:

\|\mathbf{v}_i\| = 1

for all

i

. The two conditions together can be written compactly using the Kronecker delta:

Orthonormal Set

\mathbf{v}_i \cdot \mathbf{v}_j = \delta_{ij} = \begin{cases} 1 & \text{if } i = j \\ 0 & \text{if } i \neq j \end{cases}

Orthonormal set →

Any orthogonal set can be made orthonormal by normalizing each vector:

\hat{\mathbf{v}}_i = \mathbf{v}_i / \|\mathbf{v}_i\|

. The directions are preserved, only the lengths change to

1

.

The standard basis

\{\mathbf{e}_1, \mathbf{e}_2, \dots, \mathbf{e}_n\}

for

\mathbb{R}^n

is orthonormal:

\mathbf{e}_i \cdot \mathbf{e}_j = \delta_{ij}

because each basis vector has a single

1

in a different position. It is the simplest orthonormal set, but far from the only one.

Aspect	Orthogonal set	Orthonormal set
Pairwise dot products	v_i · v_j = 0 for i ≠ j	v_i · v_j = 0 for i ≠ j
Vector lengths	any nonzero value	each ‖v_i‖ = 1
Compact condition	v_i · v_j = 0 (i ≠ j) plus each v_i ≠ 0	v_i · v_j = δ_ij (the Kronecker delta)
Coordinate of x along v_i	c_i = (v_i · x) / (v_i · v_i)	c_i = q_i · x
Linear independence	automatic (nonzero vectors)	automatic (unit vectors are nonzero)
Going between the two	divide each v_i by ‖v_i‖ → orthonormal	already orthogonal as a special case

Orthogonal and Orthonormal Bases

An orthogonal basis is an orthogonal set that spans the space. An orthonormal basis is an orthonormal set that spans the space.

In

\mathbb{R}^n

, an orthogonal set of

n

nonzero vectors is automatically a basis — independence is guaranteed by orthogonality, and

n

independent vectors in an

n

-dimensional space automatically span. So the only check needed is: do I have

n

pairwise-orthogonal nonzero vectors? If yes, they form a basis.

Orthonormal bases exist for every finite-dimensional inner product space. The Gram-Schmidt process constructs one from any given basis. This means the computational advantages of orthonormal bases are always available — any space that has a basis at all has an orthonormal one.

Coordinates via Dot Products

The defining computational advantage of orthogonal bases is that coordinates are extracted by individual dot products.

For an orthogonal basis

\{\mathbf{v}_1, \dots, \mathbf{v}_n\}

, the coordinate of

\mathbf{x}

along

\mathbf{v}_i

is

c_i = \frac{\mathbf{x} \cdot \mathbf{v}_i}{\mathbf{v}_i \cdot \mathbf{v}_i}

For an orthonormal basis

\{\mathbf{q}_1, \dots, \mathbf{q}_n\}

, the denominator is

1

, and the formula simplifies to

Coordinates via Orthonormal Basis

c_i = \mathbf{x} \cdot \mathbf{q}_i

Coordinates via orthonormal basis →

No linear system needs to be solved. No matrix needs to be inverted. Each coordinate is computed independently by a single dot product.

Worked Example

Let

\{\mathbf{q}_1, \mathbf{q}_2, \mathbf{q}_3\}

be an orthonormal basis for

\mathbb{R}^3

with

\mathbf{q}_1 = \frac{1}{\sqrt{2}}(1, 1, 0)

,

\mathbf{q}_2 = \frac{1}{\sqrt{6}}(1, -1, 2)

,

\mathbf{q}_3 = \frac{1}{\sqrt{3}}(-1, 1, 1)

.

For

\mathbf{x} = (3, 1, 2)

:

c_1 = \mathbf{x} \cdot \mathbf{q}_1 = \frac{1}{\sqrt{2}}(3 + 1 + 0) = \frac{4}{\sqrt{2}} = 2\sqrt{2}

,

c_2 = \mathbf{x} \cdot \mathbf{q}_2 = \frac{1}{\sqrt{6}}(3 - 1 + 4) = \frac{6}{\sqrt{6}} = \sqrt{6}

,

c_3 = \mathbf{x} \cdot \mathbf{q}_3 = \frac{1}{\sqrt{3}}(-3 + 1 + 2) = 0

.

So

\mathbf{x} = 2\sqrt{2}\,\mathbf{q}_1 + \sqrt{6}\,\mathbf{q}_2 + 0 \cdot \mathbf{q}_3

. The zero third coordinate means

\mathbf{x}

has no component in the

\mathbf{q}_3

direction — it is orthogonal to

\mathbf{q}_3

.

Orthogonal Matrices

An

n \times n

matrix

Q

is orthogonal if its columns form an orthonormal set. This is equivalent to

Q^TQ = QQ^T = I

, which is equivalent to

Q^{-1} = Q^T

.

The rows of an orthogonal matrix also form an orthonormal set — orthogonality of columns and rows go together.

The determinant of an orthogonal matrix is

\pm 1

, since

1 = \det(I) = \det(Q^TQ) = \det(Q)^2

. When

\det(Q) = +1

, the matrix represents a rotation. When

\det(Q) = -1

, it represents a rotation composed with a reflection.

The defining geometric property is that orthogonal matrices preserve the dot product:

(Q\mathbf{u}) \cdot (Q\mathbf{v}) = (Q\mathbf{u})^T(Q\mathbf{v}) = \mathbf{u}^TQ^TQ\mathbf{v} = \mathbf{u}^T\mathbf{v} = \mathbf{u} \cdot \mathbf{v}

. Preserving the dot product automatically preserves lengths (

\|Q\mathbf{x}\| = \|\mathbf{x}\|

), angles, and distances. An orthogonal matrix is a rigid motion of

\mathbb{R}^n

— it rearranges vectors without distorting any geometric relationship.

Columns of length one, pairwise orthogonal

Each column has unit length and every pair is orthogonal, which is exactly the condition that makes the transpose an inverse. Nothing stronger is being asked of the matrix, and nothing weaker would do. Check the transpose-times-itself product on the orthogonal matrices visualizer.

Because the inverse is free, orthogonal matrices are the ones numerical work prefers wherever a choice exists.

Matrices with Orthonormal Columns

An

m \times n

matrix

Q

with

m > n

can have orthonormal columns without being square. Such a matrix satisfies

Q^TQ = I_n

but

QQ^T \neq I_m

(the product

QQ^T

is

m \times m

and has rank

n < m

).

The matrix

QQ^T

is the projection matrix onto the column space of

Q

. For any

\mathbf{b} \in \mathbb{R}^m

, the vector

QQ^T\mathbf{b}

is the orthogonal projection of

\mathbf{b}

onto the

n

-dimensional subspace spanned by the columns of

Q

.

These rectangular matrices with orthonormal columns are the natural output of the Gram-Schmidt process applied to the columns of a matrix. If

A

is

m \times n

with independent columns, Gram-Schmidt produces an

m \times n

matrix

Q

with orthonormal columns and an

n \times n

upper triangular matrix

R

such that

A = QR

. This is the thin QR decomposition.

Property	Square orthogonal matrix Q (n × n)	Rectangular Q with orthonormal columns (m × n, m > n)
Qᵀ Q	I_n	I_n
Q Qᵀ	I_n	NOT the identity; equals the projection matrix onto Col(Q)
Inverse	Q⁻¹ = Qᵀ	no two-sided inverse (Q is not square)
det(Q)	±1	undefined (Q is not square)
Rows	also form an orthonormal set	rows are not orthonormal in general
Preserves dot product?	yes for all u, v ∈ ℝⁿ — full isometry of ℝⁿ	yes from source to image: (Q u) · (Q v) = u · v for u, v ∈ ℝⁿ
Geometric role	rotation (det +1) or rotation composed with reflection (det −1)	embedding ℝⁿ into ℝᵐ as an n-dimensional subspace
Typical source	stored basis change, rotation matrix, Householder/Givens factor	output of Gram–Schmidt; the Q in thin QR decomposition A = Q R

Parseval's Identity and Bessel's Inequality

For an orthonormal basis

\{\mathbf{q}_1, \dots, \mathbf{q}_n\}

of

\mathbb{R}^n

and any vector

\mathbf{x}

, the coordinates

c_i = \mathbf{x} \cdot \mathbf{q}_i

satisfy Parseval's identity:

Parseval Identity

\|\mathbf{x}\|^2 = \sum_{i=1}^{n} (\mathbf{x} \cdot \mathbf{q}_i)^2

Parseval identity →

The squared length of

\mathbf{x}

equals the sum of the squares of its coordinates. This is the Pythagorean theorem applied to the orthonormal decomposition

\mathbf{x} = c_1\mathbf{q}_1 + \cdots + c_n\mathbf{q}_n

.

When the orthonormal set does not span — when

k < n

— the sum accounts for only part of the length:

\sum_{i=1}^{k} (\mathbf{x} \cdot \mathbf{q}_i)^2 \leq \|\mathbf{x}\|^2

This is Bessel's inequality. The left side is the squared length of the projection of

\mathbf{x}

onto

\text{Span}\{\mathbf{q}_1, \dots, \mathbf{q}_k\}

. The deficit

\|\mathbf{x}\|^2 - \sum(\mathbf{x} \cdot \mathbf{q}_i)^2

is the squared length of the component orthogonal to the span. Equality holds if and only if

\mathbf{x}

is already in the span, leaving no perpendicular remainder.

Summary: Operations in an Orthonormal Basis

An orthonormal basis converts every routine vector operation into something that uses only dot products — no system solving, no matrix inversion, no (AᵀA)⁻¹ factor. The table below collects each operation alongside its formula in an orthonormal basis and a brief note on why orthonormality makes it simple, including Parseval's identity for length and Bessel's inequality for partial projections.

Every row below is an operation that becomes simpler in an orthonormal basis, and the simplification is always the same one: a linear system turns into a dot product. Reading them together shows that these are not six separate conveniences but six consequences of a single fact about the basis vectors.

Download

PNGRaster · 2×PDFPrint quality

Orthogonality · orthonormal bases

Operations in an orthonormal basis

Each row is an operation that costs a linear system in a general basis and costs a dot product here. The saving is not incremental — the system disappears entirely, because orthogonality makes the coordinates independent of each other.

6operations

Coordinates2

1

Finding a coordinate§ 5

one dot product per coordinate

c_i = \mathbf{x} \cdot \mathbf{q}_i

In a general basis this means solving

B\mathbf{c} = \mathbf{x}

— all coordinates at once, none available before the rest. Here each is an independent dot product, so any single coordinate can be computed without the others. This is the entry the rest depend on.

2

Reconstruction§ 5

over a full orthonormal basis

\mathbf{x} = \sum (\mathbf{x} \cdot \mathbf{q}_i)\,\mathbf{q}_i

The expansion written out. Each term is the component of

\mathbf{x}

along one basis direction, so the sum is a decomposition into perpendicular pieces rather than an arbitrary combination that happens to work.

Lengths and angles2

3

Squared length§ 8

Parseval’s identity

\|\mathbf{x}\|^2 = \sum c_i^2

Pythagoras in

n

dimensions. The length of a vector is recoverable from its coordinates alone — which fails in a general basis, where the coordinates say nothing about length without the Gram matrix to weight them.

4

Inner product§ 8

coordinates in the same basis

\mathbf{x} \cdot \mathbf{y} = \sum c_i d_i

The inner product of the vectors is the ordinary dot product of their coordinate vectors. Geometry is preserved exactly under the change of coordinates, which is what makes an orthonormal basis a rigid motion rather than a distortion.

Projection onto a subspace2

5

Projection§ 4

\{\mathbf{q}_i\}

orthonormal basis for

W

\operatorname{proj}_W \mathbf{x} = \sum_{i=1}^{k} (\mathbf{x} \cdot \mathbf{q}_i)\,\mathbf{q}_i

No

(A^{\mathsf{T}}A)^{-1}

anywhere — compare the general projection formula, which needs a matrix inverse. Orthonormality makes

A^{\mathsf{T}}A = I

, so the inverse is the identity and the formula collapses to a sum of independent terms.

6

Bessel’s inequality§ 8

equality iff

\mathbf{x} \in W

\sum_{i=1}^{k} c_i^2 \leq \|\mathbf{x}\|^2

A truncated sum can only lose length. The deficit is exactly the squared distance from

\mathbf{x}

to

W

, so the inequality measures how much of the vector the subspace fails to capture — which is the least-squares error before any least-squares problem is posed.

One fact produces all of them:

\mathbf{q}_i \cdot \mathbf{q}_j

is

1

when

i = j

and

0

otherwise, so every cross term in every expansion vanishes. That is why an orthonormal basis is worth the Gram–Schmidt cost of constructing one — the expense is paid once and every operation afterwards is cheaper.

Operations in an orthonormal basis·/linear-algebra/orthogonality/orthogonal-setsLearn Math Class

What changes is that the coordinates stop depending on each other. In a general basis, finding one coordinate of

\mathbf{x}

means solving for all of them at once — the system

B\mathbf{c} = \mathbf{x}

couples every unknown to every other. With an orthonormal basis each coordinate is an independent dot product, so a single one can be computed alone, and adding a new basis vector does not change the coordinates already found.

The projection row is where this pays off most visibly. The general formula requires

(A^{\mathsf{T}}A)^{-1}

, a matrix inverse computed before anything can be projected. When the columns are orthonormal,

A^{\mathsf{T}}A

is the identity, the inverse disappears, and the projection is a sum of independent terms — which is precisely why Gram–Schmidt is worth running before a projection-heavy computation rather than after.

Orthogonal Sets FAQ

Does an "orthogonal matrix" just have orthogonal columns?

+

No, they must be orthonormal, so unit length is part of the definition despite the name. A matrix whose columns are merely perpendicular satisfies no identity of this kind. The terminology is a historical accident that the notation refuses to repeat, which is why the condition is always written

Q^{T}Q = I

rather than described in words.Read more →

What is $\delta_{ij}$ ?

+

The Kronecker delta, a function of two indices returning

1

when they agree and

0

otherwise. It is the identity matrix written entrywise,

I = [\delta_{ij}]

. Do not read it as the

\delta

of calculus: nothing here is small or an increment, and this symbol takes two subscripts rather than measuring a magnitude.Read more →

If $Q^{T}Q = I$ , does $QQ^{T} = I$ as well?

+

Yes, for square

Q

, and this is one of the few places where such a swap is safe. The equation is a statement about the columns being orthonormal, and it forces the rows to be orthonormal too. Most matrix identities are not symmetric in that way, so the fact is worth noting rather than assuming.Read more →

Orthogonal Sets

Bases Where Coordinates Come Free

Orthogonal Sets

Orthogonal Set Notation

Orthogonal Set Notation

Orthogonal Sets Are Independent

Orthonormal Sets

Orthogonal and Orthonormal Bases

Coordinates via Dot Products

Worked Example

Orthogonal Matrices

Matrices with Orthonormal Columns

Parseval&apos;s Identity and Bessel&apos;s Inequality

Summary: Operations in an Orthonormal Basis

Operations in an orthonormal basis

Orthogonal Sets FAQ

Does an "orthogonal matrix" just have orthogonal columns?

What is δij\delta_{ij}δij​?

If QTQ=IQ^{T}Q = IQTQ=I, does QQT=IQQ^{T} = IQQT=I as well?

Parseval's Identity and Bessel's Inequality

What is $\delta_{ij}$ ?

If $Q^{T}Q = I$ , does $QQ^{T} = I$ as well?