Matrix Operations: Rules & Properties

Matrix Addition

Two matrices of the same size can be added entry by entry. If

A

and

B

are both

m \times n

, their sum is the

m \times n

matrix with entries

Matrix Addition

(A + B)_{ij} = a_{ij} + b_{ij}

Matrix addition →

For example,

Matrix Addition

\begin{pmatrix} 1 & 4 \\ -2 & 3 \\ 0 & 5 \end{pmatrix} + \begin{pmatrix} 3 & -1 \\ 6 & 0 \\ 2 & -4 \end{pmatrix} = \begin{pmatrix} 4 & 3 \\ 4 & 3 \\ 2 & 1 \end{pmatrix}

If the dimensions do not match, the sum is undefined — there is no way to add a

2 \times 3

matrix to a

3 \times 2

matrix.

RULE

Addition is commutative (

A + B = B + A

) and associative (

(A + B) + C = A + (B + C)

).

The zero matrix

O

of the same size serves as the additive identity (

A + O = A

), and the additive inverse of

A

is

-A = (-a_{ij})

, so

A + (-A) = O

.

A + B, every entry settled

Each entry of the result came from the one pair sitting in the same position, and from nothing else. That is why the two matrices must have identical shapes and why the sum keeps that shape — there is no mechanism here for a row to reach a different row. Run it entry by entry, or switch to subtraction, on the matrix addition visualizer.

Subtraction and scaling in the next two sections work the same way, one position at a time, which is why they share addition's shape rules.

Matrix Notation

Notation

Matrix Notation

Capital letters, double subscripts in a fixed order, multiplication with no symbol at all — and the two reserved letters every matrix equation leans on.

Bold vectors and their components — vector notation;

A^{T}

in row-times-column form — dot product notation.

A = (a_{ij})

, an

m \times n

matrix

The matrix A with entries a-i-j; m by n

Capital letter for the whole array, lowercase with a double subscript for one entry — row first, column second, always:

a_{23}

sits in row

2

, column

3

. The size

m \times n

reads “

m

by

n

” and obeys the same order: rows, then columns.

CasesSubscripts extract from any matrix expression:

(A + B)_{ij}

names one entry of the sum, the device the definitions on this page are written in. The

\times

in

m \times n

is a dimension separator — a third job for the glyph, after arithmetic and the cross product, and this one multiplies nothing.

Also written

[a_{ij}]

with square brackets — many American texts; this site sets matrices in parentheses. The two bracket styles are pure typography.

Do not confuseReversed subscripts.

a_{ij}

and

a_{ji}

name different entries unless the matrix is symmetric — and the transpose is exactly the operation that swaps them.

AB

A times B — written with no symbol

Matrix multiplication is written by juxtaposition — no dot, no cross. The bare notation hides a shape contract:

(m \times n)(n \times p)

works only because the inner dimensions agree, and delivers

m \times p

.

CasesOrder is load-bearing:

AB \neq BA

in general, so the language splits into premultiplying and postmultiplying. Powers stack the juxtaposition:

A^n

for square

A

, with

A^0 = I

by convention — the identity

I

(subscripted

I_n

when size matters) is the matrix world's

1

, and

O

its

0

, as in Matrix Addition above.

Also written

A \cdot B

with an explicit dot, in some European school texts — never with

\times

, which would collide with dimensions and cross products.

Do not confuseScalar habits. From

AB = AC

nothing cancels;

(A+B)^2

is

A^2 + AB + BA + B^2

, four terms — the notation looks like ordinary algebra and refuses to behave like it.

A^{T}

A transpose

The superscript

T

is a label, not an exponent: flip rows and columns, so

(A^{T})_{ij} = a_{ji}

. The operation itself is The Transpose below.

CasesIt reverses products —

(AB)^{T} = B^{T}A^{T}

, order flipped — and characterizes whole matrix types in one equation:

A = A^{T}

is symmetry,

A^{T} = -A

skew-symmetry,

Q^{T}Q = I

orthogonality.

Also written

A^{\top}

with the sans-serif glyph;

A'

in econometrics and older texts — the prime again, moonlighting far from derivatives.

Do not confuseA power.

A^{T}

multiplies nothing

T

times; the same corner of the symbol carries

A^{n}

(a genuine power),

A^{-1}

(the inverse) and

A^{T}

(a flip) — three superscripts, three unrelated jobs.

All linear algebra symbols →↑ Matrices

Matrix Subtraction

Subtraction is defined as addition of the negative:

Matrix Subtraction

A - B = A + (-B)

Matrix subtraction →

Entry by entry,

(A - B)_{ij} = a_{ij} - b_{ij}

. The same dimension requirement applies — both matrices must have identical shapes. There is nothing deeper here than combining addition and negation, but it appears often enough to warrant its own notation.

Scalar Multiplication

Multiplying a matrix by a scalar

c

scales every entry:

Scalar Multiplication of Matrices

(cA)_{ij} = c \cdot a_{ij}

Scalar multiplication of matrices →

For example,

Scalar Multiplication

-2 \begin{pmatrix} 1 & 3 & -4 \\ 0 & 5 & 2 \end{pmatrix} = \begin{pmatrix} -2 & -6 & 8 \\ 0 & -10 & -4 \end{pmatrix}

RULE

Scalar multiplication distributes over matrix addition (

c(A + B) = cA + cB

), distributes over scalar addition (

(c + d)A = cA + dA

), associates with itself (

c(dA) = (cd)A

), and has

1

as its identity (

1 \cdot A = A

).

Multiplying by

0

produces the zero matrix.

kA, entry by entry

Every entry of the result is the scalar times the entry in the same position: k·a₁₁, k·a₁₂, and so on across all six cells. Nothing moves and nothing mixes — the shape of A is kept exactly. Watch the cells fill one at a time on the matrix scalar multiplication visualizer.

Because the scalar reaches every entry alike, scaling never changes a matrix's shape or the pattern of its zeros.

Linear Combinations of Matrices

Given matrices

A_1, A_2, \dots, A_k

of the same size and scalars

c_1, c_2, \dots, c_k

, the expression

Linear Combinations of Matrices

c_1 A_1 + c_2 A_2 + \cdots + c_k A_k

is a linear combination of matrices.

THEOREM

Addition and scalar multiplication together give the set of all

m \times n

matrices the structure of a vector space.

The dimension of this space is

mn

— one degree of freedom for each entry. The standard basis consists of the

mn

matrices that have a single

1

in one position and zeros everywhere else.

αA + βB, the final sweep

A linear combination of matrices runs in two moves: scale each matrix, then add position by position. This frame is the adding sweep — the already-scaled αA and βB are combined cell by cell, some cells finished, one in progress, the rest still waiting. It is the same pattern as a linear combination of vectors, with a grid in place of a list. Run all three sweeps on the matrix linear combination visualizer.

That is why the m × n matrices behave as a vector space: every linear combination stays inside the same size.

Matrix Multiplication — Definition

For

A

of size

m \times n

and

B

of size

n \times p

, the product

AB

is an

m \times p

matrix whose

(i,j)

entry is the dot product of row

i

of

A

with column

j

of

B

:

Matrix Multiplication

(AB)_{ij} = \sum_{k=1}^{n} a_{ik} b_{kj}

Matrix multiplication →

The number of columns of

A

must equal the number of rows of

B

. If this compatibility condition fails, the product is undefined.

Worked Example

Matrix Multiplication — Definition

\begin{pmatrix} 1 & 0 & 3 \\ 2 & -1 & 4 \end{pmatrix} \begin{pmatrix} 5 & 1 \\ 2 & -3 \\ 0 & 6 \end{pmatrix}

The left matrix is

2 \times 3

and the right is

3 \times 2

, so the product is

2 \times 2

. Computing each entry:

Matrix Multiplication — Definition

(1)(5) + (0)(2) + (3)(0) = 5, \quad (1)(1) + (0)(-3) + (3)(6) = 19

Matrix Multiplication — Definition

(2)(5) + (-1)(2) + (4)(0) = 8, \quad (2)(1) + (-1)(-3) + (4)(6) = 29

Matrix Multiplication — Definition

AB = \begin{pmatrix} 5 & 19 \\ 8 & 29 \end{pmatrix}

Fact: Seven products instead of eight

In 1969 Volker Strassen multiplied two

2 \times 2

matrices with

7

multiplications instead of

8

, at the cost of extra additions. His formulas never use

ab = ba

, so they also work on blocks of a large matrix, and applied again and again they cut a

1024 \times 1024

product from about 1.07 billion multiplications to about 282 million.

Each entry required

n = 3

multiplications and

n - 1 = 2

additions. The full product required

m \times p = 4

such computations.

The same count works for any sizes. Each of the

mp

entries of the product needs

n

multiplications, so multiplying an

m \times n

matrix by an

n \times p

matrix straight from the definition takes

mnp

multiplications in all. For two square

n \times n

matrices that is

n^3

:

8

for

2 \times 2

matrices,

1000

for

10 \times 10

matrices, and a million for

100 \times 100

matrices.

Pitfall: Multiplying entry by entry

Multiplying matching positions turns

\begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}\begin{pmatrix} 5 & 6 \\ 7 & 8 \end{pmatrix}

into

\begin{pmatrix} 5 & 12 \\ 21 & 32 \end{pmatrix}

, the Hadamard product, not

AB

. The row-by-column rule gives

\begin{pmatrix} 19 & 22 \\ 43 & 50 \end{pmatrix}

.

Matrix Multiplication — Properties

Matrix multiplication obeys several familiar algebraic rules and violates one that is deeply ingrained from scalar arithmetic.

Associativity holds whenever all products are defined:

Matrix Multiplication Associativity

(AB)C = A(BC)

Matrix multiplication associativity →

Distribution holds on both sides:

Matrix Multiplication Distributivity

A(B + C) = AB + AC, \quad (A + B)C = AC + BC

Matrix multiplication distributivity →

Scalars pass through freely:

c(AB) = (cA)B = A(cB)

. The identity matrix satisfies

AI = IA = A

whenever the dimensions are compatible.

Commutativity, however, fails:

Matrix Multiplication Non-Commutativity

AB \neq BA \text{ in general}

Matrix multiplication non commutativity →

Even when both products happen to be defined, they generally produce different results. For a concrete counterexample, take

A = \begin{pmatrix} 1 & 2 \\ 0 & 0 \end{pmatrix}

and

B = \begin{pmatrix} 0 & 0 \\ 3 & 4 \end{pmatrix}

. Then

AB = \begin{pmatrix} 6 & 8 \\ 0 & 0 \end{pmatrix}

while

BA = \begin{pmatrix} 0 & 0 \\ 3 & 6 \end{pmatrix}

.

Two further properties distinguish matrix multiplication from scalar multiplication. The product of two nonzero matrices can be zero: if

A = \begin{pmatrix} 1 & 2 \\ 2 & 4 \end{pmatrix}

and

B = \begin{pmatrix} 2 & -4 \\ -1 & 2 \end{pmatrix}

, then

AB = O

even though neither

A

nor

B

is zero. Cancellation also fails:

AB = AC

does not imply

B = C

unless

A

is invertible.

Download

PNGRaster · 2×PDFPrint quality

Matrix · multiplication

Which laws survive the product

Matrix multiplication inherits most of scalar arithmetic and breaks three rules that are deeply ingrained from it. Every entry states its condition; every failure exhibits the pair that breaks it.

08laws

Unconditional5

i

Associativity

(AB)C = A(BC)

Lets you drop parentheses entirely in a chain of products — the reason matrix powers are well…

Unconditional

Requires: all products defined

Lets you drop parentheses entirely in a chain of products — the reason matrix powers are well defined without bracketing.

ii

Left distribution

A(B + C) = AB + AC

Unconditional

Requires: $B, C$ same shape

iii

Right distribution

(A + B)C = AC + BC

Left and right distribution are separate statements and each must be checked, precisely because…

Unconditional

Requires: $A, B$ same shape

Left and right distribution are separate statements and each must be checked, precisely because the product does not commute.

iv

Scalar passage

c(AB) = (cA)B = A(cB)

Unconditional

Requires: any scalar $c$

v

Identity

AI = IA = A

The one place commutativity is guaranteed — I commutes with everything its shape allows.

Unconditional

Requires: conformable dimensions

The one place commutativity is guaranteed — $I$ commutes with everything its shape allows.

Fails or undefined3

vi

Commutativity

AB \neq BA

in general

The most consequential failure in the subject.

Fails

Holds when: both diagonal, or $B = A^{-1}$ , or $B = I$

The most consequential failure in the subject. It is why the transpose reverses order and why similarity rather than equality is the working notion of "same transformation".

WitnessA = [[1, 2], [0, 0]], B = [[0, 0], [3, 4]]
AB = [[6, 8], [0, 0]]
BA = [[0, 0], [3, 6]]

vii

Zero-product law

AB = O \nRightarrow A = O

or

B = O

Matrices are not an integral domain.

Fails

Holds when: either factor invertible

Matrices are not an integral domain. Two nonzero matrices can annihilate each other — which is what a nontrivial null space means at the level of the whole product.

WitnessA = [[1, 2], [2, 4]], B = [[2, −4], [−1, 2]]
AB = O, with neither factor zero

viii

Cancellation

AB = AC \nRightarrow B = C

Fails

Holds when: $A$ invertible
Fails when: $A$ singular

WitnessA = [[1, 0], [0, 0]]
B = [[1, 1], [1, 1]], C = [[1, 1], [2, 2]]
AB = AC = [[1, 1], [0, 0]] but B ≠ C

Unconditional

Fails or undefined

AssociativityUnconditional

(AB)C = A(BC)

Requires: all products defined

Lets you drop parentheses entirely in a chain of products — the reason matrix powers are well defined without bracketing.

Left distributionUnconditional

A(B + C) = AB + AC

Requires: $B, C$ same shape

Right distributionUnconditional

(A + B)C = AC + BC

Requires: $A, B$ same shape

Left and right distribution are separate statements and each must be checked, precisely because the product does not commute.

Scalar passageUnconditional

c(AB) = (cA)B = A(cB)

Requires: any scalar $c$

IdentityUnconditional

AI = IA = A

Requires: conformable dimensions

The one place commutativity is guaranteed —

I

commutes with everything its shape allows.

CommutativityFails

AB \neq BA

in general

Holds when: both diagonal, or $B = A^{-1}$ , or $B = I$

The most consequential failure in the subject. It is why the transpose reverses order and why similarity rather than equality is the working notion of "same transformation".

WitnessA = [[1, 2], [0, 0]], B = [[0, 0], [3, 4]]
AB = [[6, 8], [0, 0]]
BA = [[0, 0], [3, 6]]

Zero-product lawFails

AB = O \nRightarrow A = O

or

B = O

Holds when: either factor invertible

Matrices are not an integral domain. Two nonzero matrices can annihilate each other — which is what a nontrivial null space means at the level of the whole product.

WitnessA = [[1, 2], [2, 4]], B = [[2, −4], [−1, 2]]
AB = O, with neither factor zero

CancellationFails

AB = AC \nRightarrow B = C

Holds when: $A$ invertible
Fails when: $A$ singular

WitnessA = [[1, 0], [0, 0]]
B = [[1, 1], [1, 1]], C = [[1, 1], [2, 2]]
AB = AC = [[1, 1], [0, 0]] but B ≠ C

Three of eight fail. Each failure traces back to the same root — a matrix can collapse directions, and a collapsed direction cannot be recovered. That is what invertibility restores, and why every "holds when" below names it.

Which laws survive the product·/linear-algebra/matrix/operationsLearn Math Class

Matrix Multiplication — Column and Row Interpretations

The entry-by-entry formula is the most common way to define matrix multiplication, but two alternative viewpoints often provide sharper insight.

The column interpretation says that column

j

of

AB

is obtained by multiplying

A

times column

j

of

B

:

Matrix Multiplication — Column and Row Interpretations

AB = \begin{pmatrix} A\mathbf{b}_1 & A\mathbf{b}_2 & \cdots & A\mathbf{b}_p \end{pmatrix}

Each column of the product is a linear combination of the columns of

A

, with weights given by the corresponding column of

B

. This is the view that connects matrix multiplication to linear transformations: the product

AB

applies the transformation

A

to each column of

B

independently.

The row interpretation says that row

i

of

AB

equals row

i

of

A

times the entire matrix

B

. Each row of the product is a linear combination of the rows of

B

, weighted by the entries in the corresponding row of

A

.

A third perspective writes the product as a sum of rank-one outer products:

Matrix Multiplication — Column and Row Interpretations

AB = \sum_{k=1}^{n} (\text{column } k \text{ of } A)(\text{row } k \text{ of } B)

Each term is an

m \times p

matrix of rank at most one, and their sum is the full product. This decomposition appears in low-rank approximation theory and in the analysis of the singular value decomposition.

Download

PNGRaster · 2×PDFPrint quality

Matrix · multiplication

Four ways to read the same product

Every row computes $AB$ and every row gives the same answer. What differs is the unit being assembled — a scalar, a column, a row, or a whole rank-one matrix — and each unit makes a different fact obvious.

4readings

One number at a time1

Entry view§ 5the definition

assembled froma single scalar

(AB)_{ij}

formula

(AB)_{ij} = \sum_k a_{ik}b_{kj}

natural wherehand computation, proofs

Row

i

of

A

dotted with column

j

of

B

. Correct and unilluminating — it says how to get a number out but not what the product is, which is why the three readings below exist.

One vector at a time2

Column view§ 7read

B

column by column

assembled froma column of

AB

formulacolumn

j

of

AB = A\mathbf{b}_j

natural wherecolumn space, transformations

Each column of the product is

A

applied to the corresponding column of

B

— so every column of

AB

is a linear combination of the columns of

A

. This is why

\operatorname{Col}(AB) \subseteq \operatorname{Col}(A)

, and hence why the rank of a product is bounded.

Row view§ 7read

A

row by row

assembled froma row of

AB

formularow

i

of

AB = \mathbf{a}_i B

natural whererow space, elimination

The mirror statement: every row of

AB

is a combination of the rows of

B

. This is the reading that explains why row operations — which are left multiplications — preserve the row space.

One matrix at a time1

Outer product sum§ 7pair column

k

of

A

with row

k

of

B

assembled froma rank-one matrix

formula

AB = \sum_k \mathbf{a}_k \mathbf{b}_k^{\mathsf{T}}

natural whereSVD, low-rank approximation

The product as a sum of rank-one pieces. Truncating that sum is exactly what low-rank approximation does, which makes this the reading that matters most in practice and the one taught last.

The definition is the entry view, but it is the one that explains least. Each of the other three is the same sum regrouped, and each is the natural reading somewhere else in the subject — the column view for linear combinations, the outer product for low-rank approximation.

Four ways to read the same product·/linear-algebra/matrix/operationsLearn Math Class

The two vector readings are worth holding together, because between them they explain the rank bound. Every column of

AB

is a combination of the columns of

A

, so the column space of the product sits inside that of

A

; every row is a combination of the rows of

B

, so the row space sits inside that of

B

. Rank is the dimension of either, so

\operatorname{rank}(AB)

cannot exceed the smaller of the two — which is the inequality stated without proof on the rank page.

The product assembled from the rows

Every piece drawn here is a row of the right-hand factor scaled by an entry of the left-hand row, laid tail to head until the row of the product is reached. Read this way the product never mentions a single entry: one whole row is consumed per step, which is what the row interpretation above claims and what makes the count of steps equal to the number of rows rather than to the number of entries. Follow one row at a time on the row picture visualizer.

The outer-product form below is the third reading of the same product, assembling it from whole columns and rows at once rather than one at a time.

The Transpose

The transpose of an

m \times n

matrix

A

is the

n \times m

matrix

A^T

obtained by converting rows into columns:

Transpose Definition

(A^T)_{ij} = a_{ji}

Transpose definition →

For example,

The Transpose

A = \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{pmatrix} \quad \Longrightarrow \quad A^T = \begin{pmatrix} 1 & 4 \\ 2 & 5 \\ 3 & 6 \end{pmatrix}

Applying the transpose twice recovers the original matrix:

Transpose Involution

(A^T)^T = A

Transpose involution →

The transpose distributes over addition:

Transpose of Sum

(A + B)^T = A^T + B^T

Transpose of sum →

and commutes with scalar multiplication:

Transpose of Scalar Multiple

(cA)^T = c\, A^T

Transpose of scalar multiple →

The product rule reverses the order:

Transpose of Product

(AB)^T = B^T A^T

Transpose of product →

This reversal is a frequent source of errors and is worth memorizing as a pattern: transposing a product is like reading it backward.

A matrix satisfying

A = A^T

is called symmetric.

COROLLARY

For any matrix

A

of any shape, the products

A^T A

and

AA^T

are both symmetric — this is immediate from the product rule, since

(A^T A)^T = A^T (A^T)^T = A^T A

.

Reflection across the main diagonal

Every entry has swapped places with its mirror image across the diagonal, and the entries sitting on the diagonal have not moved at all. Read that way the transpose is one reflection rather than a rule about indices, and it explains at a glance why transposing twice returns the original. Watch it run cell by cell instead on the transpose visualizer.

Because the diagonal is fixed, a matrix equal to its own transpose is exactly one that is symmetric about that line.

Matrix Powers

For a square matrix

A

, powers are defined by repeated multiplication:

Matrix Power

A^0 = I, \quad A^1 = A, \quad A^k = \underbrace{A \cdot A \cdots A}_{k \text{ factors}}

Matrix power →

RULE

The usual exponent laws hold:

A^j A^k = A^{j+k}

and

(A^j)^k = A^{jk}

.

When

A

is invertible, negative powers are defined as

A^{-k} = (A^{-1})^k

, extending the exponent laws to all integers.

One rule from scalar arithmetic does not carry over. Since matrix multiplication is not commutative, the identity

(AB)^k = A^k B^k

is false in general. Expanding

(AB)^2 = ABAB

, there is no way to rearrange this into

A^2 B^2 = AABB

without commutativity.

Pitfall: Squaring a sum as with numbers

Writing

(A + B)^2 = A^2 + 2AB + B^2

merges

AB

and

BA

. With

A

and

B

from the commutativity counterexample, it gives

\begin{pmatrix} 13 & 18 \\ 12 & 16 \end{pmatrix}

, but

(A + B)^2 = A^2 + AB + BA + B^2 = \begin{pmatrix} 7 & 10 \\ 15 & 22 \end{pmatrix}

.

Elementary Matrices

An elementary matrix is the result of performing a single row operation on the identity matrix. There are three types, corresponding to the three row operations: swapping two rows, multiplying a row by a nonzero scalar, and adding a multiple of one row to another.

The key property is that left-multiplying a matrix

A

by an elementary matrix

E

performs the corresponding row operation on

A

. If

E

swaps rows

2

and

3

of the identity, then

EA

swaps rows

2

and

3

of

A

. If

E

scales row

1

of the identity by

5

, then

EA

scales row

1

of

A

by

5

.

THEOREM

Every elementary matrix is invertible, and its inverse is another elementary matrix of the same type: the inverse of a row swap is the same row swap, the inverse of scaling by

k

is scaling by

1/k

, and the inverse of adding

c

times row

i

to row

j

is subtracting

c

times row

i

from row

j

.

Pitfall: Multiplying on the wrong side

Writing

AE

instead of

EA

for the row swap

E = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}

and

A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}

gives

\begin{pmatrix} 2 & 1 \\ 4 & 3 \end{pmatrix}

, with the columns swapped. The row swap is

EA = \begin{pmatrix} 3 & 4 \\ 1 & 2 \end{pmatrix}

.

Matrix Decompositions

A matrix decomposition (or factorization) expresses a matrix as a product of simpler matrices with known structure. Decompositions are among the most powerful tools in computational linear algebra, converting hard problems into sequences of easy ones.

The LU decomposition writes

A = LU

where

L

is lower triangular and

U

is upper triangular. It captures the essence of Gaussian elimination in matrix form and makes solving linear systems with multiple right-hand sides efficient: once

L

and

U

are known, each system reduces to two triangular solves.

The QR decomposition writes

A = QR

where

Q

is orthogonal and

R

is upper triangular. It is the foundation of least-squares computation and several eigenvalue algorithms.

The Cholesky decomposition writes

A = LL^T

for symmetric positive definite matrices, achieving the work of LU in roughly half the computation by exploiting symmetry.

The eigendecomposition writes

A = PDP^{-1}

where

D

is diagonal, placing the eigenvalues on the diagonal and the eigenvectors in the columns of

P

. It applies only to diagonalizable matrices.

The singular value decomposition writes

A = U\Sigma V^T

where

U

and

V

are orthogonal and

\Sigma

is diagonal with nonnegative entries. Unlike the eigendecomposition, the SVD exists for every matrix of every shape. It reveals the rank, the fundamental subspaces, and the best low-rank approximation to

A

, making it one of the most broadly applicable tools in the subject.

Each of these decompositions has its own page with full derivations and worked examples.

Decomposition	Form	Requires	Primary use
LU	A = LU (L lower, U upper triangular)	any square matrix (with pivoting)	solving Ax = b for many right-hand sides
QR	A = QR (Q orthogonal, R upper triangular)	any matrix	least squares, eigenvalue algorithms
Cholesky	A = LLᵀ	symmetric positive definite	half-cost LU for SPD systems
Eigendecomposition	A = PDP⁻¹ (D diagonal)	diagonalizable square matrix	matrix powers, spectral analysis
SVD	A = UΣVᵀ (U, V orthogonal; Σ diagonal ≥ 0)	any matrix of any shape	rank, fundamental subspaces, best low-rank approximation

Two More Products: Hadamard and Outer

Two products sit outside the main family but appear often enough to deserve their own rules.

The Hadamard product

A \circ B

, also written

A \odot B

, multiplies entry by entry:

(A \circ B)_{ij} = a_{ij}\,b_{ij}

. Both matrices must share the same

m \times n

shape, and the result has that shape too. Because positions never mix, it inherits the arithmetic of ordinary numbers: it is commutative, associative, and distributes over addition. It has nothing to do with

AB

. For a

2 \times 2

matrix,

A \circ I

keeps only the diagonal of

A

, while

AI = A

.

The outer product

\mathbf{u}\mathbf{v}^T

is ordinary matrix multiplication applied to a column with

m

entries and a row with

n

entries. The inner dimensions are both

1

, so the product exists for any two lengths and gives an

m \times n

matrix with entries

u_i v_j

:

Two More Products: Hadamard and Outer

\mathbf{u}\mathbf{v}^T = \begin{pmatrix} 1 \\ 2 \end{pmatrix} \begin{pmatrix} 3 & 4 & 5 \end{pmatrix} = \begin{pmatrix} 3 & 4 & 5 \\ 6 & 8 & 10 \end{pmatrix}

Every column is a multiple of

\mathbf{u}

and every row a multiple of

\mathbf{v}^T

, so the result has rank one whenever both vectors are non-zero. The dot product is the same two vectors multiplied in the other order,

\mathbf{v}^T\mathbf{u}

, which collapses to a single number; the outer product spreads them into a table instead.

Both products have their own visual tools: the Hadamard product and the outer product.

Dimension Summary

Fact: Same product, ten times the work

For

A

of size

10 \times 100

,

B

of size

100 \times 5

and

C

of size

5 \times 50

, the route

(AB)C

passes through the small

10 \times 5

matrix

AB

and takes 5,000 + 2,500 = 7,500 multiplications. The route

A(BC)

passes through the

100 \times 50

matrix

BC

and takes 25,000 + 50,000 = 75,000.

The dimension requirements for each operation are worth collecting in one place. Addition and subtraction require both matrices to share the same

m \times n

dimensions, and the result is

m \times n

. Scalar multiplication imposes no restriction and preserves the original dimensions. The product

AB

requires the column count of

A

to match the row count of

B

— if

A

is

m \times n

and

B

is

n \times p

, the result is

m \times p

. The transpose of an

m \times n

matrix is

n \times m

. Powers

A^k

require

A

to be square and produce a matrix of the same size.

A common source of confusion is the product rule. The product of two

n \times n

matrices is

n \times n

, but the product of a

2 \times 3

matrix with a

3 \times 5

matrix is

2 \times 5

. The "inner" dimensions must match and are consumed; the "outer" dimensions survive into the result.

Operation	Input requirement	Result size	Key property
Addition / subtraction	A and B both m × n	m × n	commutative; additive identity O
Scalar multiplication	A is m × n; c any scalar	m × n	distributes over both matrix and scalar sum
Linear combination	all Aᵢ same m × n shape	m × n	turns m × n matrices into a vector space of dimension mn
Matrix multiplication	A is m × n, B is n × p (inner dims match)	m × p	associative, NOT commutative; inner dim consumed
Transpose	A is m × n (no other restriction)	n × m	(Aᵀ)ᵀ = A; (AB)ᵀ = BᵀAᵀ (order reverses)
Power Aᵏ	A square (n × n), k ≥ 0 (or k ∈ ℤ if invertible)	n × n	Aʲ Aᵏ = Aʲ⁺ᵏ; (AB)ᵏ ≠ Aᵏ Bᵏ in general

Common Pitfalls

Four mistakes account for many wrong answers in matrix arithmetic. Three are marked with a box where they happen in the sections above, and the fourth, the order of a transposed product, is flagged in the transpose section. Here each is worked through in full, with the wrong and the right working side by side.

Multiplying entry by entry

The product

AB

is computed by multiplying the entries that sit in the same position, the way matrices are added. Rows and columns are never combined.

Wrong working:

\begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}\begin{pmatrix} 5 & 6 \\ 7 & 8 \end{pmatrix} = \begin{pmatrix} 1 \cdot 5 & 2 \cdot 6 \\ 3 \cdot 7 & 4 \cdot 8 \end{pmatrix} = \begin{pmatrix} 5 & 12 \\ 21 & 32 \end{pmatrix}

Right working: each entry is a row of the left matrix times a column of the right matrix.

\begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}\begin{pmatrix} 5 & 6 \\ 7 & 8 \end{pmatrix} = \begin{pmatrix} 1 \cdot 5 + 2 \cdot 7 & 1 \cdot 6 + 2 \cdot 8 \\ 3 \cdot 5 + 4 \cdot 7 & 3 \cdot 6 + 4 \cdot 8 \end{pmatrix} = \begin{pmatrix} 19 & 22 \\ 43 & 50 \end{pmatrix}

Why it happens: addition, subtraction and scalar multiplication really do work position by position, and the entry-by-entry product exists as an operation of its own, the Hadamard product. The pattern is carried over to

AB

.

How to check: multiply by the identity matrix, which must return the matrix unchanged. With the row-times-column rule,

AI = A

. Entry by entry,

\begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}

and

I

give

\begin{pmatrix} 1 & 0 \\ 0 & 4 \end{pmatrix}

, only the diagonal.

Keeping the order when transposing a product

The transpose is applied to each factor of a product, but the factors are left in their original order:

(AB)^T

is written as

A^T B^T

.

Wrong working: take the matrices of the worked example,

A = \begin{pmatrix} 1 & 0 & 3 \\ 2 & -1 & 4 \end{pmatrix}

and

B = \begin{pmatrix} 5 & 1 \\ 2 & -3 \\ 0 & 6 \end{pmatrix}

. Then

A^T

is

3 \times 2

,

B^T

is

2 \times 3

, and

A^T B^T = \begin{pmatrix} 7 & -4 & 12 \\ -1 & 3 & -6 \\ 19 & -6 & 24 \end{pmatrix}

a

3 \times 3

matrix, although

AB

is

2 \times 2

.

Right working: the order reverses.

(AB)^T = B^T A^T = \begin{pmatrix} 5 & 8 \\ 19 & 29 \end{pmatrix}

which is exactly

AB = \begin{pmatrix} 5 & 19 \\ 8 & 29 \end{pmatrix}

with its rows and columns exchanged.

Why it happens: the transpose distributes over a sum,

(A + B)^T = A^T + B^T

, with no change of order, and the same pattern is carried over to products.

How to check: compare sizes. If

A

is

m \times n

and

B

is

n \times p

, then

(AB)^T

is

p \times m

. The product

B^T A^T

has that size, while

A^T B^T

is defined only when

m = p

, and even then it is

n \times n

.

Squaring a sum as with numbers

(A + B)^2

is expanded with the formula from ordinary algebra, and the two middle terms

AB

and

BA

are merged into

2AB

.

Wrong working: with

A = \begin{pmatrix} 1 & 2 \\ 0 & 0 \end{pmatrix}

and

B = \begin{pmatrix} 0 & 0 \\ 3 & 4 \end{pmatrix}

from the commutativity counterexample,

A^2 + 2AB + B^2 = \begin{pmatrix} 1 & 2 \\ 0 & 0 \end{pmatrix} + \begin{pmatrix} 12 & 16 \\ 0 & 0 \end{pmatrix} + \begin{pmatrix} 0 & 0 \\ 12 & 16 \end{pmatrix} = \begin{pmatrix} 13 & 18 \\ 12 & 16 \end{pmatrix}

Right working: distribute without reordering, so both

AB

and

BA

appear.

(A + B)^2 = A^2 + AB + BA + B^2

= \begin{pmatrix} 1 & 2 \\ 0 & 0 \end{pmatrix} + \begin{pmatrix} 6 & 8 \\ 0 & 0 \end{pmatrix} + \begin{pmatrix} 0 & 0 \\ 3 & 6 \end{pmatrix} + \begin{pmatrix} 0 & 0 \\ 12 & 16 \end{pmatrix} = \begin{pmatrix} 7 & 10 \\ 15 & 22 \end{pmatrix}

Why it happens: the formula

(a + b)^2 = a^2 + 2ab + b^2

is so familiar that the step

ba = ab

hidden inside it goes unnoticed.

How to check: add first, then square.

A + B = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}

, and its square is

\begin{pmatrix} 7 & 10 \\ 15 & 22 \end{pmatrix}

, the same as the four-term expansion. The two-term middle

2AB

is correct only when

AB = BA

.

Multiplying on the wrong side

To perform a row operation, the elementary matrix is placed to the right of

A

. Multiplying on the right acts on the columns, so the operation lands on the columns instead of the rows.

Wrong working: with

E = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}

, the identity with its two rows swapped, and

A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}

,

AE = \begin{pmatrix} 2 & 1 \\ 4 & 3 \end{pmatrix}

The two columns of

A

have traded places; the rows have not.

Right working:

EA = \begin{pmatrix} 3 & 4 \\ 1 & 2 \end{pmatrix}

which is

A

with rows

1

and

2

swapped.

Why it happens: the operation is done to

A

, so

A

is written first and

E

is attached after it. For a row swap the slip is hard to see, because the same

E

is also the identity with two columns swapped.

How to check: every row of a row-swapped matrix must be a row of

A

. The first row of

AE

is

(2, 1)

, which is not a row of

A

; the rows of

EA

are

(3, 4)

and

(1, 2)

.

Matrix Operations FAQ

Does $a_{23}$ mean row 2 column 3, or row 3 column 2?

+

Row first, column second, always:

a_{23}

sits in row

2

, column

3

. The size

m \times n

follows the same order, rows then columns. Reversing the pair names a different entry, since

a_{ij}

and

a_{ji}

agree only when the matrix is symmetric, and swapping them is precisely what the transpose does.Read more →

Is $(A + B)^2$ equal to $A^2 + 2AB + B^2$ ?

+

No. Expanding gives four terms,

A^2 + AB + BA + B^2

, and they cannot be merged because

AB

and

BA

are generally different matrices. The same warning applies to cancellation: from

AB = AC

nothing follows about

B

and

C

. The notation borrows the look of ordinary algebra without the behaviour.Read more →

Why is matrix multiplication written with no symbol at all?

+

Juxtaposition is the convention:

AB

, with no dot and no cross. The bare notation hides a shape contract, since an

m \times n

times an

n \times p

works only because the inner dimensions agree, producing

m \times p

. Note that the

\times

in

m \times n

is a dimension separator and multiplies nothing.Read more →

Operations on Matrices

Manipulating Matrices

Matrix Addition

Matrix Notation

Matrix Notation

Matrix Subtraction

Scalar Multiplication

Linear Combinations of Matrices

Matrix Multiplication — Definition

Worked Example

Matrix Multiplication — Properties

Which laws survive the product

Matrix Multiplication — Column and Row Interpretations

Four ways to read the same product

The Transpose

Matrix Powers

Elementary Matrices

Matrix Decompositions

Two More Products: Hadamard and Outer

Dimension Summary

Common Pitfalls

Matrix Operations FAQ

Does $a_{23}$ mean row 2 column 3, or row 3 column 2?

Is $(A + B)^2$ equal to $A^2 + 2AB + B^2$ ?

Why is matrix multiplication written with no symbol at all?

Operations on Matrices

Manipulating Matrices

Matrix Addition

Matrix Notation

Matrix Notation

Matrix Subtraction

Scalar Multiplication

Linear Combinations of Matrices

Matrix Multiplication — Definition

Worked Example

Matrix Multiplication — Properties

Which laws survive the product

Matrix Multiplication — Column and Row Interpretations

Four ways to read the same product

The Transpose

Matrix Powers

Elementary Matrices

Matrix Decompositions

Two More Products: Hadamard and Outer

Dimension Summary

Common Pitfalls

Matrix Operations FAQ

Does a23a_{23}a23​ mean row 2 column 3, or row 3 column 2?

Is (A+B)2(A + B)^2(A+B)2 equal to A2+2AB+B2A^2 + 2AB + B^2A2+2AB+B2?

Why is matrix multiplication written with no symbol at all?

Does $a_{23}$ mean row 2 column 3, or row 3 column 2?

Is $(A + B)^2$ equal to $A^2 + 2AB + B^2$ ?