Rules of Probability

Welcome to the Rules of Probability.

Grounded in the three axioms of probability—non-negativity, normalization and finite additivity—this section translates those foundational principles into practical tools. You’ll begin by revisiting the basic axiomatic properties before moving on to set-operation rules (complements, differences and absorption), additive rules (addition, inclusion–exclusion and Boole’s inequality) and multiplicative rules (chain rule, law of total probability and Bayes’ theorem). Each law here is built on the axioms to ensure consistency and rigor. In upcoming chapters, you’ll see how these rules power classical (combinatorial) models, discrete and continuous distributions, conditional probability and independence, Bayesian inference, expectation and variance, limit theorems and stochastic processes. Keep this axiomatic framework in mind as your roadmap through the broader probability theory landscape.

Basic Axiomatic Properties

	rule	formula	explanation
	Non-negativity & Bounds	$0 \le P(A) \le 1$	The probability of any event lies between 0 and 1, inclusive.
	Empty-Set Rule	$P(\varnothing) = 0$	The probability of the empty event (impossible outcome) is zero.

Set-Operation Rules

rule	formula	explanation
Complement Rule	$P(A^c) = 1 - P(A)$	The probability of the complement of A equals one minus the probability of A.
Difference Rule	$P(A \setminus B) = P(A) - P(A \cap B)$	The probability of A excluding B equals the probability of A minus the probability of A and B.
Subset Absorption	If $B \subseteq A$ , then $P(A \cap B) = P(B)$ and $P(A \cup B) = P(A)$	If B is contained in A, the intersection has B’s probability and the union has A’s.
Complement Absorption	If $A \subseteq B^c$ , then $P(A \cap B^c) = P(A)$ and $P(A \cup B^c) = P(B^c)$	When A lies entirely outside B, intersection yields A and union yields the complement of B.
Mutual Exclusivity	If $A \cap B = \varnothing$ , then $P(A \cap B) = 0$ and $P(A \cup B) = P(A) + P(B)$	Disjoint events have zero probability of occurring together, and their union is the sum of probabilities.

Additive & Inequality Rules

rule	formula	explanation
Addition Rule	$P(A \cup B) = P(A) + P(B) - P(A \cap B)$	The probability of A or B equals the sum minus the overlap.
Inclusion–Exclusion Principle	$P\bigl(\bigcup_{i=1}^n A_i\bigr) = \sum_i P(A_i) - \sum_{i<j}P(A_i \cap A_j) + \cdots + (-1)^{n-1}P\bigl(\bigcap_{i=1}^n A_i\bigr)$	General formula for the probability of a union of n events, correcting for over-counted overlaps.
Monotonicity & Boole’s Inequality	If $A \subseteq B$ , then $P(A) \le P(B)$ ; and $P\bigl(\bigcup_i A_i\bigr) \le \sum_i P(A_i)$	Probabilities respect subset ordering, and the probability of any union is at most the sum of individual probabilities.

Multiplicative & Conditional Rules

rule	formula	explanation
Multiplication & Chain Rules	$P(A \cap B) = P(B)\,P(A \mid B)$ ; $P\bigl(\bigcap_{i=1}^n A_i\bigr) = \prod_{i=1}^n P\bigl(A_i \mid A_1 \cap \cdots \cap A_{i-1}\bigr)$	Compute joint probabilities via conditional probabilities, extended through the chain rule.
Law of Total Probability	$P(A) = \sum_i P(A \mid B_i)\,P(B_i)$	Expresses P(A) as a weighted sum over a partition of the sample space.
Bayes’ Theorem	$P(B_j \mid A) = \dfrac{P(A \mid B_j)\,P(B_j)}{\sum_i P(A \mid B_i)\,P(B_i)}$	Relates the reverse conditional probability to the forward conditional probability and priors.