June 30, 2026
This is a continuation of my previous notes on measure theory and probability. Here, we will prove that, given a probability measure space \((\Omega, \mathcal{F}, P)\), there exists a unique extension of \(P\) to \(\sigma(F)\) (i.e, a probability measure on \(\sigma(F)\) ).
The theorem we will prove is: "A probability measure on a field has a unique extension to the generated \(\sigma\) -field". Specifically: Suppose \(P\) is a probability measure on a field \(F_0\) of subsets of \(\Omega\). Put \(F = \sigma(F_0)\). Then there exists a probability measure \(Q\) on \(F\) such that \(Q(A) = P(A)\) for all \(A \in F_0\). Further, if \(Q'\) is another probability measure on \(F\) such that \(Q'(A) = P(A)\) for all \(A \in F_0\), then \(Q(A) = Q'(A)\) for all \(A \in F\).
For every subset of \(A\):
\[ \begin{align*} P^*(A) = \inf \sum_n P(A_n) \end{align*} \] where the infimum is performed over all finite and infinite collections \(\{A_n\}_n\) such that \(A \subset \cup_n A_n\). We say that a set \(A\) is \(P^*\) -measurable if:
\[ \begin{align*} P^*(A \cap E) + P^*(A^c \cap E) = P^*(E) \end{align*} \]
\(P^*\) has the following properties, which we subsequently prove:
We can first show (2). \(\sum_n P(A_n) \ge 0\) since \(P(A_n) \ge 0\) for every \(A_n \in \mathcal{F}_0\) and where the \(\{A_n\}_n\) form a cover of \(A\). So, \(P^*(A) = \inf \sum_n P(A_n) \ge 0\). Next we show (1). Using (2), \(0 \le P^*(\emptyset) \le P(\emptyset) = 0\) since \(\emptyset \in \mathcal{F}_0\). So, \(P^*(\emptyset) = 0\). We now show \(3\). Since \(A \subset B\), then every cover of \(B\) is also a cover of \(A\). That is, the set of all covers of \(A\) includes the set of all covers of \(B\). Then, \(\inf \sum_n P(A_n) \le \inf \sum_n P(B_n)\), which implies that \(P^*(A) \le P^*(B)\). Let's show \(4\). We need to show that \(P^*(\cup_n A_n) \le \sum_n P^*(A_n)\) where \(A_n \cap A_m = \emptyset\) for \(n \ne m\). Let \(\epsilon > 0\) be arbitrary. Then, there exists a cover \(\{A_{n, m} \}_m\) of \(A_n\) such that:
\begin{align*} \sum_m P(A_{n,m}) &\le P^*(A_n) + \frac{\epsilon}{2^n} \\ \sum_n \sum_m P(A_{n,m}) &\le \sum_n P^*(A_n) + \epsilon & \text{ sum over $n$ } \\ P^*(\cup_n A_n) &\le \sum_n \sum_m P(A_{n,m}) \le \sum_n P^*(A_n) + \epsilon & \text{ $\cup_n A_n \subset \cup_n \cup_m A_{n,m}$ } \end{align*}We will prove a few lemmas that will be important for proving the first part of the theorem, which is that there exists an extension of the probability measure \(P\) to \(\sigma(F_0)\). We define the class \(M\) to be the collection of sets that are \(P^*\) measurable. \(A\) is \(P^*\) measurable if: \[ P^*(A \cap E) + P^*(A^c \cap E) = P^*(E) \text{ for all $E \subset \Omega$ } \]
The class \(M\) is a field.
We will need to prove 3 things:
First we prove (1): \[ P^*(\Omega \cap E) + P^*(\Omega^c \cap E) = P^*(E) + P^*(\emptyset) = P^*(E) \] Next we prove (2): Let \(A \in M\). Then we can write:
\begin{align*} P^*(E) &= P^*(A \cap E) + P^*(A^c \cap E) \\ P^*(E) &= P^*(A^c \cap E) + P^*((A^c)^c \cap E) \end{align*}The last line implies that \(A^c \in M\). Now we prove (3):
\begin{align*} P^*(E) &= P^*(A \cap E) + P^*(A^c \cap E) \\ &= P^*(A \cap B \cap E) + P^*(A \cap B^c \cap E) + P^*(A^c \cap B \cap E) + P^*(A^c \cap B^c \cap E) & \text{ $B$ is $P^*$ -measurable (see below) }\\ &\ge P^*(\{(A \cap B) \cup (A \setminus B) \cup (B \setminus A)\} \cap E) + P^*((A \cup B)^c \cap E) & \text{ countable subadditivity } \\ &= P^*((A \cup B) \cap E) + P^*((A \cup B)^c \cap E) \end{align*}The second line follows from the fact that \(B\) is \(P^*\) -measurable, so \(P^*(A \cap E) = P^*(A \cap E \cap B) + P^*(A \cap E \cap B^c)\) because \(A \cap E \subset \Omega\).
Let \(\{A_n\}_n\) be a disjoint sequence (finite or infinite) sequence of \(M\) -sets. For each \(E \subset \Omega\), \(P^*(E \cap \Bigl(\cup_n A_n\Bigl)) = \sum_n P^*(E \cap A_n)\)
The approach we take is to first prove the lemma for finitely many \(A_n\). Then, we extend it to infinite sequences. First, let \(n = 2\). Suppose \(A_1 \cup A_2 = \Omega\). So, \(A_1 = A_2^c\). So:
\begin{align*} P^*(E \cap A_1) + P^*(E \cap A_2) = P^*(E \cap A_2^c) + P^*(E \cap A_2) = P^*(E) = P^*(E \cap (A_1 \cup A_2)) \end{align*}Now, suppose \(A_1 \cup A_2 \ne \Omega\). Then, \(A_1 \in M\) and \(A_2 \in M\), so \(A_1 \cup A_2 \in M\) (by Lemma \(3.1\)). Hence:
\begin{align*} P^*(A_1 \cap E) + P^*(A_1^c \cap E) &= P^*(E) \\ P^*(A_1 \cap A_2 \cap E) + P^*(A_1 \cap A_2^c \cap E) + P^*(A_1^c \cap A_2 \cap E) + P^*(A_1^c \cap A_2^c \cap E) &= P^*(E) \\ P^*(\emptyset) + P^*(A_1 \cap E) + P^*(A_2 \cap E) + P^*(A_1^c \cap A_2^c \cap E) &= P^*(E) & A_1 \cap A_2 = \emptyset \implies A_1 \cap A_2^c = A_1, A_1^c \cap A_2 = A_2 \\ P^*(A_1 \cap E) + P^*(A_2 \cap E) &= P^*(E) - P^*(E \cap (A_1 \cup A_2)^c) \\ &\le P^*(E \cap (A_1 \cup A_2)) & \text{ countable subadditivity } \end{align*}Since it holds for \(n = 2\), we can extend it for all \(n \ge 2\), since \(\cup_{k=1}^n A_k = \Bigl(\cup_{k=1}^{n-1} A_k\Bigl) \cup A_{n}\). Then, we can take \(n \rightarrow \infty\) to obtain the result for infinite sequences.
The class \(M\) is a \(\sigma\) -field and \(P^*\) restricted to \(M\) is countably additive
Let \(\{A_n\}_{n=1}^\infty \in M\), \(A_n \cap A_m = \emptyset\) for all \(n \ne m\). We want to show \(P^*(\cup_{n=1}^\infty) A_n = \sum_{n=1}^\infty P^*(A_n)\). By Lemma 2:
\begin{align*} P^*(\cup_{n=1}^\infty A_n) &= P^*(\Omega \cap \bigcup_{n=1}^\infty A_n) \\ &= \sum_{n=1}^\infty P^*(\Omega \cap A_n) \\ &= \sum_{n=1}^\infty P^*(A_n) \end{align*}Therefore, \(P^*\) is countably additive on \(M\). We show that if \(A_n \in M\) for all \(n \in \mathbb{N}\), then \(\cup_{n=1}^\infty A_n \in M\). We want to show that: \[P^*(E \cap \bigcup_{n=1}^\infty A_n) + P^*(E \cap \bigcap_{n=1}^\infty A_n^c) \le P^*(E)\]. First, we show that this is true for countable disjoint unions. Let \(A_m \cap A_n = \emptyset\) for all \(m \ne n\). Let \(F_n = \cup_{k=1}^n A_k\). Then:
\begin{align*} P^*(E \cap F_n) + P^*(E \cap F_n^c) &= P^*(E) & \text{$F_n \in M$ since $M$ closed under finite unions} \\ \sum_{k=1}^n P^*(E \cap A_k) + P^*(E \cap F_n^c) &\le P^*(E) & \text{Lemma 2} \end{align*}Now, \(F_n \subset \cup_{n=1}^\infty A_n\), which implies that \(\cap_{n=1}^\infty A_n^c \subset F_n^c\). So, \(P^*(E \cap \bigcap_{k=1}^\infty A_k^c) \le P^*(E \cap F_n^c)\) by monotonicity. So:
\begin{align*} \sum_{k=1}^n P^*(E \cap A_k) + P^*(E \cap \bigcap_{k=1}^\infty A_k^c) &\le P^*(E) \\ \lim_{n \rightarrow \infty} \Bigl[ \sum_{k=1}^n P^*(E \cap A_k) + P^*(E \cap \bigcap_{k=1}^\infty A_k^c) \Bigl] &\le \lim_{n \rightarrow \infty} P^*(E) \\ \sum_{k=1}^\infty P^*(E \cap A_k) + P^*(E \cap \bigcap_{k=1}^\infty A_k^c) &\le P^*(E) \\ P^*(E \cap \bigcup_{n=1}^\infty A_n) + P^*(E \cap \bigcap_{k=1}^\infty A_k^c) &\le P^*(E) \end{align*}This proves the theorem for countable disjoint unions. To show it for arbitrary unions, we can define \(B_1 = A_1\) and \(B_k = A_k \setminus \bigcup_{j=1}^{k-1} A_j\). Then, \(\cup_{k=1}^\infty B_k = \cup_{k=1}^\infty A_k\) and the \(\{B_k\}_{k=1}^\infty\) are disjoint. So, we can apply the preceding result of countable disjoint unions to the \(B_k\)'s and obtain the same result.
\(F_0 \subset M\)
Let \(A \in F_0\). We want to show that \(P^*(A \cap E) + P^*(A^c \cap E) \le P^*(E) \forall E \subset \Omega\). Let \(\epsilon > 0\). Then, there exists a cover of \(F_0\) -sets \(\{A_n\}_n\) such that \(\sum_n P(A_n) \le P^*(E) + \epsilon\). Since \(A_n \in F_0\), \(A \cap A_n \in F_0\). Also, \(E \subset \cup_n A_n\), which means \(A \cap E \subset \cup_n (A \cap A_n)\) and \(A^c \cap E \subset \cup_n (A^c \cap A_n)\). So:
\begin{align*} P^*(A \cap E) + P^*(A^c \cap E) &\le \sum_n P(A \cap A_n) + \sum_n P(A^c \cap A_n) \\ &= \sum_n P(A_n) \le P^*(E) + \epsilon & \text{ finite additivity } \end{align*}\(P^*(A) = P(A) \forall A \in F_0\)
We know from the definition of \(P^*\) that \(P^*(A) \le P(A)\). We show that \(P^*(A) \ge P(A)\). Let \(\{A_n\}_n\) be a cover of \(A\). Then, \(A_n \in F_0\) such that \(\sum_n P(A_n) \le P^*(A) + \epsilon\). Since \(A \subset \cup_n A_n\), \(A \subset \cup_n (A \cap A_n)\), and \(A \cap A_n \subset A_n\), \(\sum_n P(A \cap A_n) \le \sum_n P(A_n)\) by monotonicity. By countable subadditivity, \(P(A) = P(A \cap \cup_n A_n) \le \sum_n P(A \cap A_n) \le \sum_n P(A_n) \le P^*(A) + \epsilon\).
There exists a probability measure \(Q\) on \(F = \sigma(F_0)\) such that \(Q(A) = P(A)\) for all \(A \in F_0\).
We take \(Q = P^*\) and show it is a probability measure on \(F\) first. We know by Lemma 3 that \(M\) is a \(\sigma\) -field. By Lemma 4, \(F_0 \subset M\), so \(F = \sigma(F_0) \subset M\) by minimality of \(\sigma(F_0)\). We also know by Lemma 3 that \(Q\) restricted to \(M\) (and, therefore, to \(F\)) is countably additive. Combined with the basic properties of \(P^*\) we prove at the start of these notes (non-negativity) and the fact that \(P^*(\Omega) = P(\Omega) = 1\), this shows that \(Q\) is a probability measure on \(F\). Lemma 5 shows that \(Q(A) = P^*(A) = P(A)\) for all \(A \in F_0\), which is the second part of this theorem.
To prove uniqueness, we need some extra definitions:
A \(\pi\) -system is any class of sets \(\mathcal{P}\) where \(A, B \in \mathcal{P}\) implies \(A \cap B \in \mathcal{P}\).
A \(\lambda\) -system is any class of sets \(\mathcal{L}\) which satisfies the following criteria:
There is a small modification we can do to the definition above that will be useful for proving later results:
Given (1) and (3) hold in \(\lambda\) -system definition, condition (2) is equivalent to the following (which we'll denote (2')): If \(A, B \in \mathcal{L}\) and \(B \subset A\), \(A \setminus B \in \mathcal{L}\).
First, we show that \((2') \implies (2)\). Since \(\Omega \in \mathcal{L}\) in definition of \(\lambda\) -system, if \(A \in \mathcal{L}\), then \(A \subset \Omega\) and \(A^c = \Omega \setminus A \in \mathcal{L}\). Now, we show that \((2) \implies (2')\). Suppose \(A, B \in \mathcal{L}\) and \(B \subset A\):
\begin{align*} A \setminus B = A \cap B^c = (A^c \cup B)^c \end{align*}Note that \(B \cap A^c = \emptyset\) since \(B \cap A = A\). Then, \(A^c \cup B\) is a disjoint union, so by (3) in definition of \(\lambda\) -system, \(A^c \cup B \in \mathcal{L}\). By (2), \(A \setminus B = (A^c \cup B)^c \in \mathcal{L}\).
The next lemma relates \(\lambda\) -systems and \(\pi\) -systems to \(\sigma\) -fields:
If a class \(\mathcal{F}\) is a \(\lambda\) -system and a \(\pi\) -system, then it is a \(\sigma\) -field.
We have to show the following properties:
(1) and (2) follow from definition of \(\lambda\) -system. Also, (3) follows from definition of \(\pi\) -system. For (4), we can write \(A_1 = B_1\) and \(A_k = B_k \setminus \cup_{j=1}^{k-1} B_{j}\), and so by Turning Countable Unions to Disjoint Unions we see that the \(\{A_k\}_k\) are disjoint and \(\cup_{k=1}^\infty B_k = \cup_{k=1}^\infty A_k \in \mathcal{F}\).
If \(\mathcal{P}\) is a \(\pi\) -system and \(\mathcal{L}\) is a \(\lambda\) -system and \(\mathcal{P} \subset \mathcal{L}\), then \(\sigma(\mathcal{P}) \subset \mathcal{L}\).
The proof for this is a little intricate, so I'll lay out the strategy for showing it. Let \(M\) be the minimal \(\lambda\) -system containing \(\mathcal{P}\). Since \(\mathcal{P} \subset \mathcal{L}\) AND \(\mathcal{L}\) is a \(\lambda\) -system, \(\mathcal{M} \subset \mathcal{L}\). If we can show that \(M\) is also a \(\pi\) -system, then Lemma 8 shows that \(M\) is a \(\sigma\) -field. By minimality of \(\sigma(\mathcal{P})\), because \(\mathcal{P} \subset M \subset \mathcal{L}\), \(\sigma(\mathcal{P}) \subset M \subset \mathcal{L}\). We lay out the steps to showing that \(M\) is a \(\pi\) -system:
We now fill in the details for steps 1,2,3. First, we show that \(L_A\) is a \(\lambda\) -system. We show the 3 properties in definition of \(\lambda\) -system. First, \(A \cap \Omega \in A\), so \(\Omega \in L_A\). To show \(L_A\) is closed under complementation, we can apply Lemma 7 and prove the equivalent statement that if \(B_1, B_2 \in L_A\) and \(B_1 \subset B_2\), then \(B_2 \setminus B_1 \in L_A\). We already know that \(A \cap B_1 \in M\) and \(A \cap B_2 \in M\) and that \(A \cap B_1 \subset A \cap B_2\), so applying Lemma 7 to \(M\), \(A \cap (B_2 \setminus B_1) = (A \cap B_2) \setminus (A \cap B_1) \in M\). Let \(\{B_n\}_n\) be a collection of disjoint sets such that \(B_n \in L_A\). Then, \(A \cap B_m \in M\). So:
\begin{align*} A \cap \cup_{n=1}^\infty B_n &= \cup_{n=1}^\infty \Bigl(A \cap B_n \Bigl) \\ \end{align*}Note that \(B_n \cap B_m = \emptyset\) for all \(n \ne m\). Then, \(A \cap B_n\) and \(A \cap B_m\) are disjoint as well. So, by property 3 of definition of \(\lambda\) -system, \(\cup_{n=1}^\infty \Bigl(A \cap B_n\Bigl) \in M\).
Now, we show that step 2. If \(A, B \in \mathcal{P}\), then \(A \cap B \in \mathcal{P} \subset M\). Then, \(B \in L_A\). Hence, \(B \in \mathcal{P} \implies B \in L_A\). So, \(\mathcal{P} \subset L_A\). Since \(M\) is the minimal \(\lambda\) -system containing \(\mathcal{P}\), it follows that \(M \subset L_A\).
Step 3: If \(A \in \mathcal{P}\), then \(M \subset L_A\). That is, if \(A \in P\) and \(B \in M\), then \(B \in L_A\). This also implies \(A \in L_B\) because \(B \cap A = A \cap B \in M\). In other words, \(A \in \mathcal{P}\) and \(B \in M\) implies \(A \in L_B\). So, \(\mathcal{P} \subset L_B\). By minimality of \(M\), \(M \subset L_B\). So, if \(C \in M\), then \(C \in L_B\), which implies \(B \cap C \in M\). By definition of \(\pi\) -system, \(M\) is a \(\pi\) -system.
Now, we can prove uniqueness with the following theorem:
Let \(P_1\), \(P_2\) be 2 probability measures on \(\sigma(\mathcal{P})\) where \(\mathcal{P}\) is a \(\pi\) -system. Suppose \(P_1\) and \(P_2\) agree on \(\mathcal{P}\). Then, they agree on \(\sigma(\mathcal{P})\).
The strategy we use is to construct a \(\lambda\) -system that \(P_1\) and \(P_2\) agree on and then apply Theorem 9 to show that \(P_1\) and \(P_2\) agree on \(\sigma(\mathcal{P})\). In this case, we show that \(L = \{B \in \sigma(\mathcal{P}): P_1(B) = P_2(B)\}\) is a \(\lambda\) -system:
Let \(\{B_n\}_n\) be a collection of \(L\) -sets that are disjoint. Then, \(\cup_n B_n \in \sigma(\mathcal{P})\), so we can compute \(P_1(\cup_n B_n)\):
\begin{align*} P_1(\cup_n B_n) &= \sum_n P_1(B_n) & \text{ countable additivity } \\ &= \sum_n P_2(B_n) & \text{ $P_1(B_n) = P_2(B_n)$ } \\ &= P_2(\cup_n B_n) & \text{ countable additivity } \end{align*}So, \(\cup_n B_n \in L\).
Since every field is a \(\pi\) -system, this theorem applies for \(F_0\) and \(F = \sigma(F_0)\) since \(F_0\) is a field.
A class \(\mathcal{M}\) of subsets of \(\Omega\) is called monotone if it is closed under the formation of monotone unions and intersections:
Before we continue, there is a fact that is mentioned in the proof of the monotone class theorem (which is similar to the \(\pi\) - \(\lambda\) theorem) that we will prove here:
If \(\mathcal{F}\) is a monotone field, it is a \(\sigma\) -field.
\(\mathcal{F}\) is a monotone field. Then:
We see that (1), (2), and (3) are also properties of a \(\sigma\) -field. Now suppose that \(\{B_n\}_{n=1}^\infty\) is a sequence of sets where \(B_n \in \mathcal{F}\) for all \(n \in \mathbb{N}\). We want to show that \(\cup_{n=1}^\infty B_n \in \mathcal{F}\). Let \(A_n \in \cup_{k=1}^n B_k\) for all \(n \in \mathbb{N}\). Then, \(A_n \subset A_{n+1}\). Let \(B = \cup_{n=1}^\infty B_n\). Also, \(\cup_{n=1}^\infty A_n = \cup_{n=1}^\infty \cup_{k=1}^n B_k = \cup_{k=1}^\infty \cup_{n = k}^\infty B_k = \cup_{k=1}^\infty B_k\). So, \(B \in \mathcal{F}\) because \(A_n \uparrow A\) and \(A_n \in \mathcal{F}\) for all \(n\) by combining (2) and (3).
We have the following monotone class theorem:
If \(\mathcal{F}_0\) is a field and \(\mathcal{M}\) is a monotone class, then \(\mathcal{F}_0 \subset \mathcal{M}\) implies \(\sigma(\mathcal{F}_0) \subset \mathcal{M}\).
This proof is similar to the proof of the \(\pi\) - \(\lambda\) theorem Let \(m(\mathcal{F}_0)\) be the minimal monotone class containing \(\mathcal{F}_0\). Then, \(m(\mathcal{F}_0) \subset \mathcal{M}\). If we show that \(m(\mathcal{F}_0)\) is a field, then by Lemma 11 it is a \(\sigma\) -field. By the minimality of \(\sigma\) -fields, because \(\mathcal{F}_0 \subset m(\mathcal{F}_0) \subset \mathcal{M}\), that means that \(\sigma(\mathcal{F}_0) \subset m(\mathcal{F}_0) \subset \mathcal{M}\). So, we show now that \(m(\mathcal{F}_0)\) is a field. Like before, we will construct sets (like \(L_A\)) to obtain the requisite properties:
If \(A, B \in m(\mathcal{F}_0)\), \(A \cup B \in m(\mathcal{F}_0)\): Let:
\begin{align*} G_1 &= \{A: A \cup B \in m(\mathcal{F}_0) \forall B \in \mathcal{F}_0\} \\ G_2 &= \{B: A \cup B \in m(\mathcal{F}_0) \forall A \in m(\mathcal{F}_0) \} \end{align*}First, we show \(G_1\) and \(G_2\) are monotone.
Since \(\mathcal{F}_0 \subset G_1\), \(m(\mathcal{F}_0) \subset G_1\) by minimality (\(G_1\) is monotone). If \(B \in \mathcal{F}_0\) and \(A \in m(\mathcal{F}_0)\), then because \(m(\mathcal{F}_0) \subset G_1\), \(A \in G_1\). And, \(B \in \mathcal{F}_0 \subset m(\mathcal{F}_0) \subset G_1\). So, \(A \cup B \in m(\mathcal{F}_0)\). That is, \(B \in G_2\). Equivalently, \(\mathcal{F}_0 \subset G_2\). By minimality of \(m(\mathcal{F}_0)\) and the fact \(G_2\) is monotone, \(m(\mathcal{F}_0) \subset G_2\). In other words, if \(A, B \in m(\mathcal{F}_0)\), then \(A \cup B \in m(\mathcal{F}_0)\).
We say a probability space \((\Omega, \mathcal{F}, P)\) is complete if \(A \subset B\), \(B \in \mathcal{F}\), and \(P(B) = 0\) imply that \(A \in \mathcal{F}\).
The following statement is made in the book, but it's not immediately obvious (to me, at least):
Suppose \((\Omega, \mathcal{F}, P)\) is complete. Let \(A \in \mathcal{F}\), \(A \Delta A' \subset B \in \mathcal{F}\), and \(P(B) = 0\). Then, \(A' \in \mathcal{F}\) and \(P(A') = P(A)\).
We show first that \(A' \in \mathcal{F}\). By completeness, \(A \Delta A' \in \mathcal{F}\). By monotonicity and non-negativity of \(P\), it follows that \(P(A \Delta A') = 0\). Since \(A \in \mathcal{F}\) and \(A \Delta A' \in \mathcal{F}\), their difference, \(\Bigl(A \Delta A'\Bigl) \setminus A = A' \setminus A \in \mathcal{F}\). Using this fact, we can also deduce that \(\Bigl(A \Delta A'\Bigl) \setminus \Bigl(A' \setminus A\Bigl) = A \setminus A' \in \mathcal{F}\). Once more, we can use these facts to obtain that \(A \setminus (A \setminus A') = A \cap A' \in \mathcal{F}\). Then, we get that \(A' = (A' \cap A) \cup (A' \cap A^c)\), which is a finite union of two \(\mathcal{F}\) -sets, implying that \(A' \in \mathcal{F}\). Further, \(A = (A \cap A') \cup (A \cap (A')^c)\). Note that both \(A'\) and \(A\) are written as the union of two disjoint \(\mathcal{F}\) -sets, so by finite additivity of \(P\):
\begin{align*} P(A') &= P(A' \cap A) + P(A' \cap A^c) = P(A' \cap A) \\ P(A) &= P(A \cap A') + P(A \cap (A')^c) = P(A \cap A') = P(A' \cap A) \\ &\implies P(A) &= P(A') \end{align*}P(A' ∩ Ac) = 0 and \(P(A \cap (A')^c) = 0\) because they are two disjoint sets whose union is \((A' \cap A^c) \cup (A \cap (A')^c) = (A' \setminus A) \cup (A \setminus A') = A \Delta A'\), and \(P(A \Delta A') = 0\).
The important point about complete sets here is that for any probability space, \((\Omega, \mathcal{F}, P)\), we can enlarge the \(\sigma\) -field and extend the probability measure \(P\) to get a complete probability space, as we now show: consider the extension \(P^*\) and take \(\mathcal{F}_0 = \mathcal{F} = \sigma(\mathcal{F}_0)\). Let \(\mathcal{M}\) be the \(\sigma\) -field of \(P^*\) -measurable sets. By Lemma 3, \(P^*\) restricted to \(\mathcal{M}\) is a probability measure. We now show this space is complete. Let \(A \subset B\), \(B \in \mathcal{M}\), and \(P^*(B) = 0\). We prove that \(A \in \mathcal{M}\). Let \(E \subset \Omega\). Then:
\begin{align*} P^*(A \cap E) + P^*(A^c \cap E) \le P^*(B) + P^*(E) = P^*(E) \end{align*}where we applied monotonicity to both terms (\(A \cap E \subset B\) and \(A^c \cap E \subset E\)).
For the probability space \(((0, 1], \mathcal{B}, \lambda)\), the sets \(A\) in the completed \(\sigma\) -field \(\mathcal{M}\) are called Lebesgue sets, and \(\lambda\) is still called the Lebesgue measure.
We consider the Vitali construction, which produces a set in \((0, 1]\) that is not a Borel set (i.e, a set not in \(\mathcal{B}\)). Define, for \(x, y \in (0, 1]\): \[ x \oplus y = \begin{cases} x + y & 0 < x + y \le 1 \\ x + y - 1 & x + y > 1 \end{cases} \]
Let \(\mathcal{L}\) be the class of Borel sets \(A\) such that \(A \bigoplus x \in \mathcal{B}\) and \(\lambda(A \bigoplus x) = \lambda(A)\) for some \(x \in (0, 1]\). We show that \(\mathcal{L}\) is a \(\lambda\) -system:
Suppose \(A \in \mathcal{L}\). We show that \(A^c \in \mathcal{L}\). Since \(A \in \mathcal{B}\), \(A^c \in \mathcal{B}\) since it is a \(\sigma\) -algebra. Note that \((A \oplus x)^c = A^c \oplus x\). Since \(A \oplus x \in \mathcal{B}\), \((A \oplus x)^c \in \mathcal{B}\). Putting this together, \(A^c \oplus x \in \mathcal{B}\). Next, \(\lambda(A) = \lambda(A \oplus x)\). Since \(\Omega = (0, 1]\) and \(\lambda(A^c) = \lambda(\Omega \setminus A) = 1 - \lambda(A)\), we get that:
\begin{align*} \lambda(A^c) &= 1 - \lambda(A) \\ &= 1 - \lambda(A \oplus x) \\ &= \lambda((A \oplus x)^c) \\ &= \lambda(A^c \oplus x) \end{align*}Therefore, \(A^c \in \mathcal{L}\).
Let \(A_n \in \mathcal{L}\) and \(A_n \cap A_m = \emptyset\) for all \(m \ne n\). Then, \(A_n \in \mathcal{B}\) and \(\cup_n A_n \in \mathcal{B}\) since \(\mathcal{B}\) is a \(\sigma\) -algebra. Because, \(A \oplus x \in \mathcal{B}\), it follows that \(\cup_n (A_n \oplus x) = (\cup_n A_n) \oplus x \in \mathcal{B}\). Next:
\begin{align*} \lambda((\cup_n A_n) \oplus x) &= \lambda(\cup_n (A_n \oplus x)) \\ &= \sum_n \lambda(A_n \oplus x) & \text{ countable additivity } \\ &= \sum_n \lambda(A_n) & \text{ translation-invariance } \\ &= \sum_n \lambda(A_n) \\ &= \lambda(\cup_n A_n) & \text{ countable additivity } \end{align*}Since \(\mathcal{L}\) is a \(\lambda\) -system containing \(\mathcal{I}\), the class of intervals in \((0, 1]\), and \(\mathcal{I}\) is a field (hence a \(\pi\) -system), it follows by the \(\pi-\lambda\) theorem that \(\mathcal{B} = \sigma(I) \subset \mathcal{L}\). Define an equivalence relation \(\sim\) on \((0, 1]\) so that \(x \sim y\) if there exists \(r \in (0, 1]\) such that \(x \oplus r = y\). Let \(H\) be a subset of \((0, 1]\) consist of exactly one representative from each equivalence class (this set exists because of the axiom of choice). Since \(\mathbb{Q}\) is countable, consider the sets \(H \oplus r\) for rational \(r\). First, we show these sets are disjoint: Suppose \(H \oplus r_1\) and \(H \oplus r_2\) have a comment element \(h_r\). Then, there exists \(h_1 \in H \oplus r_1\) such that \(h_r = h_1 \oplus r_1\) and \(h_2 \in H \oplus r_2\) such that \(h_r = h_2 \oplus r_2\). Hence:
\begin{align*} h_1 \oplus r_1 &= h_2 \oplus r_2 \end{align*}If this is the case, then:
\begin{align*} h_1 + r_1 &= h_2 + r_2 & \text{ or } \\ h_1 + r_1 &= h_2 + r_2 - 1 & \text{ or } \\ h_1 + r_1 - 1 &= h_2 + r_2 & \text{ or } \\ h_1 + r_1 - 1 &= h_2 + r_2 - 1 \\ \end{align*}Either way, this would imply that \(h_1 \sim h_2\). But, \(H\) contains exactly one representative from each equivalence class, so we cannot have \(h_1 \ne h_2\), which implies that \(h_1 = h_2\). But this would imply that \(r_1 = r_2\). Hence, the sets are actually the same. Therefore, two distinct sets \(H \oplus r_1\) and \(H \oplus r_2\) are disjoint. Therefore, we can write \((0, 1] = \cup_{r \in \mathbb{Q}} H \oplus r\) because for every \(y \in (0, 1]\), then there exists some \(r \in \mathbb{Q}\) so that \(h \oplus r = y\) for some \(h \in H\) since \(H\) contains one member from each equivalence class. Now, suppose that \(H\) were in \(\mathcal{B}\). Then, \(\lambda((0, 1]) = \sum_r \lambda(H \oplus r)\) since the \(H \oplus r\) are disjoint and the union is countable because \(\mathcal{Q}\) is countable. However, we know that \(H \in \mathcal{B}\) implies \(H \in \mathcal{L}\), which means that \(\lambda(H) = \lambda(H \oplus r)\). So, the sum \(\sum_r \lambda(H) = \lambda((0, 1]) = 1\). Since \(\lambda(H) \ge 0\) and the sum is over a countably infinite quantity, either \(\lambda(H) = 0\) (which means \(1 = 0\)) or \(\lambda(H) > 0\) in which case we have a divergent sum (and so it cannot equal \(1\)). This means \(H\) is not a Borel set.