Solution (source code)

= Solution

First, $\mathcal B\subseteq\mathcal C$ by taking $B=A$. If $A\in\mathcal C$, then
$$
A^c\mathbin\triangle B^c=A\mathbin\triangle B,
$$
so $\mathcal C$ is closed under complements. Now let $A_i\in\mathcal C$ and put $A=\bigcup_iA_i$. Because $\mu(X)<\infty$, <continuity from below of a measure> gives an $N$ such that
$$
\mu\left(A\setminus\bigcup_{i=1}^NA_i\right)<\frac\epsilon2.
$$
Choose $B_i\in\mathcal B$ with $\mu(A_i\triangle B_i)<\epsilon/(2N)$ and let $B=\bigcup_{i=1}^NB_i\in\mathcal B$. The union bound for symmetric differences gives
$$
\mu(A\triangle B)<\epsilon.
$$
Thus $\mathcal C$ is a sigma-algebra containing $\mathcal B$, and
$$
\boxed{\sigma(\mathcal B)\subseteq\mathcal C}.
$$

Finiteness is essential. Take $X=\mathbb N$ with counting measure and let $\mathcal B$ be the finite-cofinite <Boolean algebra of sets>. Then $\sigma(\mathcal B)=\mathcal P(\mathbb N)$. For $\epsilon<1$, the condition $\mu(A\triangle B)<\epsilon$ forces $A=B$, so $\mathcal C=\mathcal B$. Any infinite coinfinite subset of $\mathbb N$ belongs to $\sigma(\mathcal B)$ but not to $\mathcal C$.