= Solution
Let $\mathcal G_t=\sigma(X_1,\ldots,X_t)$. Under $P_{\theta_\star}$, the true <probability density function> is positive and finite at every observation almost surely. Define the sequential <likelihood ratio>
$$
M_0=1,\qquad
M_t=\prod_{s=1}^t\frac{p_{\widehat\theta_{s-1}}(X_s)}{p_{\theta_\star}(X_s)}.
$$
A zero numerator sets the product to zero thereafter. Thus $M_t$ is a well-defined nonnegative process, and $L_t(\theta_\star)=\log M_t$, with $\log0=-\infty$.
The next observation is <independent> of $\mathcal G_{s-1}$, while $\widehat\theta_{s-1}$ is <measurable> with respect to that <sigma-algebra>. Its <conditional expectation> is therefore
$$
\begin{aligned}
\mathbb E_{\theta_\star}\left[\left.
\frac{p_{\widehat\theta_{s-1}}(X_s)}{p_{\theta_\star}(X_s)}\right|\mathcal G_{s-1}\right]
&=\int_{\{p_{\theta_\star}>0\}}
p_{\widehat\theta_{s-1}}(x)\,d\mu(x)\\
&\leq1.
\end{aligned}
$$
Consequently $\mathbb E(M_s\mid\mathcal G_{s-1})\leq M_{s-1}$; induction also gives $\mathbb EM_s\leq1$. Thus $(M_t)$ is a nonnegative <supermartingale> for the <filtration> $(\mathcal G_t)$. It need not be a <martingale>: an alternative distribution can put mass outside the support of the true distribution.
The <maximal inequality for a nonnegative supermartingale>, the corresponding form of the <Ville inequality>, gives
$$
\begin{aligned}
\mathbb P_{\theta_\star}\{\exists t\geq0:\theta_\star\notin\Theta_t\}
&=\mathbb P_{\theta_\star}\{\exists t\geq0:L_t(\theta_\star)\geq\log(1/\delta)\}\\
&\leq\mathbb P_{\theta_\star}\{\sup_{t\geq0}M_t\geq1/\delta\}\\
&\leq\boxed{\delta\mathbb EM_0=\delta}.
\end{aligned}
$$
The strict inequality in the definition of the <confidence set> correctly becomes a non-strict crossing event.
There is a minor definitional omission in the source: for false candidates $\theta$, the printed sum of <log-likelihood ratios> can involve $0/0$ or both infinite signs. To define the entire <confidence set> without a positivity assumption, use the logarithm of the ratio of the two full products, $\prod_s p_{\widehat\theta_{s-1}}(X_s)$ and $\prod_s p_\theta(X_s)$, taking $0/a=0$ for $a>0$, $a/0=+\infty$ for $a>0$, and, for example, $0/0=1$. This is a measurable extension of the printed formula wherever that formula is defined. The choice at $0/0$ cannot affect coverage of the truth, since its full denominator is positive almost surely. Alternatively, strict positivity of every density on the common support removes the omission.
Back to article page
