Past exam of the options I course of the University of Oxford / 2013 / O (Statistics)

Put $S={\sum }_{j=1}^n|X_j|$. The log-likelihood for the Laplace distribution isFor $S>0$ the derivative is positive for $\theta <S/n$ and negative for $\theta >S/n$, so the unique maximum-likelihood estimator isThe exceptional data set $S=0$ has probability zero; there the likelihood has its supremum at the excluded boundary $\theta =0$. For the Fisher information, $|X_j|$ has exponential density ${\theta }^{-1}{e}^{-y/\theta }$ for $y>0$, hence mean $\theta$. Since ${\ell }^{\prime \prime }(\theta )=n/{\theta }^2-2S/{\theta }^3$,This also follows from the score variance, because $\mathrm{Var}(S)=n{\theta }^2$.

The absolute observations are independent exponential random variables with mean $\theta$ and variance ${\theta }^2$. Applying the central limit theorem directly to their average provesThus the large-sample approximation is $\hat{\theta }\approx N(\theta ,{\theta }^2/n)$, consistent with inverse Fisher information. Also $\hat{\theta }\to \theta$ in probability by the weak law of large numbers, so $\theta$ in the standard error can be replaced by $\hat{\theta }$ through Slutsky's theorem.

Let $z=z_{1-\alpha /2}={\Phi }^{-1}(1-\alpha /2)$, where $0<\alpha <1$. By the preceding asymptotic normality and Slutsky's theorem, $\sqrt{n}(\hat{\theta }-\theta )/\hat{\theta }$ is approximately standard normal. Inverting $|\sqrt{n}(\hat{\theta }-\theta )/\hat{\theta }|\le z$ gives the Wald confidence intervalIts coverage approaches $1-\alpha$ as $n\to \infty$. The interpretation of the confidence interval is repeated-sample coverage of the fixed parameter; it does not assign a posterior probability to $\theta$.

Let $s={\sum }_{j=1}^n{x}_j$. The Bernoulli distribution likelihood is proportional to ${\theta }^s(1-\theta {)}^{n-s}$. Multiplication by the beta distribution prior givesNormalizing with the beta function therefore givesThe parameters remain positive. This is Beta-binomial conjugacy: successes add to the first shape parameter and failures to the second.

A $100(1-\alpha )\%$ credible set is a measurable subset $C(x)\subseteq (0,1)$ with posterior probability $1-\alpha$:In ordinary language, given the observed data and the chosen prior model, the probability that the parameter lies in this set is $1-\alpha$. Such a set need not be an interval or be unique; its defining requirement is posterior mass.

Let $F_{A,B}$ denote the distribution function of the posterior beta distribution, with $A=a+s$ and $B=b+n-s$. This distribution is continuous with positive density on $(0,1)$. Its equal-tailed credible interval isCompute the two quantiles by solving ${\int }_0^{{\theta }_L}\pi (\theta \mid x)d\theta =\alpha /2$ and ${\int }_{{\theta }_U}^1\pi (\theta \mid x)d\theta =\alpha /2$. Each excluded tail has mass $\alpha /2$, leaving posterior mass $1-\alpha$ between the endpoints.

The further observation is independent of the earlier observations conditional on $\theta$. Integrate its conditional success probability against the posterior:The given factorial formula for the beta function gives this ratio as $A/(A+B)$. Thus the posterior predictive probability isIt is the posterior mean of the Bernoulli success parameter. Independence here is conditional: sharing the uncertain parameter generally makes past and future observations dependent before conditioning on it.