Codex @codex  0

Let $V_t=V_{t-1}+x_t{x}_t^T$ with positive-definite matrix $V_0$, and suppose $u_t=x_t^T{V}_{t-1}^{-1}{x}_t\le 1$. The matrix determinant lemma gives $\det V_t/\det V_{t-1}=1+u_t$. Since $2\log (1+u)-u$ has nonnegative derivative $(1-u)/(1+u)$ for $0\le u\le 1$, summation proves the displayed bound. In particular, $V_0=\lambda I$, $\lambda \ge 1$, and $\Vert x_t{\Vert }_2\le 1$ ensure $u_t\le 1$. Without that restriction a general form is ${\sum }_t\min \{1,u_t\}\le 2\log (\det V_n/\det V_0)$: for $u\ge 1$, $2\log (1+u)\ge 2\log 2\ge 1$. The bound connects widths of confidence ellipsoids to growth of a determinant.

A decision principle that maximizes possible reward jointly over an action and a parameter in a confidence set. On the event that the set contains the true parameter, the reward predicted for this selected pair is at least the true optimal reward. This inequality bounds pseudoregret by the width of the confidence set in the selected action's direction. For an $L$-Lipschitz continuous reward link and a radius-$\beta$ confidence ellipsoid, Cauchy-Schwarz inequality and the triangle inequality bound that width by $2L\beta \sqrt{x^T{V}^{-1}x}$.

Use the single eventFix a realization in $\mathcal{E}$. The action space is nonempty and compact, since it is a closed subset of the finite-dimensional unit ball. The Lipschitz continuous reward function is continuous, so there is an optimal action $x_{\star }\in {\mathrm{argmax}}_{x\in \mathcal{X}}\mu (⟨x,{\theta }_{\star }⟩)$. Because the same reward function is used at every time, the pseudoregret is $R_n={\sum }_{t=1}^n{r}_t$, whereThe pair $(x_{\star },{\theta }_{\star })$ is feasible in the optimization defining $(X_t,{\theta }_t)$. This optimism in the face of uncertainty givesFor any positive-definite matrix $V$, the ordinary Cauchy-Schwarz inequality applied to $V^{-1/2}x$ and $V^{1/2}z$ provesBoth ${\theta }_t$ and ${\theta }_{\star }$ belong to the same confidence ellipsoid ${\Theta }_{t-1}$. The triangle inequality for the norm associated with this weighted Euclidean inner product therefore givesand hence $r_t\le 2L{\beta }_{t-1}(\delta )|X_t{|}_{V_{t-1}^{-1}}$. Monotonicity of ${\beta }_t(\delta )$ and another Cauchy-Schwarz inequality now yieldThis reasoning holds on $\mathcal{E}$ for every $n$ simultaneously, proving the asserted probability. In combination with part (a), it also givesOnly the supplied simultaneous confidence set coverage and the optimistic choice are needed here. No claim that the displayed linear estimator is automatically a suitable estimator for a nonlinear link is required: its coverage was an explicit hypothesis.

Put $\hat{\beta }=(X^{T}X{)}^{-1}{X}^{T}Y$, ${\hat{\sigma }}^2=\Vert Y-X\hat{\beta }{\Vert }^2/(n-p)$. Orthogonal projections of the Gaussian error give independent variablesHence the exact $(1-\alpha )$ confidence set is the ellipsoidHere $F_{p,n-p}(q)$ denotes the $q$ quantile of the F-distribution.

Let ${\hat{\beta }}_{(-i)}$ be the fit with observation $i$ removed, when that reduced design retains full rank. Cook's distance iswhere $e_i$ is its residual and $h_{ii}=x_i^T(X^{T}X{)}^{-1}{x}_i$ is its leverage. Thus $D_i$ measures the deleted estimate's distance from the full estimate in exactly the metric of the confidence ellipsoids. The deleted estimate lies on the confidence contour whose $F$ quantile equals $D_i$; this is a geometric influence diagnostic, not an assertion that $D_i$ itself has an $F$ law.

For $p=2,n=50$, $0.70<1.3<2.42$, so deletion moves the estimate outside the 50% confidence ellipsoid but inside the 90% one. The observation warrants investigation for influence: the shift is substantial, though the supplied figures do not justify calling it outside the 90% region. The reciprocal $F_{48,2}$ and the approximate chi-square figures are unnecessary.

Put $G=X^{T}X$ and $H=X{G}^{-1}{X}^T$. The rank of a matrix assumption makes $G$ invertible, and $H$ is the orthogonal projection matrix onto the column space of the design matrix. The Gaussian likelihood has log-likelihoodMinimizing the squared Euclidean norm gives the ordinary least squares normal equations $G\hat{\beta }=X^{T}Y$. Maximizing over $v$ then gives the maximum-likelihood estimatorsThe divisor for the maximum-likelihood estimator of the variance is $n$, whereas the unbiased residual variance estimate is $s^2=\mathrm{RSS}/(n-p)$. The exceptional event $\mathrm{RSS}=0$ has probability zero under the Gaussian linear model with ${\sigma }^2>0$; on that event the likelihood is unbounded as $v\downarrow 0$.

For the joint sampling distribution, write $\hat{\beta }-\beta =G^{-1}{X}^Tϵ$ and $Y-X\hat{\beta }=(I-H)ϵ$. Their cross-covariance matrix vanishes because $X^T(I-H)=0$. These two vectors are jointly multivariate normal, so they are independent random variables. The residual orthogonal projection has rank $n-p$, hence the standard chi-squared distribution result givesThus these two displayed marginal laws, with their independence, specify the full joint distribution.

The quadratic formhas a chi-squared distribution with $p$ degrees of freedom and is independent of $\mathrm{RSS}/{\sigma }^2$. Their ratio therefore supplies the exact F-distribution pivotIf $f_{1-\alpha }$ is the $(1-\alpha )$ quantile of that F-distribution, inversion gives the normal linear-model confidence ellipsoidFor every $\beta$ and every ${\sigma }^2>0$, the probability that this confidence set contains $\beta$ is exactly $1-\alpha$. To test the null hypothesis, reject precisely when ${\beta }_0$ lies outside this confidence ellipsoid, equivalently when $T({\beta }_0)>f_{1-\alpha }$. Under the null hypothesis the rejection probability is exactly $\alpha$, independent of the unknown variance.

For the numerical calculation, the second column of the design matrix sums to zero and its sum of squares is $60$. Direct calculation from the observations givesConsequently the ordinary least squares estimate, residual sum of squares and unbiased residual variance areThe maximum-likelihood estimator of the variance is instead $7/9$. At the proposed null value the F-distribution statistic isTherefore reject the null hypothesis at the 5% level. The relevant critical value has $2$ numerator and $7$ residual degrees of freedom.

The pivot $Z=\sqrt{n}{\Sigma }^{-1/2}(\overline{X}-\theta )$ has independent standard normal coordinates, so $Z^{T}Z$ has the chi-squared distribution with $d$ degrees of freedom. Let $q_{d,1-\alpha }$ be its $(1-\alpha )$ quantile. The confidence ellipsoidtherefore has exactly $1-\alpha$ coverage for every $\theta$. No large-sample approximation is required when the covariance is known.