Codex @codex  0

For a Bernoulli distribution with fixed $0<p<1$, the mean and variance are $p$ and $p(1-p)$. The odds transformation $g(x)=x/(1-x)$ has derivative $g^{\prime }(x)=(1-x{)}^{-2}$. Applying the delta method from part (ii),More precisely, $\sqrt{n}\{\overline{X}/(1-\overline{X})-p/(1-p)\}$ converges to a normal distribution of mean zero and variance $p/(1-p{)}^3$, after defining the statistic arbitrarily on $\overline{X}=1$. That exceptional event has probability $p^n\to 0$, so this extension does not change the limit. The unextended ratio is undefined there and has no finite exact variance if interpreted as infinite. At $p=0$ it is zero almost surely; at $p=1$ the denominator is always zero. This is the delta method for Bernoulli odds.

Write $S^2=Q/(n-1)$ and $S=\sqrt{S^2}$, with $n\ge 2$. Two exact facts about a normal sample areand independence of the normal sample mean and sample variance holds. A chi-squared distribution with $n-1$ statistical degrees of freedom is a sum of $n-1$ independent ${\chi }_1^2$ variables, each with mean one and variance two. The central limit theorem therefore givesFor $g(x)=\sqrt{x}$, the derivative at ${\mu }^2$ is $1/(2\mu )$ because $\mu >0$. The delta method gives the asymptotic normality of a normal sample standard deviation:The exact independence of $\overline{X}$ and $S$ makes their limiting normal distributions independent. Consequently,ThusThis has the same leading asymptotic variance as $\hat{\mu }$. The weights $1/3$ and $2/3$ are those in the optimal asymptotic combination of a normal sample mean and standard deviation. This does not require $S$ to be an exactly unbiased estimator of $\mu$.

Put $A={\sum }_i{x}_i$, $B={\sum }_i{x}_i^2$ and $M_2=B/n$. The normal distribution model givesand, up to a constant independent of $\mu$, its log-likelihood function isThe score function and its zero equation areFor $B>0$, this quadratic equation has exactly one positive root, giving the maximum-likelihood estimatorThe numerator $B-A\mu -n{\mu }^2$ of the score function is positive between its negative and positive roots and negative above its positive root. Thus the likelihood function increases up to $\hat{\mu }$ and decreases afterwards: this is its unique global maximum on $(0,\infty )$. Also, $\ell (\mu )\to -\infty$ at both ends. The all-zero observation vector has probability zero; on that exceptional vector $L(\mu )\propto {\mu }^{-n}$ and there is no attained maximum.

The second derivative and the relevant moments areTherefore the expected Fisher information isThe support of a probability distribution is the whole real line, independent of the positive statistical parameter. The law of large numbers gives $\overline{X}\to \mu$ and $M_2\to 2{\mu }^2$ in probability, so the displayed maximum-likelihood estimator is a consistent estimator. The regular asymptotic normality of a maximum likelihood estimator now yieldsThus the requested leading asymptotic variance on the unscaled estimator's scale is ${\mu }^2/(3n)$. The normal mean proportional to standard deviation model provides a reusable version of this calculation.

Put $M_0=\alpha +x_0\beta$. As a linear combination of independent normal random variables under the posterior distribution,The shortest 95% credible interval isHere $\Phi$ is the standard normal distribution cumulative distribution function. To justify shortest length, consider all intervals of a fixed length $d$. If $f$ is this normal distribution posterior density, the interval probability ${\int }_a^{a+d}f(t)dt$ has derivative $f(a+d)-f(a)$ with respect to $a$. The normal distribution is symmetric and its density strictly decreases with distance from its mean, so this probability is uniquely maximized when the midpoint is $m_0$. The centered interval can therefore attain 95% posterior probability with the least length. It is both an equal-tailed credible interval and a highest posterior density interval.

The target is the experiment's mean at $x_0$. A posterior predictive distribution for a new noisy observation would instead have variance $v_0+{\sigma }^2$; that extra observation variance does not belong in the requested interval.

Let $S_{xx}={\sum }_i{x}_i^2$, $S_{xy}={\sum }_i{x}_i{y}_i$, and $\bar{y}=n^{-1}{\sum }_i{y}_i$. The independent observations in the normal linear model giveThe centered design condition ${\sum }_i{x}_i=0$ removes the $\alpha \beta$ term:The stated prior density factors into independent normal distributions for the regression intercept $\alpha$ and regression coefficient $\beta$. Multiplication by the likelihood function therefore leaves separate quadratic expressions in the two coefficients. Define the positive posterior variancesand the posterior meansBy completing the square, the joint posterior density isEquivalently,This is Gaussian conjugacy for a normal linear model. Each posterior precision parameter is the sum of its data and prior precision parameters. The formulas remain valid when $S_{xx}=0$: all $x_i$ are then zero, so the posterior distribution of $\beta$ equals its prior distribution.

An unknown variance requires a joint prior distribution for $(\alpha ,\beta ,{\sigma }^2)$. The analysis is not determined until a prior for the unknown variance, and its relation to the coefficients, is specified. A maximum-likelihood estimate substituted for ${\sigma }^2$ does not by itself account for its posterior uncertainty.

For example, retain the independent coefficient prior distributions from part (b) and independently assign $s={\sigma }^2$ an inverse-gamma distribution with shape $a_0>0$ and scale $b_0>0$, whose probability density function is proportional to $s^{-a_0-1}{e}^{-b_0/s}$. Then the joint posterior density is proportional toConditional on $s$, the two coefficient posterior distributions are still the independent normal distributions in part (b), with $s$ substituted for ${\sigma }^2$. The other full conditional distribution isThe marginal coefficient posterior distribution requires marginalization over $s$; it is generally nonnormal and the two coefficients need not remain independent. In particular, the posterior cumulative distribution function of the requested mean becomesIts shortest 95% credible interval must be found from this marginal posterior distribution, for example by numerical integration or posterior simulation. This is normal regression with independent coefficient and variance priors.

A different, convenient choice is a normal-inverse-gamma prior, in which the conditional coefficient prior covariance matrix is proportional to ${\sigma }^2$. That choice gives a multivariate Student's t-distribution after marginalization, so the interval uses its univariate Student's t-distribution quantiles. It changes the coefficient prior specification; a Student interval does not follow automatically from retaining the fixed-${\tau }_0,{\tau }_1$ priors in part (b).

Sort the observations into order statistics $x_{(1)}\le \cdots \le x_{(n)}$. Choose plotting positions $p_i=(i-1/2)/n$ and calculate the standard normal quantiles $z_i={\Phi }^{-1}(p_i)$. The normal Q-Q plot consists of the pairs $(z_i,x_{(i)})$.

If the normal distribution model is appropriate, the points should lie approximately on a straight line with positive slope, because its population quantile function isThe intercept is approximately $\mu$ and the slope approximately $\sigma$; neither needs to be known in advance. The reference need not be the identity line. One justification is the probability integral transform: for an independent normal statistical sample, $\Phi ((X_{(i)}-\mu )/\sigma )$ are uniform order statistics, whose typical values are near their plotting positions. The equality is approximate for a finite sample, particularly at the extremes. Systematic curvature, unusually heavy tails, or isolated extreme departures argue against the proposed distributional shape. A straight marginal Q-Q plot does not itself establish independence of observations.

Put $S={\sum }_i(x_i-\bar{x}{)}^2$ and $q=S/{\sigma }_0^2$. Under the null hypothesis, maximizing over the nuisance mean gives ${\hat{\mu }}_0=\bar{x}$. The unrestricted normal mean and variance maximum-likelihood estimators are $\hat{\mu }=\bar{x}$, $\widehat{{\sigma }^2}=S/n$. Thus, for $S>0$,The likelihood-ratio test statistic is consequentlyThe usual model assumes $n\ge 2$. The event $S=0$ has probability zero under a nondegenerate normal distribution; at that observation the unrestricted likelihood is unbounded as the variance tends to zero, and the ratio has limiting value zero. This is the likelihood-ratio test for a normal variance with unknown mean.

The diffusion generator is $\mathcal{L}=\mu {\partial }_x+\frac{1}{2}{\sigma }^2{\partial }_{xx}$. By the Feynman-Kac formula, the required conditional characteristic function solvesUse separation of variables in the form $F(t,x)=e^{iux}h(t)$. Since $F_x=iuF$ and $F_{xx}=-u^{2}F$, the temporal factor satisfies the ordinary differential equationThus $h(t)=\exp [(iu\mu -\frac{1}{2}{\sigma }^2{u}^2)(T-t)]$, andThis is bounded in absolute value by one, so it lies in the uniqueness class of the Feynman-Kac formula. As an independent check, integrating the stochastic differential equation gives $X_T=x+\mu (T-t)+\sigma (B_T-B_t)$. The independent Brownian increments have the normal distribution with mean zero and variance $T-t$, yielding exactly the same characteristic function. For $\sigma =0$, the conditional distribution is a point mass at $x+\mu (T-t)$, and the same boxed expression still applies.

For $a<0$, set $v_{\infty }=-{\sigma }^2/(2a)>0$. The candidate invariant probability law of a Markov process isTo prove invariance rather than merely guess a limiting probability distribution, initialize $X_t\sim \pi$ independently of future Brownian increments. Integrating the conditional characteristic function from part (i) against this centered normal distribution givesbecause $v_{\Delta }=v_{\infty }(1-e^{2a\Delta })$. The uniqueness theorem for characteristic functions implies $X_{t+\Delta }\sim \pi$ for every $\Delta \ge 0$. This proves $\pi P_{\Delta }=\pi$, as required. The time-homogeneous Markov property then also makes the process with this Ornstein-Uhlenbeck stationary initialization a strictly stationary process.

Here the diffusion generator is $\mathcal{L}=ax{\partial }_x+\frac{1}{2}{\sigma }^2{\partial }_{xx}$. The Feynman-Kac formula givesTry $F(t,x)=\exp [iu\alpha (t)x+\beta (t)]$, where $\alpha (T)=1$ and $\beta (T)=0$. Substitution and collection of the coefficients of $x$ and the constant term giveThe solutions of these ordinary differential equations, with the terminal conditions imposed, areFor $\Delta =T-t$, define the conditional varianceThen the required conditional characteristic function isIn particular, for $a\ne 0$ this exponent is $iu{e}^{a\Delta }x-\frac{{\sigma }^2{u}^2}{4a}(e^{2a\Delta }-1)$. The variance $v_{\Delta }$ is nonnegative for either sign of $a$, and its limit at $a=0$ is ${\sigma }^2\Delta$. Hence the solution is bounded in absolute value by one and the Feynman-Kac formula identifies it with the required expectation. The value $u=0$ gives one directly, including that case without dividing by $u$.

For an independent check, the integrating factor $e^{-as}$ and the Itô product rule giveThe stochastic integral has a normal distribution because its integrand is deterministic; the Itô isometry gives its variance as $v_{\Delta }$. For $a<0$ this is the Ornstein-Uhlenbeck transition distribution. The formula also covers the unstable case $a>0$, and degenerates to deterministic motion when $\sigma =0$.

With $G$ a standard normal random variable,For $f\in C_b^1$, in the usual sense that $f$ and its continuous derivative are bounded, the mean value theorem bounds the difference quotient by $a(t)\Vert f^{\prime }{\Vert }_{\infty }$. Dominated convergence theorem therefore allows differentiation inside the integral:This also follows pathwise from $X(t,x+h)-X(t,x)=a(t)h$. If the textual description were read as assuming only bounded differentiability, some control of $f^{\prime }$ is needed for the displayed right side; the stated $C_b^1$ hypothesis supplies it. Gaussian smoothing itself still differentiates a bounded measurable $f$, using the derivative of the Normal distribution instead.

For a partition $0=t_0<\cdots <t_n=1$, approximate the integral by $X_{\pi }={\sum }_j{t}_j(B_{t_{j+1}}-B_{t_j})$, using left endpoints. The defining independent Brownian increment of Brownian motion make this a centered Gaussian variable with variance ${\sum }_j{t}_j^2(t_{j+1}-t_j)$. As the mesh tends to zero, that variance tends to ${\int }_0^1{t}^{2}dt=1/3$.

The stochastic integral is the mean-square convergence of these sums. mean-square convergence implies convergence in distribution, while the characteristic function converge to $e^{-u^2/6}$. HenceThe probability is exactly one half because this centered nondegenerate normal distribution is symmetric and has no atom at zero.

Lévy's characterization of Brownian motion states that a real continuous local martingale $M$ with $M_0=0$ and quadratic variation $⟨M{⟩}_t=t$ is a standard Brownian motion with respect to its filtration. Equivalently, its increments are independent of the past and have centered normal distribution with variances equal to elapsed times.

Let $H_s=\mathrm{sgn}({\sup }_{r\le s}{B}_r-1)$. The running maximum is continuous and adapted, so $H$ is bounded and progressively measurable process. It has a predictable version for integration; changing its value at the single switching time does not change its stochastic integral. Thus $X=\int HdB$ is a continuous local martingale starting at zero. Since $H_s^2=1$,Lévy's characterization of Brownian motion yields $X$ is standard Brownian motion. The sign depends on the entire past through the running maximum, but the unit quadratic-variation rate is exactly what the characterization needs.

The Itô formula for $f(x)=x^4$ givesThe stochastic integral has expectation zero: its square integrability on a finite horizon follows from the finite sixth moments of a normal distribution. Since $\mathbb{E}{B}_s^2=s$, integration and expectation yield

For $t>0$, $B_t$ has the normal distribution $N(0,t)$. Evaluate its Gaussian integral:At $t=0$ the defining random variable is one, consistent with the same formula. Thus

Two assumptions of the usual normal linear model used for the reported coefficient Student t-tests are:

The second statement includes homoscedasticity. Normality concerns the conditional errors, rather than requiring the weights themselves to have a normal distribution.

Each observation has normal distribution probability density functionThe observations are independent and identically distributed, so their joint probability density function, viewed as a function of the unknown mean, is the product:Independence justifies multiplication; the common mean and known unit variance justify using this same density factor for every observation.

The exponential distribution prior has rate $\lambda =1/3$ and support $\mu \ge 0$, so its prior density is $(1/3)e^{-\mu /3}{1}_{\{\mu \ge 0\}}$. By Bayes' theorem,where $C$ is a normalizing constant independent of $\mu$.

Completing the square gives a useful exact reference for the numerical sampling. PutThen this normal mean posterior with an exponential prior is a normal distribution with location $m$ and variance $s^2$, truncated below at zero. With $\varphi$ the standard normal distribution density and $\Phi$ its cumulative distribution function,The exponential rate is $1/3$, not three; the prior mean is three. The support restriction must be retained in both the formula and the sampler.

The probability density function isEvery term is nonnegative because its weight is nonnegative, its scale is positive and its exponential is positive. By linearity of the integral and normalization of each normal distribution,Thus $f$ is nonnegative and integrates to one. It is a weighted sum of densities, rather than a product or an unweighted sum whose integral would equal two.