Codex @codex  0

For the normal linear model, the log-likelihood isThe full column rank of the design matrix makes $X^{T}X$ invertible. Differentiating in $\beta$ gives the normal equations $X^{T}X\hat{\beta }=X^{T}Y$. With the minimized residual sum of squares substituted, differentiation in ${\sigma }^2$ givesThese are the maximum-likelihood estimators almost surely; $RSS>0$ almost surely when ${\sigma }^2>0$ and $n>p$. The linear transformation of the multivariate normal distribution yieldsDefine the hat matrix $H=X(X^{T}X{)}^{-1}{X}^T$. The fitted values are $\hat{Y}=HY=X\hat{\beta }$, the regression residuals are $e=Y-\hat{Y}=(I-H)Y$, and $RSS=e^{T}e$. The orthogonal projection $I-H$ has rank $n-p$ and annihilates $X\beta$. By Cochran's theorem,Consequently $E({\hat{\sigma }}^2)=(n-p){\sigma }^2/n$, with bias $-p{\sigma }^2/n$. The unbiased estimator isIts square root, the reported residual standard error, estimates $\sigma$; the assertion of unbiasedness applies to the variance, not generally to its square root.

For the paper-strength analysis, let $h_i$ be the measured percentage and $z_i=h_i-\overline{h}$. Both normal linear models use independent errors of common variance ${\sigma }^2$. Their mean functions are ${\beta }_0+{\beta }_1{z}_i$ for lm1, and ${\beta }_0+{\beta }_1{z}_i+{\beta }_2{z}_i^2$ for lm2, with separately fitted coefficients. The reported residual standard errors are $11.82$ and $4.42$, respectively.

For lm1, the estimated conditional expectation at a new percentage $x$ isThe original data mean $\overline{h}$ must be used for centering the new percentage; its numerical value is not supplied in the excerpt. Because ${\sum }_i{z}_i=0$, the two columns of this design matrix are orthogonal, and $\mathrm{Cov}({\hat{\beta }}_0,{\hat{\beta }}_1)=0$. Hence the coefficient standard errors in the output giveThis estimates uncertainty in the mean. Predicting an individual future batch would additionally require the new-error variance, estimated by $11.82^2$.

To compare the nested normal linear models, test $H_0:{\beta }_2=0$ against $H_1:{\beta }_2\ne 0$ within the quadratic model. The Student t-test statistic isIts two-sided p-value is $1.89\times 10^{-8}$. Equivalently $T^2$ is a partial F-test statistic with null distribution $F_{1,16}$. Reject the linear restriction and prefer lm2 at the 5% level. Its residual spread is much smaller; the increase in the coefficient of determination from $0.3054$ to $0.9085$ supports the same conclusion, although the test is the relevant complexity-adjusted comparison.

For regression diagnostics, inspect regression residuals against fitted means and hardwood percentage for omitted curvature, a scale-location plot for nonconstant variance, and a quantile-quantile plot against the normal distribution for departures from the error assumption. Check unusual observations using regression leverage and Cook's distance, and examine residuals in collection or batch order if dependence is plausible. Independence and a common variance require substantive justification as well as these plots; a small p-value for a polynomial term does not itself check the error model.

For woman $i$, write $m_i$ for the number of births, $Y_i$ for the number with defects, $a_i$ for baseline age, and $e_i$ for exposure. Let $p_i$ be the common marginal defect probability for her births. The first fit is a grouped grouped-binomial logistic regression:It assumes independent women and, conditionally on their covariates, independent births with the same probability within each woman. The supplied birth totals are the binomial denominators; fitting the proportions with weights $m_i$ is equivalent to this grouped-binomial specification. The fitted linear predictor is $-1.74272+0.02030a_i+2.33028e_i$.

The second fit is a generalized additive model with the same logit link function and the mean specificationThe functions $f_a,f_e$ are penalized cubic regression splines with natural boundary conditions. Centering constraints such as ${\sum }_i{f}_a(a_i)={\sum }_i{f}_e(e_i)=0$ separate them from the intercept; the fitted centered intercept is $-0.2031$. Their second derivative roughness penalties control complexity.

The call estimates a common scale rather than fixing it at one. Its working variance specification for the counts isEquivalently the variance of $Y_i/m_i$ is $\varphi p_i(1-p_i)/m_i$. This is the working moment interpretation of an overdispersed binomial generalized additive model: the code uses binomial deviance for fitting and estimated scale for inference. With $\varphi \ne 1$, it is not an exact independent-binomial sampling model. Women are still treated as independent groups, but within-woman dependence or unobserved heterogeneity can motivate the extra dispersion. The quasibinomial regression interpretation states what the scaled analysis assumes without inventing a full probability distribution for its counts.

For independent and identically distributed random variables sampled from a probability density function $f$, a kernel density estimator with kernel for density estimation $K$ and smoothing bandwidth $h>0$ isThe usual kernel for density estimation is nonnegative; finite $\int K^2$ makes the integrated variance finite. The mean integrated squared error and its bias-variance decomposition of mean squared error areThe interchange is justified by the Tonelli theorem.

For the exponential distribution and the specified unit-width uniform kernel, put $g_h(x)=\mathbb{E}{\hat{f}}_h(x)$. The expected value is $h^{-1}\mathbb{P}\{x-h/2\le X\le x+h/2\}$. Intersecting this interval with the positive half-line givesOn $[-h/4,0)$ the true probability density function vanishes, whereasHere $1-e^{-z}\ge z/2$ for $0\le z\le 1/4$. This leakage across the support boundary proves the integrated squared bias from a density jump:In particular, $h_0=1$ and $c=1/256$ suffice. The phenomenon differs from a smooth interior bias of a kernel density estimator: the order-one boundary error persists over a region of width proportional to $h$.

By independence, $\mathrm{Var}({\hat{f}}_h(x))=n^{-1}\{\mathbb{E}{K}_h(x-X{)}^2-g_h(x{)}^2\}$. Integrating the first term and using $\int K^2=1$ yieldsAlso $I(h)\le \int f^2=1/2$: the kernel density estimator mean is an average of translates of $f$, so the Jensen inequality followed by the Tonelli theorem bounds its squared integral.

To optimize over every $h>0$, we must establish that useful smoothing bandwidths approach zero, rather than optimize only a formal small-$h$ expression. An exact calculation supplies that justification. The autocorrelation integral of the exponential distribution density isAveraging over the two uniform windows, or integrating the piecewise expression for $g_h$, givesThese formulas show that $B$ is continuous on $(0,\infty )$, is strictly positive there because $g_h>0$ on $(-h/2,0)$, and tends to $1/2$ as $h\to \infty$. Consequently, $B$ is bounded away from zero on $[\delta ,\infty )$ for every $\delta >0$. Its expansion agrees with the supplied $B(h)=h/12+o(h)$.

The choice $h_n=\sqrt{12/n}$ givesChoose smoothing bandwidths within $1/n$ of the infimum. Their mean integrated squared error tends to zero, and nonnegativity of the integrated variance gives $B(h_n)\to 0$, hence $h_n\to 0$. For any $\epsilon \in (0,1/12)$, eventuallyThe last step is the arithmetic-geometric mean inequality. Taking the lower limit and then $\epsilon \downarrow 0$ matches the upper bound. ThusThe original PDF has this constant; the TeX aid's $1/\sqrt{2}$ is a transcription error.

For fixed-design nonparametric regression, let $x_i=i/n$ and $K_h(u)=h^{-1}K(u/h)$. The degree-$p$ local polynomial estimator takes the fitted intercept ${\hat{m}}_h(x;p)={\hat{\beta }}_0$, whereUse a nonnegative regression kernel and an invertible local polynomial Gram matrix to obtain a unique fit. For $p=0$, solving the single weighted least squares normal equation gives the local constant estimator:At the interior point $x_0$, use the standard unit-integral regression kernel convention ${\mu }_0(K)=1$. Otherwise the leading variance below is $R(K)/(nh{\mu }_0(K{)}^2)$; unlike a kernel density estimator, the local constant estimator itself is unchanged by rescaling $K$.

The local constant estimator is a linear estimator in nonparametric regression with weights $W_i=K_h(x_i-x_0)/(n{s}_{0,h})$. The independent unit-variance errors implyTo control the bias of an estimator, differentiability at this one point is enough. WriteSymmetry makes ${\mu }_1(K)=0$, and the supplied moment approximation gives $s_{1,h}=o(h)$. Therefore the linear contribution to the bias of an estimator is $m^{\prime }(x_0)s_{1,h}/s_{0,h}=o(h)$. The remaining contribution has absolute value at most $\eta (h)h$, because only $|x_i-x_0|\le h$ receive positive weight. This is the interior first-order bias cancellation for local constant regression, and gives squared bias of an estimator $o(h^2)$. Combining with the variance provesFor fixed $\epsilon >0$, take $h_{\epsilon }={\epsilon }^{-1/3}{n}^{-1/3}$. It satisfies both asymptotic smoothing bandwidth conditions, and the displayed remainder, multiplied by $n^{2/3}$, tends to zero for this fixed $\epsilon$. ThusLetting $\epsilon \downarrow 0$ proves the optimized scaled pointwise error tends to zero. The order of these limits matters: no remainder uniform in $\epsilon$ has been assumed.

For the lower bound over the whole function class, the absence of a uniform bound on derivatives permits a particularly direct argument. It also avoids assuming a normal distribution for errors when the model only supplies their first two moments. Fix $n$ and $x_0$. Choose a smooth bump function $b$ on the interval, with $b(x_0)=1$, whose support contains no design point except possibly $x_0$. Such a bump exists because the design is finite; at an endpoint use the restriction of a smooth bump on the real line. Every $m_a=ab$, $a\in \mathbb{R}$, belongs to the differentiability class.

If $x_0$ is not a design point, all observation means are zero for every $a$. The Le Cam two-point lemma applied to $a=\pm A$ has total variation distance zero and gives a lower bound tending to infinity with $A$. In fact, the supremum of the mean squared error is infinite at every such $x_0$.

If $x_0$ is a design point, only its response depends on $a$. The resulting location family is the single observation $Y=a+ϵ$; all remaining observations are independent of $a$ and give no extra information. The single-observation location minimax bound isHere is a proof covering discrete as well as continuous errors. Give $a$ the prior distribution $U[-A,A]$, independently of the noise. Put $p_M=\mathbb{P}(|ϵ|\le M)$ and $v_M=\mathrm{Var}(ϵ\mid |ϵ|\le M)$. Consider an easier experiment in which an oracle also reveals whether $|ϵ|\le M$, and reveals $a$ itself on the complementary event. On the event $|ϵ|\le M$ and $|Y|\le A-M$, every possible truncated noise value places $a=Y-ϵ$ within $[-A,A]$. The flat prior distribution then makes the Bayesian posterior noise law exactly its truncated original law. The smallest conditional squared-error loss is its conditional variance $v_M$, attained by the posterior mean.

The probability of that event is at least $p_M(1-2M/A)$ for $A>2M$, by restricting further to $|a|\le A-2M$. Thus every rule's supremum risk function is at least the original Bayes risk, which is at least the easier experiment's Bayes risk, and therefore at least $p_M(1-2M/A)v_M$. Let $A\to \infty$ and then $M\to \infty$. Finite second moment and zero mean give $p_M{v}_M\to 1$. The rule $T(Y)=Y$ has constant mean squared error one, proving the equality.

Consequently, at every $x_0\in [0,1]$, the unrestricted differentiability class satisfies the stronger conclusionThus $c=1$ works for every $n\ge 1$. There is no conflict with the preceding fixed-function limit: the worst functions, including arbitrarily narrow bumps, can vary with $n$. This is the pointwise versus uniform risk distinction. A smoothness ball with a common derivative bound would pose a different minimax risk problem.

By Poisson-gamma conjugacy, the posterior density is proportional toThus the shape-rate posterior isIts posterior mean is $A/B$ and its variance is $A/B^2$. The parameter $B$ is a rate, rather than a scale.

For a binomial distribution, the score function isIts squared expected value is $I(\theta )=n/[\theta (1-\theta )]$, using the binomial distribution variance $n\theta (1-\theta )$. Hence the Jeffreys prior isthe Beta distribution $\mathrm{Beta}(1/2,1/2)$. The factor $\sqrt{n}$ is independent of $\theta$ and disappears on normalization.

Start with the Beta distribution draw in the Beta-binomial exchangeable coupling. Write $s=\alpha +\beta$ andThese are the expected value and variance of $p_M$. The fixed $n$ in this coupling is a dependence parameter chosen by the investigator, not necessarily the observed number of cups. The four numbered stages together address the single printed correlation request.

The next Beta distribution has conditional expected value and varianceFor large $n$, its expected value approaches $X/n$, while its variance is at most $1/[4(s+n+1)]$. The final draw therefore stays close to the first. These are prior distributions for the cup probabilities: the auxiliary count is a device for constructing dependence, rather than additional observed tea data.

Represent the independent locally flat intercept prior distributions by broad finite uniform priors, for example on $(-10,10)$; this is proper and approximately constant over plausible mortality logits. With $A$ calibrated above, rough BUGS code isUse $J=6$ and supply treated death counts $(3,7,5,102,32,22)$ with totals $(38,114,69,1533,209,680)$, and control death counts $(3,14,11,127,40,39)$ with totals $(39,116,93,1520,218,674)$. In BUGS, the second dnorm argument is a precision parameter, so 0.25 corresponds to variance four. Initialize the positive scale away from zero. Monitor $\mu ,\tau$ and study odds ratios, checking Markov chain Monte Carlo convergence diagnostics and sensitivity to the finite intercept bounds and scale prior distribution. The fitted hierarchy combines binomial sampling uncertainty with between-study heterogeneity.

The autoregressive polynomial factors as $(1-0.4z)(1+0.7z)$, with zerosBoth have modulus greater than one. The causality root criterion for an autoregressive model therefore gives a causal stationary solution. The moving-average polynomial has its only zero at $-2$, also outside the unit circle, so the invertibility of a moving-average model holds. There is no common root to cancel.

To use unit-variance white noise, put $W_t=Z_t/2$. One suitable pair isThen $\tilde{\varphi }(B)X_t=\tilde{\theta }(B)W_t$ with $W\sim \mathrm{WN}(0,1)$. The factor two changes the innovation scale, not the zero of the moving-average polynomial.

Write $X_t={\sum }_{j\ge 0}{\psi }_j{Z}_{t-j}$, with innovation variance four. The generating function of this linear process isEquating coefficients gives ${\psi }_0=1$, ${\psi }_1=0.2$ and ${\psi }_j=-0.3{\psi }_{j-1}+0.28{\psi }_{j-2}$ for $j\ge 2$. Thus the first five coefficients, including lag zero, areFor all remaining lags, partial fractions giveThe coefficients are absolutely summable, so the series converges in mean square and defines the causal moving-average expansion. In the unit-variance convention of part (c), $X_t={\sum }_{j\ge 0}{c}_j{W}_{t-j}$ with $c_j=2{\psi }_j$; its first five coefficients are $(2,0.4,0.44,-0.02,0.1292)$. This is the two-geometric-coefficient expansion of a causal ARMA(2,1) process.

White noise orthogonality gives, for any integer $h$,To see this directly, expand the covariance of the two convergent series. Only matching noise indices contribute. Cauchy-Schwarz inequality makes the coefficient-product sum finite, justifying the covariance limit.

There is also a closed expression. Set $a=9/11$, $d=2/11$, $r=0.4$, $s=-0.7$. For $h\ge 0$,and negative lags follow by symmetry. Dividing by the expression at zero gives the same autocorrelation function.

Interpret stationarity in the usual second-order time-series sense and assume nondegenerate noise, ${\sigma }_z^2>0$. For a two-sided autoregressive equation, the missing existence condition isIt is important to separate this from causality. If $|\varphi |<1$, the unique stationary solution is $X_t={\sum }_{j\ge 0}{\varphi }^j{Z}_{t-j}$. If $|\varphi |>1$, there is still a stationary solution, but it is anticausal:Both expansions converge in L2 because their coefficients are square summable. Substitution verifies the equation. Their means are zero and their covariance functions depend only on lag. Uniqueness follows by iterating the equation backward in the first case and forward in the second: the remainders ${\varphi }^n{X}_{t-n}$ or ${\varphi }^{-n}{X}_{t+n}$ tend to zero in L2 for any stationary finite-variance solution. This is the stationary versus causal solution of a two-sided AR(1) equation.

For $\varphi =\pm 1$, iteration givesThe variance of the right side is $n{\sigma }_z^2$. The variance of the left side is at most $4\mathrm{Var}(X_t)$ by stationarity and Cauchy-Schwarz inequality. These are incompatible as $n\to \infty$. Thus no weakly stationary finite-variance solution exists at those unit roots.

If the intended claim includes a causal innovation representation, its condition is instead $|\varphi |<1$, as in the next part. The stated white noise equation alone does not say that $Z_t$ is orthogonal to the past of $X$. If zero innovation variance is allowed, the unit-root exclusion has degenerate exceptions, such as random constant solutions when $\varphi =1$; the nondegenerate convention is necessary for the asserted nonexistence.