Codex @codex  0

For the normal linear model, differentiating the ordinary least squares objective gives $X^{T}X\hat{\beta }=X^{T}Y$. Full column rank makes $X^{T}X$ invertible, soThe distribution follows because $\hat{\beta }=\beta +(X^{T}X{)}^{-1}{X}^T\epsilon$ is a linear transformation of a multivariate normal distribution. Its expectation is $\beta$, and direct multiplication gives the displayed covariance matrix.

Let $H=X(X^{T}X{)}^{-1}{X}^T$, the orthogonal projection onto the design's column space. The residual sum of squares is $RSS=\Vert Y-X\hat{\beta }{\Vert }^2=Y^T(I-H)Y$. Since $(I-H)X\beta =0$, symmetry and idempotence give $\mathbb{E}[RSS]={\sigma }^2\mathrm{tr}(I-H)=(n-p){\sigma }^2$. Thusis an unbiased estimator.

For the two independent regressions choose $\beta =(a,b,c,d{)}^T$. The first $m$ rows of the design matrix are $(1,u_i,0,0)$ and the last $m$ rows are $(0,0,1,w_i)$. The centering conditions and distinct predictors giveIt follows thatThe two estimated slopes are independent normal random variables, as they depend on independent error blocks. ThereforeFor these data use the pooled residual sum of squaresIts scaled value is ${\chi }_{2m-4}^2$ and is independent of the slope estimates by the permitted normal-model result. Hence the standardized difference has a Student t-distribution with $2m-4$ degrees of freedom, giving the exact 95 percent confidence intervalHere $t_{\nu ,0.975}$ is the 0.975 quantile of that distribution. This is a confidence interval for the difference of independent regression slopes. As required by the original $p<n$ framework, this specialization needs $m>2$. Distinct predictors alone allow $m=2$, but then the fit is saturated, $RSS=0$ and there are no residual degrees of freedom to estimate the unknown variance or form this interval.

For the normal linear model, the log-likelihood isThe full column rank of the design matrix makes $X^{T}X$ invertible. Differentiating in $\beta$ gives the normal equations $X^{T}X\hat{\beta }=X^{T}Y$. With the minimized residual sum of squares substituted, differentiation in ${\sigma }^2$ givesThese are the maximum-likelihood estimators almost surely; $RSS>0$ almost surely when ${\sigma }^2>0$ and $n>p$. The linear transformation of the multivariate normal distribution yieldsDefine the hat matrix $H=X(X^{T}X{)}^{-1}{X}^T$. The fitted values are $\hat{Y}=HY=X\hat{\beta }$, the regression residuals are $e=Y-\hat{Y}=(I-H)Y$, and $RSS=e^{T}e$. The orthogonal projection $I-H$ has rank $n-p$ and annihilates $X\beta$. By Cochran's theorem,Consequently $E({\hat{\sigma }}^2)=(n-p){\sigma }^2/n$, with bias $-p{\sigma }^2/n$. The unbiased estimator isIts square root, the reported residual standard error, estimates $\sigma$; the assertion of unbiasedness applies to the variance, not generally to its square root.

For the paper-strength analysis, let $h_i$ be the measured percentage and $z_i=h_i-\overline{h}$. Both normal linear models use independent errors of common variance ${\sigma }^2$. Their mean functions are ${\beta }_0+{\beta }_1{z}_i$ for lm1, and ${\beta }_0+{\beta }_1{z}_i+{\beta }_2{z}_i^2$ for lm2, with separately fitted coefficients. The reported residual standard errors are $11.82$ and $4.42$, respectively.

For lm1, the estimated conditional expectation at a new percentage $x$ isThe original data mean $\overline{h}$ must be used for centering the new percentage; its numerical value is not supplied in the excerpt. Because ${\sum }_i{z}_i=0$, the two columns of this design matrix are orthogonal, and $\mathrm{Cov}({\hat{\beta }}_0,{\hat{\beta }}_1)=0$. Hence the coefficient standard errors in the output giveThis estimates uncertainty in the mean. Predicting an individual future batch would additionally require the new-error variance, estimated by $11.82^2$.

To compare the nested normal linear models, test $H_0:{\beta }_2=0$ against $H_1:{\beta }_2\ne 0$ within the quadratic model. The Student t-test statistic isIts two-sided p-value is $1.89\times 10^{-8}$. Equivalently $T^2$ is a partial F-test statistic with null distribution $F_{1,16}$. Reject the linear restriction and prefer lm2 at the 5% level. Its residual spread is much smaller; the increase in the coefficient of determination from $0.3054$ to $0.9085$ supports the same conclusion, although the test is the relevant complexity-adjusted comparison.

For regression diagnostics, inspect regression residuals against fitted means and hardwood percentage for omitted curvature, a scale-location plot for nonconstant variance, and a quantile-quantile plot against the normal distribution for departures from the error assumption. Check unusual observations using regression leverage and Cook's distance, and examine residuals in collection or batch order if dependence is plausible. Independence and a common variance require substantive justification as well as these plots; a small p-value for a polynomial term does not itself check the error model.

In the Gaussian white noise model, observe the entire pathwhere $W$ is standard Brownian motion and the deterministic drift belongs to L2 space on $[0,1]$. Equivalently, $dY(t)=f(t)dt+n^{-1/2}dW(t)$. For every deterministic $h\in L^2[0,1]$, the observed stochastic integral satisfiesThe noise $W(h)=\int hdW$ is an isonormal Gaussian process. In particular its variance is $\Vert h{\Vert }_2^2$. Gaussian white noise is interpreted through these integrals, rather than as an ordinary random function with a pointwise value at every time. No smoothness of $f$ is needed.

Here is the Gaussian maximum bound without independence. Put $M={\max }_{1\le i\le N}|g_i|$. It is integrable since it is bounded by ${\sum }_i|g_i|$. For every $\lambda >0$,The second inequality uses only the moment-generating function of the standard normal distribution, so independence is unnecessary. Jensen inequality gives $e^{\lambda EM}\le E{e}^{\lambda M}$, henceMinimizing at $\lambda =\sqrt{2\log (2N)}$ proves

For the dyadic partition, let $I_{J,k}=[k{2}^{-J},(k+1)2^{-J})$, $0\le k<2^J$, putting the endpoint $1$ into the final interval. The Haar scaling functionsform an orthonormal basis of the space $V_J$ of functions constant on each interval. The Haar wavelets can be written asFor $J\ge 1$, the constant function ${\varphi }_{0,0}=1$ together with ${\psi }_{j,k}$, $0\le j<J$, is another orthonormal basis of $V_J$. Indeed, each successive level splits a cell's two constants into their sum and difference; the number of basis elements is $1+{\sum }_{j=0}^{J-1}{2}^j=2^J$. Thus the Haar projection is the orthogonal projectionOn cell $I_{J,k}$ its value is $2^J{\int }_{I_{J,k}}f(t)dt$. The first formula also applies to $J=0$.

Estimate each coefficient by its observed stochastic integral:This is the Haar projection estimator in Gaussian white noise. The noise integrals over disjoint intervals form a Gaussian vector with zero off-diagonal covariance. The principle that uncorrelated jointly Gaussian variables are independent then makes these coefficients independent. ThereforeTaking expectations shows that ${\hat{\Pi }}_{V_J}f$ is an unbiased estimator of ${\Pi }_{V_J}f$, both coefficientwise and pointwise for the stated step-function representatives.

Only one scaling function is nonzero in each cell, so the supremum norm of the error has the exact formThe endpoint convention ensures the same identity at $1$. Apply the Gaussian maximum bound without independence with $N=2^J$ to obtain the supremum norm risk of a Haar projection estimator:The factor $2^{J/2}$ comes from cellwise scaling, while the extra square root of $J+1$ comes from taking the maximum over $2^J$ Gaussian errors.

Choose a proposal probability density function $q$ positive wherever $|h|f$ is nonzero, up to null sets, and draw independent and identically distributed random variables $Y_1,\dots ,Y_n$ from $q$. The ordinary importance sampling estimator isThe support condition for importance sampling and absolute integrability give ${\mathbb{E}}_q[h(Y)f(Y)/q(Y)]=\int hf=\mu$, so this is an unbiased estimator. Its variance, possibly infinite, isLet $A=\int |h|f$. If $A>0$, the Cauchy-Schwarz inequality yieldsEquality holds for $q$ proportional to $|h|f$. Thus the minimum-variance importance distribution and the minimum variance areFor a constant-sign integrand this is zero variance. If $A=0$, the integrand vanishes almost everywhere and the zero estimator already has zero variance. Requiring the proposal to cover all of the target's support, including where $h=0$, can exclude $q_{\ast }$; in that stricter class, $(1-ϵ)q_{\ast }+ϵf$ approaches the same infimum as $ϵ\downarrow 0$. This distinguishes coverage of the integral from coverage of every target event.

Use the allowed ratio-of-expectations approximation where $G(u)>0$. The unconditional numerator and denominator from part (b) giveThe common censoring survivor function cancels in this approximation. By differentiating the logarithm, its value is the population integrated hazard:This is not generally the arithmetic mean of the $H_i(t)$; it is the cumulative hazard function of the population mixture from part (a). The cancellation requires a common independent censoring law and adequate observation of the time range.

The approximation establishes approximate unbiasedness, not exact finite-sample unbiasedness. For a direct counterexample, take one uncensored individual with exponential distribution of rate $\lambda$. The Nelson–Aalen estimator is then $1_{T\le t}$, whose expectation is $1-e^{-\lambda t}$, whereas $\overline{H}(t)=\lambda t$. Thus it is not an unbiased estimator in general.

More explicitly, for independent individuals with a common event hazard $h$, the exact expected estimator isThis exhibits finite-sample bias of Nelson–Aalen estimation and dependence on $G$ when empty risk sets are possible. In a heterogeneous population, replacing the random ratios by ratios of their means introduces an additional approximation. On adequately observed intervals with large risk sets, the population-mixture expression is the appropriate large-sample target.

First, the absolute values in the printed limit are an error: the correct long-run variance of a stationary process is the signed sum of autocovariances. Directly,soEach coefficient tends to $1$ and has absolute value at most $1$. Absolute summability and the dominated convergence theorem therefore giveThe sum of absolute values is an upper bound, not the general limit. For a concrete counterexample to the printed assertion, take $X_t={\eta }_t-{\eta }_{t-1}$ with unit-variance iid noise. Then $\gamma (0)=2$, $\gamma (\pm 1)=-1$, and other covariances vanish. The absolute sum is $4$, but ${\overline{X}}_T=({\eta }_T-{\eta }_0)/T$, so $T\mathrm{Var}({\overline{X}}_T)=2/T\to 0$.

For correlated data the central limit theorem can still hold under suitable strong mixing of a stationary process and moment conditions, but the limiting variance is the long-run variance of a stationary process, rather than the one-observation variance. When it is positive,Positive serial dependence usually increases the standard error, while negative dependence can reduce it. The approximate effective sample size of a stationary sample is $T{\gamma }_X(0)/{\sum }_h{\gamma }_X(h)$ when that denominator is positive. A long-memory time series can require a different normalization or a different limit law. Absolute covariance summability alone does not prove a CLT: if $X_t=V{\eta }_t$ with a common independent random scale $V$ taking values $1$ and $2$ with equal probabilities and iid ${\eta }_t$ with the standard normal distribution, the off-diagonal covariances are zero, yet $\sqrt{T}{\overline{X}}_T$ has the nonnormal scale-mixture law $VN(0,1)$. The example is not an ergodic stationary process. If the long-run variance of a stationary process is zero, the usual nondegenerate square-root-$T$ CLT is unavailable.

For the causal AR(1), $|\varphi |<1$ andThe conditional distribution follows because the current innovation is independent of the past.

For a fixed known $\varphi$, condition on the observed $X_1$ and use the $n=T-1$ transitions $t=2,\dots ,T$. Their conditional maximum likelihood criterion is, up to constants,Differentiating in $\mu$ yieldsSince $X_t-\varphi X_{t-1}=(1-\varphi )\mu +{\epsilon }_t$,Thus it is an unbiased estimator, even conditionally on $X_1$, andIt has statistical consistency with mean-square convergence, and the iid-noise strong law of large numbers also gives almost-sure statistical consistency. Its conditional distribution is exactly normal with the displayed mean and variance. If $X_0$ is also observed and all $T$ transitions are used, replace $T-1$ by $T$.

The fixed-$\varphi$ qualification is necessary for the exact finite-sample claims. If $\varphi$ is jointly estimated, conditional likelihood function is linear regression with an intercept: writing ${\bar{X}}_{-}=n^{-1}{\sum }_{t=2}^T{X}_{t-1}$ and ${\bar{X}}_{+}=n^{-1}{\sum }_{t=2}^T{X}_t$, the unconstrained estimators areThis ratio is not generally an unbiased estimator and does not have the preceding finite-sample variance. Under the usual stationary regression conditions it has statistical consistency; its asymptotic variance is ${\sigma }_{\epsilon }^2/(1-\varphi {)}^2$. Indeed, it differs from ${\bar{X}}_{+}$ by $\hat{\varphi }(X_T-X_1)/(n(1-\hat{\varphi }))$, an asymptotically negligible endpoint term. Profiling an unknown innovation variance does not change the fixed-$\varphi$ estimate of $\mu$.