Codex @codex  0

Write $a=(2,1{)}^T$, so the data map is $a^{T}u$. The Gaussian likelihood and prior distribution giveCompleting the square, or using Gaussian conjugacy for a normal linear model, identifies a multivariate normal distribution with precision matrix $C_{\delta }^{-1}=I_2+{\delta }^{-2}a{a}^T$. Since $a^{T}a=5$, its posterior mean and covariance matrix areThe parameters areFor example, $(I_2+{\delta }^{-2}a{a}^T)C_{\delta }=I_2$, which checks the square completion.

The orthogonal vectorsare eigenvectors of the covariance matrix. Their respective variances areThey are independent under the posterior distribution, by independence of uncorrelated jointly normal variables. Thus uncertainty is different in the two directions even for positive noise: the data constrain $e_{\parallel }$, whereas $e_{\perp }$ spans the null space of the data map.

As $\delta \to 0$, the posterior distribution converges to the law ofThis is supported on $2u_1+u_2=m$, and its posterior mean is the minimum-norm least-squares solution. The prior distribution still determines the distribution along the unobserved null space; it has not disappeared. This example of a Gaussian posterior in the zero-noise limit distinguishes exact recovery of an observed component from recovery of the whole unknown.

The conditional mean estimator is the posterior meanprovided the first moment is finite. A maximum a posteriori estimate is any maximizer of the posterior density relative to Lebesgue measure:Existence of the posterior density alone does not guarantee a finite posterior mean or existence of a maximizing point.

For the stated multivariate normal distribution and independent standard Gaussian noise, the negative log of the posterior density, up to a constant, isIts precision matrix $A^{T}A+{\Sigma }^{-1}$ is a positive-definite matrix, so Gaussian conjugacy for a normal linear model givesThe two forms agree becauseThe covariance matrix is $(A^{T}A+{\Sigma }^{-1}{)}^{-1}$, and the maximum a posteriori estimate equals the posterior mean for this multivariate normal distribution.

The time-zero budget constraint isso ${\theta }^0=w_0-{\theta }^T{S}_0$. The terminal wealth is thereforePutThenand the target mean is the linear constraint ${\theta }^{T}b=c$.

Since the invertible covariance matrix $V$ is positive definite, the Lagrange multiplier equation for minimizing ${\theta }^{T}V\theta$ isThusThe constraint determinesThis is the one-period Gaussian minimum-variance portfolio; the riskless holding is then ${\theta }^0=w_0-{\theta }^T{S}_0$.

Let $L(t)=f(x,t)$ be the likelihood function. Since the prior distribution is $N(0,I_p)$, the posterior density has the formwhere $Z$ is the normalizing constant.

Choose $0<\delta \le 1/2$ and use the Gaussian autoregressive proposal reversible with respect to a standard normal distributionThuswhose covariance matrix is the required $2\delta I_p$. Reversibility with respect to the standard normal distribution sayssoThe Metropolis–Hastings acceptance probability therefore reduces toThis is the Preconditioned Crank–Nicolson algorithm with proposal scale $\beta =\sqrt{2\delta }$.

Principal component analysis centers the word-count vectors, diagonalizes their sample covariance matrix, and projects onto the eigenvectors with the largest eigenvalues. Fitting logistic regression to $d<n$ principal-component scores removes exact collinearity and yields a lower-dimensional design.

Each principal component is an unsupervised linear combination of many words chosen to explain predictor variance; it is not a word selected for association with spam. The dimension can be chosen from a scree plot or cumulative explained variance, but cross-validating the downstream classification loss better targets prediction.

Write the Cepheid measurement as $\hat{\mu }=\mu +{ϵ}_{\mu }$, where ${ϵ}_{\mu }\sim N(0,{\sigma }_{\mu }^2)$, and define the calibrated absolute magnitude dataThus $q=(q_1,\dots ,q_K{)}^T$ has a multivariate normal distribution with mean $M_{0}1$ and covariance matrix

The Hubble law and the definition of distance modulus giveConsequently, withthe Hubble-flow observations are independent random variables satisfying $x_i\sim N(M_0-\theta ,{\sigma }_{\mathrm{tot}}^2)$. Apart from a factor independent of the parameters, the joint likelihood function isThe expression extends by continuity to a singular limiting covariance such as ${\sigma }_I=0$.

For $\varphi (x)=x$ and centered data, the objective is the Rayleigh quotientThis is the first-direction optimization in principal component analysis. Therefore $\hat{u}$ is any unit eigenvector of the sample covariance matrix $S$ corresponding to its largest eigenvalue.

Define the softmax weightsThey obey $p_i\ge 0$ and ${\sum }_i{p}_i=1$. The gradient is their weighted mean,Differentiating once more gives the covariance-form Hessian matrixFor every unit vector $u$,where $G={\max }_i\Vert a_i{\Vert }_2$. The Hessian is a covariance matrix, so it is positive semidefinite; the displayed upper bound also gives ${\nabla }^2{f}_{\beta }⪯\beta G^{2}I$ in the Loewner order. Consequently $f_{\beta }$ has a Lipschitz gradient with

Put $f=(f(x_1),\dots ,f(x_n){)}^T$, so $Y=f+\epsilon$ and $Y^{\ast }=f+{\epsilon }^{\ast }$ with independent noise vectors having covariance matrix $I_n$. For any deterministic linear smoother $H$,whereas independence givesExpanding the first trace shows that the second expression exceeds the first by $2\mathrm{tr}(H)$, proving the identity.

For ridge regression,Its effective degrees of freedom are $\mathrm{tr}(H_{\lambda })$. Thus training error is optimistically biased for independent-copy prediction error by $2\mathrm{tr}(H_{\lambda })/n$. The graph exhibits exactly this effect: the dashed training curve keeps falling as $\lambda$ decreases, while the solid test curve eventually rises through overfitting.

Let $X_c$ be the column-centered data matrix. Its sample covariance matrix is $S=X_c^T{X}_c/(n-1)$. The loadings matrix $V=(v_1,\dots ,v_p)$ has orthonormal eigenvectors of $S$ as columns, ordered by nonincreasing eigenvalues:Each principal component loading specifies the coefficients of a principal direction in the original variables. Signs are arbitrary, and equal eigenvalues permit orthogonal changes of basis within their eigenspaces.

The least-squares estimators depend only on the observed responses and design, so the formulas in i still hold. Their variances and covariance need only the error covariance matrix, now $I_n$. ThereforeFor example the first formula follows from $\mathrm{Var}({\sum }_i{x}_i{ϵ}_i)={\sum }_i{x}_i^2$, and the second from ${\sum }_i{x}_i=0$. The formulas remain the same as before with ${\sigma }^2=1$; this calculation does not establish independence or a normal distribution for the estimators.

For the random vector $\mathbf{ϵ}=({ϵ}_1,\dots ,{ϵ}_n{)}^{\top }$, its covariance matrix isIts diagonal entries are variances, and its off-diagonal entries measure pairwise covariance. Finite second moments make every entry well defined.

The future mean response is $E_{\ast }=\alpha +\beta x_{\ast }+\gamma z_{\ast }$. By invariance of maximum likelihood estimation, its estimator is the fitted meanIts sampling variance is the quadratic form of the coefficient covariance matrix with $c=(1,x_{\ast },z_{\ast }{)}^T$:This estimator is unbiased because each fitted coefficient is unbiased. The future error ${ϵ}_{\ast }$ does not enter the estimator of the mean, so its independent variance is not part of this expression.

Write $A=(X^{T}X{)}^{-1}{X}^T$. Since $AX=I_p$,The errors have mean zero and covariance matrix ${\sigma }^2{I}_n$. ThereforeThus $\hat{\theta }$ is an unbiased estimator. In addition, the multivariate normal distribution is preserved by linear maps, so $\hat{\theta }$ is multivariate normal with the displayed mean and covariance matrix.

Let $v_i=[(X^{T}X{)}^{-1}{]}_{ii}>0$. Part (b) givesThe residual vector and coefficient estimator are independent: they are jointly normal and their cross-covariance matrix is zero, since $X^T(I-H)=0$. Hence $Z_i$ is independent of $U=(n-p)s^2/{\sigma }^2\sim {\chi }_{n-p}^2$. By the definition of Student's t-distribution,Let $t_{n-p,1-\alpha /2}$ be the $1-\alpha /2$ quantile of this distribution. Inverting the central probability statement gives the exact marginal confidence intervalIts coverage is $1-\alpha$ under the stated Gaussian linear model; simultaneous coverage for all coefficients would require a separate adjustment.

Assuming independent Gaussian measurement errors with the quoted standard deviations, use the inverse-variance weighted mean:Here ${\sum }_i{w}_i=0.654514$ and ${\sum }_i{w}_i{x}_i=65.805556$, givingThe consistency statistic is ${\chi }^2=4.45$ for four degrees of freedom, so there is no evident excess scatter requiring error rescaling. A shared systematic error would require a covariance matrix and would not be reduced by this averaging rule.

A chi-squared test compares discrepancies with their expected random size. For independent Gaussian observations with known standard deviations, a fitted model givesUnder a correct regular model with $p$ fitted parameters, its reference statistical degrees of freedom are usually $\nu =n-p$. Correlated errors instead require ${\chi }^2={\mathbf{r}}^T{C}^{-1}\mathbf{r}$ with the correct covariance matrix. The reduced statistic has mean about one and standard deviation $\sqrt{2/\nu }$, so it should not be forced to equal exactly one.

For binned counts, Pearson's statistic is ${\sum }_i(O_i-E_i{)}^2/E_i$. With a fixed total and $k$ populated categories, fitting $p$ distribution parameters generally gives $\nu =k-1-p$. The approximation needs adequately populated expected bins and the appropriate dependence assumptions. Sparse data may require a Poisson likelihood, exact calculation or simulated reference distribution rather than a casual chi-squared approximation.

The upper-tail probability $p_{\mathrm{value}}=P({\chi }_{\nu }^2\ge {\chi }_{\mathrm{obs}}^2)$ measures how unusual the discrepancy would be under the null model; it is not the probability that the model is true. A large statistic can indicate a wrong model, underestimated noise or unmodelled systematic effects; an unusually small one can indicate overestimated errors or correlations. Residual plots and checks of the noise model are needed alongside the numerical test.

Define $s_i=\sin [2\pi (t_i-T)/P+\pi ]$ and use the independent quoted Gaussian errors $\sigma =10\mathrm{m}{\mathrm{s}}^{-1}$, with period and exoplanet transit epoch fixed. The model is linear in $K,V_0$ despite its sinusoidal time dependence. Minimize ${\chi }^2={\sum }_i(V_i-K{s}_i-V_0{)}^2/{\sigma }^2$. With $N=5$ the normal equations areUsing $\sum s_i=-0.741472$ and $\sum s_i^2=2.824633$, the solution isThe residuals in observation order are $(3.67,11.01,2.74,-7.90,-9.51)\mathrm{m}{\mathrm{s}}^{-1}$. Therefore ${\chi }_{\min }^2=2.950$ for $5-2=3$ degrees of freedom, giving reduced ${\chi }^2=0.983$. The upper-tail probability is about $0.40$: the fit is consistent with the stated errors, although five points do not establish a unique physical explanation.

For known errors the covariance matrix is ${\sigma }^2(X^{T}X{)}^{-1}$. Equivalently, profiling out the fitted offset givesThis is the $\Delta {\chi }^2=1$ one-parameter uncertainty, not the curvature with $V_0$ incorrectly held fixed. Propagating it through the supplied mass relation, and treating its coefficient and $P$ as fixed,The symbol $\pm nP$ in the exoplanet transit epoch denotes periodic copies of the same exoplanet transit phase, not an uncertainty distribution to propagate. No independent error for the empirical mass coefficient was supplied.

Additional independent observations retain positive weight even with twice the error: each has one quarter of the previous per-observation weight. Their information adds to the existing information matrix, so the formal mass uncertainty decreases for informative additional orbital phases and cannot increase under this fixed-error model. It can remain unchanged in a degenerate phase configuration that supplies no further amplitude information after fitting the offset. The conclusion assumes the same correct model and independent errors, not a new fitted error rescaling or unmodelled systematic effects.

A Gaussian distribution with mean $\mu$ and standard deviation $\sigma$ has probability densityIts symmetry, finite variance and simple behavior under linear combinations make it useful for repeated measurements. Roughly $68.3\%$, $95.4\%$ and $99.7\%$ of observations lie within one, two and three true standard deviations of the mean. These are coverage probabilities for a specified distribution, not guarantees that every experimental error is normal.

Why it appears. The central limit theorem explains why many sums of small independent finite-variance contributions approach a Gaussian law after centering and scaling, provided no single contribution dominates. Detector noise, small mechanical disturbances and multiple electronic fluctuations can combine approximately this way. The theorem does not remove a common calibration bias, and it does not justify assuming normal tails for rare large disturbances or arbitrarily correlated errors. A large-mean Poisson distribution also approaches a Gaussian with variance equal to its mean, making normal approximations useful for high-count photon or particle data; low counts require the discrete model.

A classical particle at thermal equilibrium in a one-dimensional harmonic potential has a Gaussian position distribution directly from the Boltzmann distribution:Thus a Gaussian law can arise from the energy dependence of equilibrium probability, rather than only from an informal appeal to many independent errors.

Precision of repeated measurements. For $N$ independent observations with the same mean and variance, the sample mean has variance ${\sigma }^2/N$. For Gaussian observations this mean is exactly Gaussian:Consequently quadrupling the number of independent readings halves the random standard error. When the true variance is unknown, a small-sample confidence interval for a normal mean uses a Student t-distribution with $N-1$ statistical degrees of freedom; replacing that by a normal quantile can overstate precision. Shared systematic errors and correlations do not decrease by the same averaging rule.

Combining results and fitting models. Independent Gaussian estimates $x_i$ of one quantity, with known variances ${\sigma }_i^2$, combine through the inverse-variance weighted meanFor a model $y_i=g_i(\theta )+{ϵ}_i$ with independent zero-mean Gaussian noise, maximizing the likelihood is equivalent to minimizing the weighted least-squares statisticFor a correct linear model with known independent Gaussian errors and $p$ fitted parameters of full rank, the minimized statistic has a chi-squared distribution with $N-p$ degrees of freedom. Nonlinear fits only inherit that familiar approximation under additional regularity and adequate-data assumptions. Known correlated Gaussian errors require the full inverse covariance matrix in the quadratic form.

These properties support confidence intervals, consistency checks, calibration and error propagation. They also give practical diagnostics: inspect residuals for trends, changing spread, correlations or outliers before trusting a Gaussian fit. Gaussian models connect physical fluctuations to quantitative uncertainty, but their assumptions must be checked rather than inferred from the shape of a convenient bell curve.

A chi-squared test compares squared deviations with the size expected from the assumed random errors. For independent observations with a normal distribution $y_i$ with known standard deviations ${\sigma }_i$ and predictions ${\mu }_i(\theta )$, useFor a correct fixed model, the standardized independent residuals are unit-variance normal distributions, so $Q$ has a chi-squared distribution with $\nu =N$. Fitting $p$ independent linear parameters removes $p$ residual directions, giving $\nu =N-p$. This distribution is exact for a full-rank linear model with known errors with a normal distribution; it is an approximation for many nonlinear fits. With known correlated errors with a multivariate normal distribution, replace the sum by $Q={\mathbf{r}}^{\mathsf{T}}{\Sigma }^{-1}\mathbf{r}$, where $\Sigma$ is the covariance matrix and its rank and fitted constraints determine the available statistical degrees of freedom.

Under the null hypothesis, $\mathbb{E}Q=\nu$ and $\mathrm{Var}Q=2\nu$. The reduced statistic $Q/\nu$ should therefore be near one, with standard deviation $\sqrt{2/\nu }$, rather than equal to one exactly. An upper-tail p-value isCompare it with a significance threshold chosen in advance. A small value indicates tension with the model together with its error assumptions; it can arise from a wrong mean model, underestimated uncertainties or neglected correlations. An unusually small statistic can suggest overestimated uncertainties, correlations or overfitting. A p-value is not the probability that the model is true, and failure to reject is not proof of the model.

For counts in categories, the Pearson form is $Q={\sum }_{j=1}^k(O_j-E_j{)}^2/E_j$. Its chi-squared distribution is an asymptotic approximation requiring adequate expected counts. For a fixed-total multinomial sample there is one normalization constraint, so the usual statistical degrees of freedom are $k-1-p$ after fitting $p$ other parameters. Independent Poisson bins instead start with $k$ statistical degrees of freedom before parameters are fitted. Sparse counts require an appropriate count likelihood or a calibrated exact or simulation-based test; plugging tiny expected counts into the asymptotic formula is unreliable. The error model, binning, fitted parameters and chosen test must therefore be reported with the result.