Codex @codex  0

The main sequence is the band in the Hertzsprung-Russell diagram occupied by stars whose main energy source is sustained core hydrogen burning. Different positions along it primarily reflect different masses: low-mass stars are cooler and fainter, while higher-mass stars are generally hotter and more luminous. Composition and age broaden the band, so it is not one exact universal curve.

A star spends a large fraction of its luminous lifetime in this comparatively long-lived stage. Its track begins near the zero-age main sequence and moves as its core composition changes, ending at the terminal-age main sequence. In a coeval population, the main-sequence turnoff identifies which stars have exhausted core fuel and therefore provides an age diagnostic.

Stellar nuclear fusion converts light atomic nuclei into more tightly bound products, releasing energy from the change in nuclear binding energy. In a main-sequence star, the net conversion of hydrogen to helium proceeds mainly through the proton–proton chain or the CNO cycle, depending on core conditions. The high temperature supplies energetic collisions, while quantum tunnelling allows reactions through the repulsive electrical barrier.

The released energy heats the interior and replaces energy lost through the photosphere and by neutrinos. A stellar thermostat arises because extra heating can expand the star, reduce core temperature, and lower the reaction rate. Later stellar evolution changes the available fuel and core structure, enabling helium burning and, in sufficiently massive stars, further burning stages.

Stellar structure follows from mass conservation, hydrostatic equilibrium, energy generation, and energy transport together with an equation of state and opacity law.

Flooring an exponential variable of rate $\theta$ produces a geometric variable on the nonnegative integers with success probability $1-e^{-\theta }$.

For a regular scalar model with score function $S_{\theta }$ and Fisher information $I(\theta )={\mathbb{E}}_{\theta }{S}_{\theta }^2$, an estimator $T$ with mean $\psi (\theta )$ satisfiesIndeed, differentiating ${\mathbb{E}}_{\theta }T=\psi (\theta )$ gives ${\mathrm{Cov}}_{\theta }(T,S_{\theta })={\psi }^{\prime }(\theta )$, and the Cauchy-Schwarz inequality gives the bound.

The risk of an estimator is its expected loss as a function of the unknown parameter. Under quadratic loss,

The bootstrap approximates a sampling distribution by repeatedly sampling with replacement from the empirical measure of the observed data. In the ordinary nonparametric version, observations are assumed independent and identically distributed; a bootstrap sample has the same size as the original statistical sample. A statistic, such as a sample mean or sample median, is recomputed on each resample. Its variation across resamples estimates how the original statistic would vary across repeated sampling from the unknown probability distribution.^[ref]

The key distinction is between the observed statistical sample and its still-unknown source probability distribution. Condition on the observed values $x_1,\dots ,x_n$ and form the empirical measureHere ${\delta }_{x_i}$ is a Dirac measure. Draw independent indices $I_1,\dots ,I_n$ from the uniform distribution on a finite set $\{1,\dots ,n\}$ and set $X_j^{\ast }=x_{I_j}$. Each observed slot has probability $1/n$ on every draw. If equal values occupy several slots, their total probability is the sum of those slot probabilities; distinct observed values need not be equally likely. A star denotes this conditional resampling experiment. There are $n^n$ equally likely ordered index sequences, although bootstrap count vectors and statistics can combine many such sequences.

A complicated sampling distribution need not stop statistical inference when recomputing the statistic is easy. The bootstrap reuses that calculation on simulated datasets. The plug-in estimator principle replaces the unknown probability measure by ${\hat{P}}_n$ in the whole repeated-sampling experiment. Whether that substitution approximates the target sampling distribution needs an appropriate bootstrap consistency theorem for the sample mean or a result for the particular statistic. A larger number of resamples reduces Monte Carlo error; it cannot repair the bootstrap failure for a uniform endpoint, an unrepresentative statistical sample or ignored statistical dependence.^[ref]

Quadratic discriminant analysis uses class-conditional Gaussian distributions with class-dependent covariance matrices. Their log density ratio is a quadratic polynomial in the observation.

LDA assumes class-conditional Gaussian distributions with a shared covariance matrix, producing affine log odds.

Let the joint random vector $(Y,S)$ have a multivariate normal distribution, with expectations ${\mu }_Y,{\mu }_S$, covariance matrices ${\Sigma }_Y,V$ and cross-covariance matrix $C=\mathrm{cov}(Y,S)$. If $V$ is invertible, the conditional expectation is the affine functionand the conditional covariance matrix is ${\Sigma }_Y-C{V}^{-1}{C}^T$.

To prove both assertions, define $R=Y-{\mu }_Y-C{V}^{-1}(S-{\mu }_S)$. Direct covariance calculation gives $\mathrm{cov}(R,S)=0$. Since $(R,S)$ has a multivariate normal distribution, this zero cross-covariance implies independence. The decomposition of $Y$ into its affine part and $R$ therefore gives the conditional expectation and the remaining covariance matrix. This is the conditional multivariate normal distribution calculation.

If $V$ is singular, replace $V^{-1}$ by its Moore-Penrose inverse $V^{+}$. The formulas hold for $S$ in its affine probability support, ${\mu }_S+\mathrm{range}V$; values away from that probability support do not determine a unique version of the conditional expectation. For example, if $S=(Z,Z{)}^T$ and $Y=Z$ for a standard normal random variable $Z$, then $V$ is the all-ones $2\times 2$ matrix. Although $V^{-1}$ does not exist, $C{V}^{+}S=(S_1+S_2)/2=Z$ on the probability support $S_1=S_2$.

The logistic location family has density and distribution function

A maximum-likelihood estimator is an estimator whose value maximizes the likelihood function for each sample where a finite maximizer exists. Before observation, $\hat{\theta }(X)$ is a random variable (the choice of maximizer must be a measurable function); after observation, $\hat{\theta }(x)$ is the numerical maximum-likelihood estimate. A formula must specify the allowed parameter set, including whether its endpoints belong to it.

For ten independent Bernoulli trials, write $K={\sum }_{i=1}^{10}{X}_i$. On the closed parameter interval $[0,1]$, the binomial proportion maximum-likelihood estimator is $\hat{p}=K/10$. Eight successes give the estimate $0.8$. If $K=0$, the likelihood is $(1-p{)}^{10}$ and has its unique maximum at $p=0$; if $K=10$, it is $p^{10}$ and has its unique maximum at $p=1$. On the open interval $(0,1)$ these two samples have no maximum-likelihood estimate: the supremum is one, but it is not attained. This is the same issue as likelihood supremum at an excluded boundary.

A complete continuous example uses independent and identically distributed random variables $X_i\sim N(\mu ,1)$ with unknown mean $\mu \in \mathbb{R}$ and known standard deviation one. Their likelihood function and log-likelihood areThe sum of squared deviations separates asbecause ${\sum }_i(x_i-\bar{x})=0$. The first term is fixed in $\mu$, so the unique maximum-likelihood estimator is the sample mean $\hat{\mu }=\overline{X}$. This also proves the equivalence with ordinary least squares for this model. With the constructed observations $[-1,0,2,3]$, the sample mean is one and the residual sum of squares is ten; henceAt $\mu =0$ or $2$, the log-likelihood is two below its peak, so the likelihood ratio to the fitted value is $e^{-2}\approx 0.135335$. The absolute likelihood at the peak is $(2\pi {)}^{-2}{e}^{-5}\approx 0.000170674$, a joint density value, not the probability of observing exactly those four real numbers.

The following figure compares that sample with a constructed sixteen-observation sample consisting of four repetitions of the displayed four values, modeled as sixteen independent draws. Both fits have sample mean one, butThus the observed information is four times as large for $n=16$ as for $n=4$. The distance from the peak at relative height $e^{-2}$ decreases from one to $1/2$: the width scales as $1/\sqrt{n}$. This is a comparison of two stipulated samples; merely copying recorded observations would not create new independent evidence.

Even uniqueness needs checking. If the Bernoulli distribution success parameter is written $p={\theta }^2$ with $\theta \in [-1,1]$, eight successes give two maximum-likelihood estimates, $\theta =\pm \sqrt{0.8}$, because both give the same probability distribution. The problem is a failure of identifiability. The special cases below show other ways an MLE can have bias or fail the usual asymptotic normality.

A log-linear model makes the logarithm of a positive expectation linear in model parameters. For independent cell counts with a Poisson distribution $N_a$ in a contingency table, write ${\mu }_a=\mathbb{E}{N}_a$ and $\log {\mu }_a=x_a^T\beta$. This is a Poisson regression with log link; finite linear predictors give strictly positive fitted cell expectations.

For a two-way table, the additive modelimplies ${\mu }_{ij}=e^{\alpha }{e}^{r_i}{e}^{c_j}$. Conditional on the total count, the cell counts have a multinomial distribution whose cell probabilities factor into row and column probabilities; this is independence of the two categorical classifications. For example, the cell-mean table $\left(\begin{array}{cc}2& 4\\ 3& 6\end{array}\right)$ has common column ratio two and odds ratio $(2\cdot 6)/(4\cdot 3)=1$.

Adding interaction terms ${\gamma }_{ij}$ to the linear predictor permits association. In a $2\times 2$ table the association is measured byIt vanishes in the additive model. Hence additive main effects describe one particular log-linear model; log-linear models also include interactions and more general design matrices. Independent Poisson sampling and multinomial sampling conditional on the total use different sampling schemes, while producing this same independence structure for the normalized cell probabilities.

The odds ratio is the ratio of two event odds. For rare events it is close to the corresponding risk ratio, because $p/(1-p)\simeq p$ when $p$ is small.

The risk ratio compares event probabilities in two groups:

In fixed-design ordinary least squares with a full-column-rank $n\times p$ design matrix $X$, the fitted response is $\hat{Y}=HY$, whereThe regression leverage of observation $i$ is the diagonal entry $h_{ii}$ of this hat matrix. Since $H$ is an orthogonal projection, $h_{ii}=\Vert H{e}_i{\Vert }^2$ lies between zero and one, and ${\sum }_i{h}_{ii}=\mathrm{tr}H=p$. Thus $p/n$ is the average regression leverage. These values depend on the design matrix, before any responses are observed.

If the errors have zero expectation and covariance matrix ${\sigma }^2{I}_n$, the regression residual vector is $(I_n-H)\epsilon$, and idempotence givesFor simple linear regression with an intercept and a nonconstant predictor,An observation far from the predictor sample mean has large regression leverage: it can pull the fitted regression function toward itself and therefore have a small regression residual. Large regression leverage, a regression outlier, and an influential observation describe different properties; their combination, rather than regression leverage alone, determines how much deleting an observation changes the fit.

$Y=X\beta +\epsilon$, $\epsilon \sim N(0,{\sigma }^{2}I)$, and full-rank $X$ gives $\hat{\beta }=(X^{T}X{)}^{-1}{X}^{T}Y$.

Independent Poisson variables conditioned on their sum are multinomial, with cell probabilities proportional to their rates.

A generalized linear model takes independent responses from an exponential dispersion family and relates their means to linear predictors by $g({\mu }_i)=x_i^T\beta$. The response distribution, systematic component, link, dispersion, and known observation weights together specify the model.

An exponential family has probability density function or probability mass function, relative to a common measure,Its natural parameter space consists of the vectors for which this integral is finite. The carrier $h$, sufficient statistic $T$ and probability support do not change with the natural parameter $\theta$.

For a Bernoulli distribution with $0<p<1$, take $T(y)=y$, $\theta =\log [p/(1-p)]$ and $A(\theta )=\log (1+e^{\theta })$. For a normal distribution with known variance $v$, take $h$ to be the $N(0,v)$ probability density function, $T(y)=y$, $\theta =\mu /v$ and $A(\theta )=v{\theta }^2/2$.

For independent and identically distributed observations, the likelihood function depends on them through ${\sum }_{i}T(Y_i)$, proving the sufficient statistic assertion by the Fisher-Neyman factorization theorem. At interior natural parameters, the gradient and Hessian matrix of $A$ give the expected value and covariance matrix of $T$. This common structure explains why otherwise different probability distributions share estimation methods.