Codex @codex  0

The empirical score outer-product matrix isUnder regularity and consistency of $\hat{\theta }$, $i_n(\hat{\theta })$ converges in probability to the Fisher information matrix. This convention differs from the negative-Hessian convention for Observed Fisher information.

Differentiating the log-likelihood again givesThus the Observed Fisher information for the rate is proportional to the total number of observed events. At a positive interior maximum-likelihood estimator, the large-sample variance estimate and standard error arePrecision is therefore driven by the number of observed events rather than by the number recruited alone. Heavy right censoring can leave a large study with little information about its hazard function. Under the regularity conditions for the Fisher information, the expected information is $\mathbb{E}[D]/{\theta }^2$.

A right-censored observation is not wholly uninformative: its exposure contributes to $E$, changes the maximum-likelihood estimator, and affects the expected number of events under a sampling design. Its contribution $-\theta x_i$ happens to have zero second derivative in this rate parameterization. For example, if no events occur, the likelihood still declines as $e^{-\theta E}$ and disfavors large rates, despite its zero curvature. The usual interior variance approximation is then invalid.

Write $u(Y_i;\theta )={\partial }_{\theta }\log f(Y_i;\theta )$ and $U_n(\theta )={\sum }_{i}u(Y_i;\theta )$ for the individual and total score functions. Under the usual regularity assumptions, ${\mathbb{E}}_{{\theta }_0}u=0$ and ${\mathrm{Cov}}_{{\theta }_0}(u)=I({\theta }_0)$, the nonsingular per-observation Fisher information matrix. The central limit theorem and weak law of large numbers giveRearranging the stipulated score expansion, including its remainder, givesSlutsky theorem therefore proves the asymptotic normality of a maximum likelihood estimator:A consistent plug-in Fisher information matrix yields the Wald statistic $W_n=n({\hat{\theta }}_n-{\theta }_0{)}^{T}I({\hat{\theta }}_n)({\hat{\theta }}_n-{\theta }_0)$. Under the null hypothesis, $W_n\stackrel{d}{\to }{\chi }_d^2$, so a test of asymptotic size $\alpha$ rejects when $W_n$ exceeds the $(1-\alpha )$ quantile of that chi-squared distribution. A consistent Observed Fisher information matrix gives the same limiting test.

With an $r$-dimensional parameter of interest $\psi$ and nuisance parameter $\lambda$, partition $I$ accordingly. The marginal asymptotic covariance matrix of $\hat{\psi }$ is the $\psi \psi$ block of $I^{-1}$, whose inverse is the efficient informationThus the nuisance-adjusted Wald statistic is $W_{\psi ,n}=n(\hat{\psi }-{\psi }_0{)}^T{I}_{\mathrm{eff}}(\hat{\theta })(\hat{\psi }-{\psi }_0)$, with limiting null distribution ${\chi }_r^2$. Using $I_{\psi \psi }$ alone is generally incorrect when the nuisance parameter is estimated; it is valid when the cross-information vanishes.

For the Inverse Gaussian distribution in this problem, expand the exponent to obtain the log likelihood, up to terms independent of both parameters,Its score functions areFor each positive $\psi$, the first score changes sign from positive to negative at $\lambda =\bar{Y}$, so this is the global maximizing mean parameter of an exponential family. Substituting it in the second score givesThe reciprocal function is strictly convex, so the denominator is positive for a nonconstant sample. The profile log likelihood is strictly concave in $\psi$ and tends to minus infinity at both endpoints, proving this is a maximum. Under continuous model sampling, a sample of size at least two is nonconstant almost surely. For a constant sample, including a one-observation sample, no finite joint maximum-likelihood estimator exists: the likelihood increases without bound as $\psi \to \infty$ at $\lambda =\bar{Y}$.

For one observation, differentiating again and using $\mathbb{E}Y=\lambda$ givesThis parameter orthogonality proves that estimating $\lambda$ does not reduce the efficient information for $\psi$. The Wald statistic has the same algebraic form in the two cases:Using information evaluated at ${\psi }_0$ instead is another first-order equivalent convention.

The printed claim of coincidence needs a first-order interpretation. Known and unknown $\lambda$ need not give numerically identical Wald statistics, because their maximum-likelihood estimators differ. With $\lambda$ known, direct maximization givesFor example, observations $1,2$ and known $\lambda =1$ give ${\hat{\psi }}_K=4$ and ${\hat{\psi }}_U=12$. At ${\psi }_0=1$, the displayed Wald convention gives $9/16$ and $121/144$, respectively. Thus exact equality is false. Under the null hypothesis, however, $\bar{Y}-\lambda =O_p(n^{-1/2})$, hence ${\hat{\psi }}_K-{\hat{\psi }}_U=O_p(n^{-1})$. Their centered, standardized estimates differ by $o_p(1)$, so their Wald statistics differ by $o_p(1)$. This is the valid asymptotic coincidence supplied by inverse Gaussian shape estimation and nuisance orthogonality.

For independent observations in a natural exponential family, the log likelihood isIn a minimal exponential family the Observed Fisher information matrix is positive definite, so the log likelihood is strictly concave. Any finite solution of the likelihood equation $\nabla \kappa (\hat{\theta })=\bar{Y}$ is therefore the unique global maximum likelihood estimator. Existence is a separate issue: the observed sufficient-statistic mean must belong to the image of the mean parameter map $\nabla \kappa$.

For a full regular minimal natural exponential family this image is the interior of the convex support of an exponential family, $C=\overline{\mathrm{conv}(\mathrm{supp}(h\nu ))}$. The necessity follows from a supporting hyperplane argument. If a finite parameter had its mean on a boundary face, some nonzero $v$ would satisfy $v^{T}Y\le b$ almost surely and $\mathbb{E}(v^{T}Y)=b$. Equality of the expectation forces $v^{T}Y=b$ almost surely. Exponential tilting preserves the base-measure null sets, so this contradicts minimality.

For sufficiency, let $\bar{Y}$ be an interior point of $C$. Choose finitely many points $y_1,\dots ,y_m$ of the support whose convex hull contains a ball about $\bar{Y}$, say of radius $2\epsilon$. Such points exist because $\bar{Y}$ is interior to the closed convex hull; one may approximate the vertices of a small simplex surrounding it by finite convex combinations of support points. Take support neighbourhoods of radius at most $\epsilon$ with positive finite base-measure masses $a_j$. These masses can be taken finite by restricting to bounded neighbourhoods and using finiteness of the normalizing integral at any fixed interior parameter. For every vector $\theta$, at least one $j$ has ${\theta }^T(y_j-\bar{Y})\ge 2\epsilon \Vert \theta \Vert$. Integrating over that neighbourhood yieldsThus the negative log likelihood grows to infinity as $\Vert \theta \Vert \to \infty$. At a finite boundary point of the full open natural parameter space, the normalizing integral is infinite, since otherwise that point would itself belong to the full domain. Fatou's lemma makes $\kappa (\theta )$ diverge on approach to such a boundary. A minimizing sequence for $\kappa (\theta )-{\theta }^T\bar{Y}$ therefore stays in a bounded subset away from the boundary; it has an interior limit where the minimum is attained. Its gradient vanishes, giving the required likelihood equation. Strict convexity gives uniqueness.

Hence, under these full, regular and minimal conventions,This is maximum likelihood existence in a full regular natural exponential family. Regularity alone does not make every sample admissible: in a Bernoulli distribution an all-zero or all-one sample has its sample mean on the boundary of $C=[0,1]$, and the canonical maximum is approached only as $\theta \to -\infty$ or $+\infty$. The usual probability parameter then has a boundary maximum in an enlarged model, but no finite canonical maximizer. A regular restriction of the natural parameter space requires membership in its actual mean-map image, rather than merely in $\mathrm{int}C$. Without minimality, affine redundancies may prevent uniqueness of the canonical parameter even when the fitted probability distribution is unique.

Since the observed times are ordered, the final risk set is $R_m=\{m\}$. Its Cox partial likelihood factor is thereforefor either value of the last event indicator. Every earlier risk set and earlier event factor is unaffected by changing $v_m$. Thus the entire partial likelihood as a function of $\beta$ is unchanged, not merely its value at one fitted coefficient. Its maximizing set, its score function and its Observed Fisher information are identical. In particular, any consistently chosen finite fitted $\hat{\beta }$ is unchanged. The lone terminal event conveys no relative-hazard information because there is no other individual with whom to compare it.

Use the usual semiparametric Cox proportional-hazards model with observed covariates $z_i$:The covariates are implicit in the question's regression fit, even though the displayed data notation only lists exit times and event indicators. Assume entry at a common origin, time-constant covariates, distinct observed times and independent censoring. Write $R_i=\{j:x_j\ge x_i\}$ and $w_j(\beta )=e^{{\beta }^T{z}_j}$. Conditional on a single event at $x_i$ in its risk set, the chance that individual $i$ supplies it isThe baseline hazard cancels. Consequently estimate $\beta$ by maximizing Cox partial likelihoodFor example, the score and information of Cox partial likelihood follow by differentiation. Define $S_i^{(0)}={\sum }_{j\in R_i}{w}_j$, $S_i^{(1)}={\sum }_{j\in R_i}{w}_j{z}_j$ and $S_i^{(2)}={\sum }_{j\in R_i}{w}_j{z}_j{z}_j^T$. ThenWhen a finite identifiable maximum exists, iterate $\beta \leftarrow \beta +I(\beta {)}^{-1}U(\beta )$ until convergence. The Observed Fisher information also gives the usual asymptotic coefficient uncertainty. Censored subjects enter the relevant risk sets, even though they supply no numerator event factor.

After fitting $\hat{\beta }$, estimate the cumulative hazard of the reference subject by the Breslow estimatorIts jump at an event is the observed event count divided by the total fitted hazard multiplier at risk. The estimate is an integrated baseline hazard, with hazard represented by a measure placing these jumps at observed event times. If a smooth ordinary hazard function is wanted, smooth these increments with a chosen kernel and bandwidth; differentiating the step function as an ordinary function would incorrectly give zero away from the jumps. The fitted individual cumulative hazard is ${\hat{H}}_i(t)=e^{{\hat{\beta }}^T{z}_i}{\hat{H}}_0(t)$ over that individual's follow-up. The invariances below concern this Cox partial likelihood fit and this Breslow estimator; they need not hold for a fully parametric proportional-hazards fit or an arbitrarily smoothed hazard estimator.

Let $\ell (\theta ;x)=\log p(x\mid \theta )$. The maximum-likelihood estimator maximizes this function; the score function is $U={\partial }_{\theta }\ell$, the Observed Fisher information is $\hat{j}=-{\partial }_{\theta }^2\ell (\hat{\theta };x)$, and the Fisher information is $I_n(\theta )={\mathbb{E}}_{\theta }{U}^2=-{\mathbb{E}}_{\theta }{\ell }^{\prime \prime }$ under the stated differentiation regularity. Differentiating normalization gives $\mathbb{E}U=0$. For iid observations the score is a sum of iid centered terms with variance $I_1$, so the central limit theorem gives $U/\sqrt{n{I}_1}\Rightarrow N(0,1)$ when $0<I_1<\infty$. Under the additional usual consistency, identifiability and interior-MLE regularity, $\sqrt{I_n}(\hat{\theta }-\theta )\Rightarrow N(0,1)$. Differentiation under the integral alone does not imply all these MLE conditions.

For the specified autoregression, index the innovations so $X_i=\theta X_{i-1}+E_i$, with $x_0=1$. Conditional normal densities giveSet $A=\sum x_{i-1}^2\ge 1$, $B=\sum x_{i-1}{x}_i$ and $C=\sum x_i^2$. Completing the square givesThe likelihood ratio between two data vectors is independent of $\theta$ precisely when their coefficients $A,B$ agree, because its log is a constant plus $\theta \Delta B-{\theta }^2\Delta A/2$. The positive-density likelihood-ratio criterion therefore makes $(A,B)$ minimal sufficient. Since $(A,B)\leftrightarrow (\hat{j},\hat{\theta })$ is bijective, the requested pair is also minimal sufficient.

With the flat improper prior, the posterior kernel is $\exp [-A(\theta -\hat{\theta }{)}^2/2]$, which is integrable because $A\ge 1$. ThusConditionally on the observations, $S(\theta -\hat{\theta })$ is exactly standard normal, as is $E_1$. This is a posterior statement, not a claim that the estimator standardized by the random information is an exact normal sampling pivot.

Write $\ell (\beta )$ for the log-likelihood and $U=\nabla \ell$ for the score function. Newton method uses the Observed Fisher information $M=-{\nabla }^2\ell$; Fisher scoring uses the Fisher information $M={\mathbb{E}}_{\beta }[-{\nabla }^2\ell ]$.

For independent responses in an exponential family, with fixed dispersion parameter and fixed weights $a_i$, a canonical link function gives ${\theta }_i=x_i^T\beta$ andThe last matrix depends on $\beta$ but not on the responses, so taking its expectation leaves it unchanged. Hence the two updates are identical, wherever this Fisher information matrix is invertible. Equivalently both solve the same iteratively reweighted least squares problem. This statement concerns the regression coefficients at a fixed dispersion parameter; a joint update of an unknown dispersion parameter need not have identical observed and expected full Fisher information matrices.

Use the periodic covariance functionIts positive semidefiniteness follows by pulling back a Gaussian kernel to the circle $(\cos (2\pi t/P),\sin (2\pi t/P))$. A zero Gaussian process mean is an ensemble statement, whereas an exactly zero long-term average for every periodic light curve is stronger. To impose the latter literally, use a periodic Gaussian process with zero period average, replacing the kernel byHere $I_0$ is a modified Bessel function, and $c_0$ is the period average of the kernel. Subtracting each process's random period average produces this covariance, so it remains positive semidefinite. We use $k_0$ to enforce the stated mean subtraction; using $k_P$ gives the usual ensemble-zero-mean formulation and the same likelihood derivation.

Let $K_{ij}=k_0(t_i,t_j)$ and $R=\mathrm{diag}({\sigma }_1^2,\dots ,{\sigma }_N^2)$. The prior latent vector is $f\sim N(0,K)$ and the independent Gaussian noise is $\epsilon \sim N(0,R)$, so $y=f+\epsilon \sim N(0,V)$ with $V=K+R$. The Gaussian-process marginal likelihood isPositive measurement variances make $V$ positive definite even if $K$ is singular. Maximize $\ell (\mathcal{H})=-\frac{1}{2}{y}^T{V}^{-1}y-\frac{1}{2}\log |V|-\frac{N}{2}\log (2\pi )$ over all three hyperparameters, using a broad search in $P$ to find competing aliases, followed by joint optimization. Cholesky decomposition evaluates the likelihood without explicitly forming an inverse.

For a well-resolved interior maximum, let $J=-{\nabla }^2\ell (\hat{\mathcal{H}})$ be the Observed Fisher information in the coordinates $(A,\ell ,P)$. Then $\boxed{\hat{P}=(\hat{\mathcal{H}}{)}_P,{\sigma }_P\approx \sqrt{(J^{-1}{)}_{PP}}}$. Equivalently, the profile-likelihood curvature in $P$ gives the same local nuisance-adjusted variance; keeping the other hyperparameters fixed would generally underestimate it. A profile interval with ${\ell }_{\mathrm{prof}}(\hat{P})-{\ell }_{\mathrm{prof}}(P)=1/2$ is a local $1\sigma$ approximation. If the period likelihood is multimodal or poorly constrained, fit the complete posterior with proper hyperparameter priors and report its modes and marginal credible intervals rather than a single Hessian error bar. The sampling pattern can create period aliases that a single local search misses.

The second derivative isIt is negative when $v_{+}>0$, verifying a strict maximum. The Observed Fisher information is $v_{+}/{\theta }^2$: more observed side-effects sharpen the likelihood, while the curvature vanishes when no side-effect is observed and the maximum then lies at the boundary $\hat{\theta }=0$.