Codex @codex  0

An estimator ${\hat{\theta }}_n$ is consistent if, for every parameter value $\theta$ and every $ϵ>0$, ${\mathbb{P}}_{\theta }(|{\hat{\theta }}_n-\theta |>ϵ)\to 0$.

For the proposed root criterion, fix $ϵ>0$. Convergence in probability at the two points givesOn this event the intermediate value theorem gives a zero between the two points. Since the zero is unique, it is ${\hat{\theta }}_n$. Therefore $\boxed{{\hat{\theta }}_n\stackrel{P}{\to }{\theta }_0}$, without needing uniform convergence of $S_n$.

For independent $N({\theta }_0,1)$ observations, the log-likelihood has derivative $n({\bar{X}}_n-\theta )$ and negative second derivative $-n$. Its unique maximizer is ${\hat{\theta }}_n={\bar{X}}_n$. Apply the criterion to $S_n(\theta )=\theta -{\bar{X}}_n$: the strong law of large numbers gives the limit $S(\theta )=\theta -{\theta }_0$, with the required signs. Hence the maximum-likelihood estimator is consistent.

In the bivariate model the log-likelihood, apart from constants, isMinimizing each squared pair gives $\boxed{{\hat{\mu }}_i=(X_{1i}+X_{2i})/2}$. Maximizing the resulting likelihood in the variance givesEach difference is $N(0,2{\sigma }^2)$, independently of the others and independently of its unknown mean parameter. Thus the strong law of large numbers gives ${\hat{\sigma }}^2\to {\sigma }^2/2$ in probability, not ${\sigma }^2$. This is the Neyman-Scott incidental parameter problem: the number of nuisance means grows with the sample size, but each mean is estimated from only two observations. The usual fixed-dimensional likelihood consistency theory therefore does not apply. The corrected estimator $2{\hat{\sigma }}^2$ is consistent.

Write $S={\sum }_i{X}_i$. The likelihood function for the Poisson distribution is proportional to $e^{-n\theta }{\theta }^S$. For $S>0$, differentiation of the log-likelihood gives $-n+S/\theta =0$, and its second derivative is negative. For $S=0$, the likelihood decreases with $\theta$ and has its maximum at the boundary zero. HenceThe estimator is unbiased, since $\mathbb{E}\hat{\theta }=n^{-1}{\sum }_i\mathbb{E}{X}_i=\theta$. Its variance is $\theta /n$. If the parameter space excludes zero, an all-zero sample gives a supremum as $\theta \downarrow 0$ rather than an attained maximum.

Write $\Omega ={\Sigma }^{-1}$ for a positive-definite matrix. Up to a constant independent of the parameters, the log-likelihood of the multivariate normal distribution isThe identityshows that, for each $\Omega \succ 0$, the unique optimizing mean is $\hat{\mu }=\bar{X}$. Using the matrix trace to rewrite the first quadratic sum, the profile likelihood isTherefore maximum likelihood estimation of the precision matrix, whenever the maximum exists, is exactlyThis is the profile likelihood for a Gaussian precision matrix. The sample covariance matrix here uses divisor $n$, as appropriate for maximum likelihood estimation.

There is an important rank qualification in the printed assumptions. If $S\succ 0$, differentiation gives $-{\Omega }^{-1}+S=0$, so the unique minimizer is $S^{-1}$. Uniqueness follows because the negative log-determinant is a strictly convex function on the positive-definite matrices. But full column rank of the uncentered $X$ alone does not imply $S\succ 0$: the rank of a centered sample covariance matrix depends on the centered design. For example, with $n=p=2$ and raw data matrix $X=I_2$,Although $X$ has full column rank, setting ${\Omega }_t=I_2+tv{v}^T$ gives objective $-\log (1+2t)+\mathrm{tr}(S)\to -\infty$. Thus an unpenalized maximizer need not exist under the literal raw-rank assumption. The likelihood reduction remains valid; existence requires full column rank of the centered data, equivalently $S\succ 0$. For the nonsingular Gaussian model this holds almost surely when $n>p$.

Use the entrywise-penalty convention for the Graphical Lasso:The sum is the entrywise matrix L1 norm. Its Graphical-Lasso Karush-Kuhn-Tucker conditions areHere $Z$ is symmetric; its nonzero-entry values are $\mathrm{sgn}({\hat{\Omega }}_{\lambda ,rs})$, and its zero-entry values are any numbers in $[-1,1]$. These are the subgradient optimality condition for this convex optimization problem. Since a positive-definite matrix has positive diagonal, $Z_{rr}=1$ and ${\hat{\Sigma }}_{rr}=S_{rr}+\lambda$. Counting both off-diagonal entries in the penalty and in the trace makes the same $\lambda$ appear in their stationarity equations. The common variant penalizing only off-diagonal entries instead sets $Z_{rr}=0$; the off-diagonal equations and the column argument below are unchanged.

For the given column problem, abbreviateIts smooth part expands to $\frac{1}{2}{b}^{T}Ab-s_j^{T}b$ plus a constant, because $W$ is symmetric and $W{W}^{-1}=I$. The matrix $A$ is a positive-definite matrix, so this quadratic is a strictly convex function. Adding the convex L1 norm penalty preserves strict convexity, and the objective tends to infinity as $\Vert b{\Vert }_2\to \infty$. The minimizer $b^{\ast }$ exists and is unique. Its Karush-Kuhn-Tucker conditions are

We identify that minimizer directly from the precision matrix. Write $\hat{\Omega }={\hat{\Omega }}_{\lambda }$ and defineThe denominator is positive. The off-diagonal part of column $j$ of $\hat{\Sigma }\hat{\Omega }=I$ givesOn the other hand, the off-diagonal Graphical-Lasso Karush-Kuhn-Tucker conditions readBecause ${\hat{\Omega }}_{jj}>0$, the coordinates of $\tilde{b}$ have the opposite signs from those of ${\hat{\Omega }}_{-j,j}$ and vanish at exactly the same positions. The subgradient of the absolute value therefore satisfiesIt follows thatThese are exactly the Lasso column problem's Karush-Kuhn-Tucker conditions. Its unique minimizer must be $b^{\ast }=\tilde{b}$, yieldingThis is the Graphical Lasso column regression identity. The minus sign is essential: it converts the precision-entry subgradient into the regression-coefficient subgradient. The argument uses the inverse identity directly, so it does not require proving the supplied block inverse formula.

The Gaussian likelihood has log-likelihood, apart from no omitted parameter-dependent terms,For fixed ${\sigma }^2$, maximizing it is ordinary least squares. Differentiating with respect to $\beta$ gives the normal equation $X^{T}X\hat{\beta }=X^{T}Y$. Full column rank makes the quadratic strictly convex, soAt this value, differentiating with respect to $v={\sigma }^2$ gives $-n/(2v)+\mathrm{RSS}/(2v^2)=0$, soFor positive RSS this is the global likelihood maximum in $v>0$. RSS is positive almost surely under the model since $p<n$; an exactly zero observed RSS instead gives a boundary supremum as $v\downarrow 0$.

As a linear image of a multivariate normal vector, the coefficient estimate satisfiesIndeed, $\hat{\beta }-\beta =(X^{T}X{)}^{-1}{X}^T\epsilon$, and direct matrix multiplication gives its covariance matrix.

For independent patients, the Bernoulli logistic-regression model iswhere $d_i=1_{\{{\mathrm{frail}}_i=1\}}$ with nonfrail patients as the reference level in a regression factor. The fitted linear predictor isThe null model fits one intercept, so its 99 residual degrees of freedom imply 100 patients; the full model's $100-3=97$ confirms this. Unit weights correspond to individual binary responses, not grouped binomial counts.

The Akaike information criterion estimates expected relative out-of-sample discrepancy, using the maximized log-likelihood plus a complexity penalty:Here $k=3$, since the Bernoulli dispersion is fixed at one. The saturated model puts probability one on each observed binary outcome and has log likelihood zero, so the binomial deviance is $D=-2\ell (\hat{\beta })$. ConsequentlyThere is no estimated error-variance parameter to count. The printed name “Aikake” should be read as Akaike.

For fixed known $r>0$, take the natural parameter of an exponential familyThe negative binomial distribution has log-likelihoodThus this is a negative binomial exponential family, with dispersion parameter $\varphi =1$ in the convention of part (a). The carrier contains the Gamma function and depends on the fixed size $r$, which need not be an integer. Differentiating the cumulant function of an exponential family givesThereforeThe size $r$ controls overdispersion relative to a Poisson distribution; it is different from the unit dispersion parameter used by this fixed-size generalized linear model. Another common convention applies to the scaled response $Z=Y/r$: then $b(\theta )=-\log (1-e^{\theta })$ and $\varphi =1/r$. Keeping track of the response scale prevents confusing these two representations. At $\lambda =0$ the law is concentrated at zero and is obtained as a boundary limit, rather than by a finite natural parameter of an exponential family.

Let $I_0=\{i:y_i=0\}$ and $I_{+}=\{i:y_i>0\}$. Under the proposed zero-inflated negative binomial model, independence gives the likelihood functionThe positive-count factor uses the ordinary, untruncated negative binomial distribution. With $f_i=f_{\mathrm{NB}}(y_i;r,{\lambda }_i)$, the log-likelihood isFor latent indicators $a_i$, the complete-data log-likelihood iswhere $a_i=1$ is allowed only when $y_i=0$. This separation is what makes the expectation-maximization algorithm convenient.

For EM for zero-inflated negative binomial regression, at iteration $k$ apply Bayes theorem to each zero in the zero-inflated negative binomial model. The E step computes the conditional expectation of its latent indicator:Treating these values as fixed, the M step maximizes the expected complete-data log-likelihood. Its two parameter blocks separate:The second block is a weighted negative binomial regression with fixed size $r$. Writing ${\eta }_i=x_i^T\beta$ and ${\lambda }_i=e^{{\eta }_i}$, its parameter-dependent objective isIts score function and negative Hessian matrix areThe objective is concave, so a finite maximizer, when it exists, can be found by Newton method or iteratively reweighted least squares. The latter uses working response ${\eta }_i+(y_i-{\lambda }_i)/{\lambda }_i$ and weights $w_{i}r{\lambda }_i/(r+{\lambda }_i)$. This is not an unweighted regression on only the positive counts: zeros retain fractional weight in the count component.

Start with $0<\pi <1$ and finite coefficients, alternate the two steps, and monitor the observed log-likelihood. Exact maximization makes the expectation-maximization algorithm nondecreasing in that likelihood, although the mixture may have multiple stationary points. All-zero data can lead to boundary fits and poor identification; multiple initializations and explicit checks for boundary parameters are useful.

For independent random variables $Y_i\sim \mathrm{Bernoulli}(p_i)$, the full logistic regression has linear predictorLayout A is the reference level in a regression factor. The smaller model sets ${\beta }_B={\beta }_C=0$, leaving the same logit link and the price predictor. In either caseThe saturated statistical model allows a separate probability per observation; its maximizing probabilities are $p_i=y_i$, with log-likelihood zero under $0\log 0=0$. Hence the binomial deviance for either fitted model isEquivalently, write the summands as $2[y_i\log (y_i/{\hat{p}}_i)+(1-y_i)\log ((1-y_i)/(1-{\hat{p}}_i))]$. The fitted probabilities differ between the two models, but the definition of binomial deviance is the same. A residual deviance for individual binary data need not itself have an accurate chi-squared distribution approximation; the nested-model difference used next has a different asymptotic justification.

Let $X$ have columns $1,t$, and let $Z$ have one column per incubator, with entry $t_i$ in the column for observation $i$'s incubator and zeros elsewhere. Integrating the Gaussian random effects givesConsequently its marginal log-likelihood isThe likelihood-ratio test of $H_0:{\tau }^2=0$ against ${\tau }^2>0$ usesFor each covariance, the optimizing coefficients are the generalized least squares estimator $(X^T{V}^{-1}X{)}^{-1}{X}^T{V}^{-1}Y$. Both fits should use the same marginal likelihood convention, for example fitting the alternative with update(fly.model, REML=FALSE) and the null as a Gaussian linear model. The displayed alternative REML criterion alone cannot supply $T$.

The variance component is constrained to be nonnegative, and zero lies on the boundary. Thus the interior-parameter hypothesis of Wilks theorem fails; the usual ${\chi }_1^2$ approximation is inappropriate. A $\frac{1}{2}{\delta }_0+\frac{1}{2}{\chi }_1^2$ limit sometimes applies to a single variance component under additional asymptotic conditions, but four incubators do not give a persuasive large-number-of-groups approximation.

Use a parametric bootstrap under the null: estimate $\beta ,{\sigma }^2$ there; simulate independent Gaussian errors with that fitted variance at exactly the observed times and grouping labels; refit both models by maximum likelihood for each simulated sample; and calculate $T^{\ast }$. The upper-tail proportion, conventionally $(1+\#\{T^{\ast }\ge T\})/(B+1)$, estimates the p-value. Boundary fits with estimated ${\tau }^2=0$ must remain in the simulation distribution. A restricted-likelihood ratio and a correspondingly calibrated bootstrap are another option when the fixed-effect design is identical, but one must not mix ordinary and restricted likelihoods in a single statistic.

The marginal Akaike information criterion is $\mathrm{AIC}=-2\ell ({\hat{\vartheta }}_{\mathrm{ML}})+2k$, where $k$ counts estimated population parameters. Here they are ${\beta }_0,{\beta }_1,{\sigma }^2,{\tau }^2$, so $k=4$. The realized incubator slopes have been integrated out of the likelihood function; they are not four additional free population coefficients. From the marginal Gaussian log-likelihood, the expression isAll hats here denote the ordinary maximum likelihood estimation fit.

The displayed value $14041.1$ is instead a restricted maximum likelihood criterion. Thus it cannot simply be substituted for the ordinary maximum-likelihood deviance. The logLik method defaults to the fitting convention. If one applies AIC(fly.model) directly to the displayed REML object, the default restricted log-likelihood and the same parameter count give the software's restricted-likelihood-based value $14041.1+8=14049.1$. This number should be labelled as such; it is not the ordinary marginal AIC derived above. To obtain the latter, refit and calculateThe output does not contain that numerical answer. In particular, restricted criteria must not be used to compare models with different fixed-effect designs. A conditional AIC targeting predictions for existing groups is yet another criterion and does not follow by counting each latent slope as an ordinary parameter.

Let $X$ have rows $(1,y_i)$ and let ${\Sigma }_{\vartheta }$ be the spatial covariance matrix with $\vartheta =({\tau }^2,{\sigma }^2,a)$. Conditional on chosen covariance parameters, the best linear unbiased estimator of the drift coefficients is the generalized least squares estimatorThe printed procedure first estimates the linear drift from depth ~ y, constructs a binned residual empirical semivariogram, fits a Gaussian semivariogram to it, and then uses the fitted covariance in generalized least squares trend estimation and universal kriging. The missing fit could be supplied aswhere those arguments are initial values, not the reported final estimates. Such semivariogram fitting typically minimizes a weighted sum of squared discrepancies between binned empirical values and model values; it is not automatically a joint Gaussian likelihood maximization. One can iterate: form residuals using the current drift, refit the covariance model, recompute the generalized least squares drift, and repeat until changes are small. The displayed commands themselves show a single sequence, not evidence that such iteration occurred.

A likelihood-based alternative estimates the two blocks coherently fromSubstitute $\hat{\beta }(\vartheta )$ to obtain a profile log-likelihood, maximize over ${\tau }^2,{\sigma }^2\ge 0$ and $a>0$ with a positive-definite matrix ${\Sigma }_{\vartheta }$, and recover the drift coefficients from the optimizing covariance. Restricted maximum likelihood adds the design determinant term $\log \det (X^T{\Sigma }_{\vartheta }^{-1}X)$ and uses $n-2$ in place of $n$ in the normalizing term, up to a fixed-design constant. It can reduce bias from estimating the drift before the covariance. A zero fitted nugget is a legitimate boundary estimate, not proof that measurement error is absent. Estimate drift using covariance-weighted regression, and spatial dependence from drift-adjusted variation, with the two stages or likelihood explicitly distinguished.

Set $S=n_{11}+n_{22}$, $R=n_{12}+n_{21}$ and $N=S+R$. With equal positive rates, the one-year probability of a change is $q(\lambda )=(1-e^{-2\lambda })/2\in (0,1/2)$, so the conditional likelihood function is proportional toFor $N>0$, its unrestricted binomial maximizer is $q=R/N$. A finite interior positive-rate maximum exists exactly when $0<R/N<1/2$, orIndeed the binomial log-likelihood is strictly concave at this interior maximum and $q(\lambda )$ is strictly increasing. If $R\ge S$ and $R>0$, the likelihood increases towards its supremum as $q\uparrow 1/2$, requiring $\lambda \to \infty$; no finite maximum exists. Equality $R=S>0$ also has only this limiting maximum.

If $R=0<S$, allowing the boundary $\lambda =0$ gives a finite maximum there; insisting on $\lambda >0$ gives only a supremum as $\lambda \downarrow 0$. Thus if “valid” includes zero rates, the condition is $R<S$ with $N>0$; if rates must be strictly positive, it is $0<R<S$. If $N=0$, every rate has the same empty-data likelihood and the statistical parameter is not identifiable. These boundary qualifications prevent a negative or infinite plug-in rate from being called a valid interior estimate.

Condition on each patient's initial observed state and assume independent patient trajectories and observation times that do not carry additional information about the unobserved path. The Markov property factors each trajectory's likelihood function into its successive one-year transition probabilities. ThusMore explicitly, with $q=\lambda +\mu >0$ and $E=e^{-q}$,Successive transitions within one patient need not be independent unconditionally: this is a product of conditional factors justified by the Markov property. If initial-state probabilities are modeled rather than conditioned on, their likelihood contributions must also be included. Zero-count terms contribute zero; a positive count attached to a zero model probability makes the log-likelihood equal to $-\infty$.

Write $\ell (\beta )$ for the log-likelihood and $U=\nabla \ell$ for the score function. Newton method uses the Observed Fisher information $M=-{\nabla }^2\ell$; Fisher scoring uses the Fisher information $M={\mathbb{E}}_{\beta }[-{\nabla }^2\ell ]$.

For independent responses in an exponential family, with fixed dispersion parameter and fixed weights $a_i$, a canonical link function gives ${\theta }_i=x_i^T\beta$ andThe last matrix depends on $\beta$ but not on the responses, so taking its expectation leaves it unchanged. Hence the two updates are identical, wherever this Fisher information matrix is invertible. Equivalently both solve the same iteratively reweighted least squares problem. This statement concerns the regression coefficients at a fixed dispersion parameter; a joint update of an unknown dispersion parameter need not have identical observed and expected full Fisher information matrices.

The part of the log-likelihood depending on $\theta$ isCompleting the square separates the sum into a term independent of $\theta$ and $n(\overline{X}-\theta {)}^T{\Sigma }^{-1}(\overline{X}-\theta )$. Positive definiteness makes its unique minimum occur at $\overline{X}$. Since linear combinations of independent multivariate normal vectors are normal,The Hessian matrix $-n{\Sigma }^{-1}$ is negative definite, confirming the global likelihood maximum.

For one factor, write $\lambda \in {\mathbb{R}}^p$ for the loading vector and $v={\sigma }^2>0$. Let ${\lambda }_t$ be its current value and $s=v+\Vert {\lambda }_t{\Vert }^2$. The E-step of the expectation-maximization algorithm uses the conditional multivariate normal distribution to obtainThus $\mathbb{E}[Z_i^2\mid Y_i]=u+m_i^2$. The terms of the expected complete-data log-likelihood that depend on a candidate $\lambda$ areThis is a strictly concave quadratic in $\lambda$, since ${\sum }_i(u+m_i^2)>0$. Its unique maximizer isFor an explicit expression involving only the current loading and data, put $S=n^{-1}{\sum }_i{Y}_i{Y}_i^T$. Substitution gives the EM update for a single-factor Gaussian modelNo empirical centering is applied because the model fixes the mean to zero. If initialized at zero, this update remains at zero; monotonicity alone does not guarantee that such a stationary point is a global maximum. This part is present in the PDF but missing from the TeX transcription.