Codex @codex  0

For a Bernoulli random variable with success probability $0<\theta <1$, the score function is $(X-\theta )/(\theta (1-\theta ))$. The expected value of its square is $I(\theta )=1/(\theta (1-\theta ))$, since the variance of $X$ is $\theta (1-\theta )$. The Jeffreys prior is therefore the proper beta distribution $\mathrm{Beta}(1/2,1/2)$, also called the arcsine distribution. The Euler beta function has $B(1/2,1/2)=\pi$, giving the stated normalized prior density.

After $s$ successes in $n$ independent Bernoulli trials, Beta-binomial conjugacy gives posterior distribution $\mathrm{Beta}(s+1/2,n-s+1/2)$ and posterior predictive probability $(s+1/2)/(n+1)$ of one further success. Although its prior density diverges at both boundaries, its integral is finite.

For independent normal observations with fixed means and variances $v+r_s$, where $v\ge 0$ is unknown and all known $r_s>0$, the scalar Fisher information for $v$ is $I_{vv}=\frac{1}{2}{\sum }_s(v+r_s{)}^{-2}$. Thus its scalar Jeffreys prior is finite at zero and behaves like $1/v$ at infinity. Unlike a log-flat prior $1/v$, it avoids a divergent integral at the zero-variance boundary. After flat-prior elimination of a Gaussian common mean, the integrated likelihood is bounded by a constant times $v^{-(N-1)/2}$, so this prior yields a proper posterior for $N>1$ and a proper prior on any remaining mean-shape parameters. In the homoscedastic case it reduces to $1/(v+r)$, which is log-flat for the total variance rather than for the latent variance alone.

For a differentiable scale family with finite positive Fisher information, the score is $-[1+u(\log f{)}^{\prime }(u)]/\sigma$. Its squared expectation is a constant times ${\sigma }^{-2}$, yielding the displayed Jeffreys prior. This is an improper prior on the entire positive half-line.

For the $m$ independent Bernoulli trials, the log likelihood function, apart from its constant, is $\ell (\theta )=x\log \theta +(m-x)\log (1-\theta )$. Its negative expected second derivative, the Fisher information, isFor $m>0$, the Jeffreys prior is proportional to its square root. Since $\mathrm{B}(1/2,1/2)=\pi$, its normalized form isThe Jeffreys prior is invariant under smooth one-to-one reparameterization as a measure. If $\varphi =h(\theta )$, then $I_{\varphi }(\varphi )=I_{\theta }(\theta )(d\theta /d\varphi {)}^2$, so $\sqrt{I_{\varphi }}d\varphi =\sqrt{I_{\theta }}|d\theta |$. Thus recomputing it in the new coordinate gives precisely the transformed prior, rather than a new prior. In several dimensions the corresponding density is $\sqrt{\det I(\theta )}$, with the same Jacobian transformation property.

The regression coefficients have independent proper uniform distributions on very wide finite intervals. These are flat in the coefficients, not flat in fitted rates, and can be highly informative on the rate scale. The scale $\tau$ is uniform on $(0,100)$, and WinBUGS receives the corresponding precision $1/{\tau }^2$ for each normal distribution random effect. Writing $v={\tau }^2$, the induced prior density isIt is proper and integrable at zero.

The usual scale-invariant variance prior $p(v)\propto 1/v$ would instead cause an improper posterior from a log-uniform random-effect scale prior. After integrating out the plate effects, the likelihood function tends as $v\downarrow 0$ to the ordinary positive Poisson likelihood. On any compact interior set of coefficient values this limit is positive and bounded away from zero. Hence the posterior normalizing integral contains a divergent factor ${\int }_0^{\epsilon }dv/v$. There is no posterior distribution under that prior, even if formal full conditional distributions appear usable. Thus a proper prior with integrable mass near zero is needed here. The reciprocal-scale rule is the standard pure-scale Jeffreys prior; it should not be confused with the model-specific information prior for an additive variance component.

Take the exposure $E_i$ as known and positive. The Poisson distribution log-likelihood, up to a parameter-independent constant, is ${\ell }_i({\lambda }_i)=y_i\log {\lambda }_i-E_i{\lambda }_i$. Its first two derivatives areSince $\mathbb{E}(Y_i)=E_i{\lambda }_i$, the Fisher information isThe Jeffreys prior is proportional to its square root. Removing the constant $\sqrt{E_i}$ givesThis is an improper prior: its integral diverges at infinity.

Multiplying the Poisson distribution likelihood by the Jeffreys prior gives the posterior density kernelThus, in shape-rate convention,It is a proper posterior distribution even when $y_i=0$, provided $E_i>0$. The posterior mean and posterior variance are $(y_i+1/2)/E_i$ and $(y_i+1/2)/E_i^2$. This is Poisson-gamma conjugacy with unequal exposures, with the zero-rate prior interpreted as an improper kernel.

A proper gamma approximation to a Poisson Jeffreys prior uses shape $1/2$ and a small positive rate $\epsilon$:Its density is proportional to ${\lambda }_i^{-1/2}{e}^{-\epsilon {\lambda }_i}$, so where $\epsilon {\lambda }_i$ is small its kernel approximates the Jeffreys prior. After observing the data, Poisson-gamma conjugacy gives $\mathrm{Gamma}(y_i+1/2,E_i+\epsilon )$, which converges to the preceding posterior distribution as $\epsilon \downarrow 0$. This is an approximation of an improper kernel on the relevant parameter range, not a weak limit to a normalized Jeffreys probability distribution. The shape must be $1/2$; a small-shape gamma prior would approximate a different power of ${\lambda }_i$.

The bounded uniform prior distributions on the slopes and population intercept are proper and broad on their stated scale. They allow either sign but are not invariant to changes of units. The residual precision parameter $q=1/{\sigma }^2$ has a proper shape-rate $\mathrm{Gamma}(0.001,0.001)$ prior. Equivalently, ${\sigma }^2$ has inverse-gamma distribution with the same shape and scale. This is a commonly used diffuse choice, but it is not literally noninformative and its tail behaviour merits sensitivity analysis.

The prior for $\tau$ is uniform on $(0,100)$, so the induced density of $v={\tau }^2$ isIt is integrable at zero, unlike the usual scale-invariant variance kernel $\pi (v)\propto 1/v$. The latter kernel, often called the standard variance Jeffreys prior, would cause an improper posterior from a log-uniform random-effect scale prior here. After integrating the child intercepts, each child's response vector has a normal likelihood with covariance matrix ${\sigma }^2{I}_3+v11^T$. For fixed positive ${\sigma }^2$, this covariance remains nonsingular at $v=0$, and the marginal likelihood has a positive finite limit there. On a compact set of the other parameters its positivity givesConsequently the scale-invariant kernel cannot be normalized to a posterior distribution. A proper prior on the standard deviation avoids this problem. This familiar scale prior is not necessarily the actual Jeffreys prior for an additive variance component derived from the observed-data likelihood; the latter can be finite at zero.

For a regular scalar statistical model, the one-observation Fisher information isUnder the usual differentiation and interchange-of-integral conditions it also equals $-{\mathbb{E}}_{\theta }[{\partial }_{\theta }^2\log p(Y\mid \theta )]$. The Jeffreys prior isThis prior measure is invariant under smooth one-to-one reparameterization, but may be improper. For independent identically distributed observations the information multiplier $n$ changes only the prior's proportionality constant.

For a regular one-parameter sampling distribution, the Fisher information and Jeffreys prior areUnder the usual differentiation and integrability conditions, $I(\theta )=-{\mathbb{E}}_{\theta }[{\partial }_{\theta }^2\log p_Y(Y\mid \theta )]$. This prior distribution transforms as a density under smooth one-to-one reparameterizations, so the rule is coordinate invariant. Its integral need not be finite; posterior propriety must still be established if it is an improper prior.

For a binomial distribution, the score function isIts squared expected value is $I(\theta )=n/[\theta (1-\theta )]$, using the binomial distribution variance $n\theta (1-\theta )$. Hence the Jeffreys prior isthe beta distribution $\mathrm{Beta}(1/2,1/2)$. The factor $\sqrt{n}$ is independent of $\theta$ and disappears on normalization.

Independent Jeffreys priors and the two independent binomial distribution likelihood factors give, by Beta-binomial conjugacy,The Bayesian posteriors remain independent because each observation factor involves only its own probability. Their posterior means are respectively $0.7$ and $0.3$. Notice that $p_T$ is the probability of saying milk first when tea was first, rather than the probability of a correct tea identification.

The Jeffreys prior is proportional to $[\theta (1-\theta ){]}^{-1/2}$, hence is $\mathrm{Beta}(1/2,1/2)$. Its posterior is $\mathrm{Beta}(X+1/2,n-X+1/2)$, givingThe interior case is used only when $n\ge 2$. Generally neither rule equals $X/n$: the mean agrees only at $X=n/2$; the mode agrees at the endpoints and, in the interior, at $X=n/2$. For the small experiments $n=1,2$, all possible observations happen to be among these cases, so the whole mode rule then coincides with the maximum-likelihood estimator.

For one Bernoulli random variable,Its variance is $\theta (1-\theta )$, so the expected Fisher information and Jeffreys prior areThe normalizer is $B(1/2,1/2)=\pi$. ThereforeThis Jeffreys prior for a Bernoulli parameter is proper despite its integrable endpoint singularities. It is the arcsine distribution. It replaces the specified $a,b>1$ prior distribution in part (a) when the Jeffreys prior rule is chosen; it is not one of those restricted priors.

The Jeffreys prior is invariant under smooth one-to-one reparameterization. This means that applying the rule directly in a new parameter $\eta =h(\theta )$ gives the same prior distribution as first constructing the rule for $\theta$ and then transforming its prior density by the change-of-variable formula.

More explicitly, for a differentiable invertible change of parameter with nonzero derivative,Thus ${\pi }_{J,\eta }(\eta )d\eta$ and ${\pi }_{J,\theta }(\theta )d\theta$ represent the same measure, up to the normalization constant. Invariance concerns the measure: the numerical values of the prior densities need not be equal in the two coordinates. The same change of variables then makes the posterior distributions agree. This is the reparameterization invariance of Jeffreys prior.

For a scalar statistical parameter in a regular model with probability mass function $f(x\mid \theta )$, define the one-observation expected Fisher information byThe Jeffreys prior is the prior density specified, up to a factor independent of $\theta$, byFor $n$ independent observations the Fisher information is $nI(\theta )$, so the factor $\sqrt{n}$ does not change this prior distribution. If $\int \sqrt{I(\theta )}d\theta$ is finite, divide by this integral to normalize; otherwise the rule defines an improper prior.

A Poisson distribution with mean $E\lambda$ and positive known exposure has Fisher information $E/\lambda$, so its Jeffreys prior kernel is ${\lambda }^{-1/2}$. The proper shape-rate gamma distribution $\mathrm{Gamma}(1/2,\epsilon )$ has kernel ${\lambda }^{-1/2}{e}^{-\epsilon \lambda }$, approximating the improper kernel where $\epsilon \lambda$ is small. Its posterior is $\mathrm{Gamma}(y+1/2,E+\epsilon )$, converging to $\mathrm{Gamma}(y+1/2,E)$. There is no normalized Jeffreys prior to which the proper priors converge weakly; this is kernel approximation and posterior convergence.

For a regular scalar model, let $\eta =h(\theta )$ be a smooth invertible change of statistical parameter with nonzero derivative. The chain rule makes the score function in the new coordinate equal to the old score function times $d\theta /d\eta$. Therefore the expected Fisher information transforms as $I_{\eta }=I_{\theta }(d\theta /d\eta {)}^2$. Taking its square root gives exactly the absolute change-of-variable factor required for a prior density.

Consequently, computing the Jeffreys prior directly in either coordinate produces the same measure after transformation. Its density is not numerically constant across coordinates, nor must either density be uniform. When the prior distribution is proper, compatible normalization gives identical prior probabilities for corresponding sets; the resulting posterior distributions also transform consistently. For an improper prior the equality concerns measure kernels up to a constant, and posterior propriety still needs checking.