Codex @codex  0

In Type Ia supernova cosmology, the predicted apparent magnitude depends on the common absolute magnitude $M_0$ and the Hubble constant only through $\beta =M_0-5{\log }_{10}(H_0/H_{\mathrm{ref}})$, after choosing a fixed reference $H_{\mathrm{ref}}>0$. A flat improper prior on $M_0$ makes the likelihood integrated over $M_0$ independent of $H_0$, by translation of the integration variable. If the other prior factors are independent and the posterior is proper, the marginal posterior of $H_0$ equals its prior. An external calibration of $M_0$ can break this lack of identifiability.

A Bayes factor measures how data change the relative support for two specified statistical models. Write $m_j(D)=p(D\mid H_j)$ for their Bayesian model evidence and define the direction explicitly:For positive finite $m_0(D),m_1(D)$, a value above one favors $H_1$ over $H_0$; a value below one favors $H_0$ over $H_1$. Reversing the comparison gives $B_{01}={\mathrm{BF}}_{01}=1/{\mathrm{BF}}_{10}$, the opposite direction used in some formulas below.

For mutually exclusive, exhaustive $H_0,H_1$ with positive prior probabilities, divide their two Bayes' theorem formulas. Their common data normalizer cancels, leavingThe Bayes factor therefore multiplies prior odds into posterior odds. It is evidence conditional on the specified statistical models and their within-model prior distributions; it is not $\mathbb{P}(H_1\mid D)$. Model prior probabilities supply a separate input.^[ref]

For an original stipulated example, consider ten independent and identically distributed Bernoulli trials, with heads as success. Compare $H_0:p=0.4$ with $H_1:p=0.7$, both fixed before seeing the data. Take $D$ to be one particular ordered sequence containing eight heads and two tails. The likelihood function under either point model is $p^8(1-p{)}^2$, soIf $\mathbb{P}(H_1)=0.1$, the prior odds for $H_1$ against $H_0$ are $1/9$. Hence the posterior odds are $21.9909706/9\approx 2.4434412$, andEqual model prior probabilities would instead give ${\mathrm{BF}}_{10}/(1+{\mathrm{BF}}_{10})\approx 0.956505$. The same Bayes factor can thus accompany different posterior probabilities. These statements condition on the two-model comparison; they do not establish that either statistical model describes a real coin.

Using the heads count $K=8$ instead of the ordered sequence changes the sample space. The binomial distribution assigns it mass $\left(\genfrac{}{}{0ex}{}{10}{8}\right)p^8(1-p{)}^2$. The same binomial coefficient multiplies both models' Bayesian model evidence and cancels in their Bayes factor. Mixing an ordered-sequence numerator with a count denominator would introduce an erroneous factor of $45$.

Fix the candidate statistical models, their parameter prior distributions, and model prior probabilities before using the current data for this update. The Bayes factor is useful because it separates evidence from model prior odds, but it still depends on the parameter prior distributions. An improper prior has an arbitrary multiplicative normalization: multiplying its kernel by $c$ multiplies its formal marginal likelihood by $c$, even if the within-model posterior distribution is normalizable. Without a separately justified convention, the ordinary Bayes factor is then undefined up to arbitrary constants.^[ref]

For one uniformly sampled integer $Y\in \{1,\dots ,N\}$, the likelihood function for the unknown endpoint is $N^{-1}{1}_{N\ge y}$. A flat improper prior on positive integers gives an improper posterior distribution, since the harmonic series diverges. A reciprocal prior instead gives a proper posterior proportional to $N^{-2}$ for $N\ge y$, with approximate survival probability $y/n$ and finite quantiles, but infinite posterior mean. A uniform prior cut off at $M$ gives posterior mean $(M-y+1)/(H_M-H_{y-1})\sim M/\log M$, demonstrating sensitivity to that arbitrary cutoff.

If a model configuration leaves a parameter absent from its likelihood function, giving that parameter an independent flat improper prior makes the posterior distribution normalization infinite whenever the configuration has positive prior distribution mass. Integrate first over that parameter: a positive constant likelihood function factor is repeated over all of $\mathbb{R}$. This occurs for the second segment mean of a Gaussian change-point detection model when the proposed split is after the final observation. There is no posterior distribution or valid Gibbs sampler for that joint kernel. A proper prior distribution on the unused parameter is necessary if the no-change configuration is retained.

For positive group totals, the improper prior $du/u$ on an independent Poisson baseline yields the multinomial likelihood kernel after integration. This prior is flat in $\log u$. A proper finite-variance log-rate prior generally introduces an additional coefficient-dependent factor.

For independent observations $y_s\sim N(\beta +d_s,V_s)$, put $a_s=V_s^{-1}$, $S={\sum }_s{a}_s$, $r_s=y_s-d_s$, $\bar{r}=S^{-1}{\sum }_s{a}_s{r}_s$ and $Q={\sum }_s{a}_s(r_s-\bar{r}{)}^2$. Integrating the likelihood function against a flat improper prior on $\beta$ gives, up to the arbitrary prior constant,Indeed ${\sum }_s{a}_s(r_s-\beta {)}^2=Q+S(\beta -\bar{r}{)}^2$, and the remaining one-dimensional Gaussian integral is $\sqrt{2\pi /S}$. The conditional Bayesian posterior of $\beta$ is $N(\bar{r},S^{-1})$.

The Haldane prior has formal beta distribution parameters $(0,0)$. It is an improper prior, with infinite mass at both endpoints of $(0,1)$. For a binomial distribution likelihood with $s$ successes and $f$ failures, its posterior is $\mathrm{Beta}(s,f)$ only when $s,f>0$. A density flat in the log odds induces this kernel by the Jacobian determinant $d\log [p/(1-p)]/dp=1/[p(1-p)]$.

The Jeffreys prior uses the square root of the determinant of the Fisher information matrix as a parameter-density kernel. It is invariant under smooth one-to-one reparameterization, since the information and density Jacobians transform compatibly. It may be an improper prior; invariance does not guarantee posterior propriety. With nuisance parameters, a scalar conditional Jeffreys prior and the joint Jeffreys prior need not coincide.

For a differentiable scale family with finite positive Fisher information, the score is $-[1+u(\log f{)}^{\prime }(u)]/\sigma$. Its squared expectation is a constant times ${\sigma }^{-2}$, yielding the displayed Jeffreys prior. This is an improper prior on the entire positive half-line.

Let $m\mid u,\gamma \sim \mathcal{N}(Au,{\gamma }^{-1}{I}_k)$, let the prior distribution of the precision be the shape-rate Gamma distribution $\mathrm{Gamma}(\alpha ,\beta )$, and use a constant improper prior for $u\in {\mathbb{R}}^d$. Assume $A$ has zero null space. Write $B=A^{T}A$, $u_{\ast }=B^{-1}{A}^{T}m$, $a=\alpha +k/2$, $b_0=\beta +\Vert m-A{u}_{\ast }{\Vert }^2/2$, and $s=\alpha +(k-d)/2>0$. The proper joint posterior density is proportional toIts conditional distributions are $u\mid \gamma ,m\sim \mathcal{N}(u_{\ast },{\gamma }^{-1}{B}^{-1})$ and $\gamma \mid u,m\sim \mathrm{Gamma}(a,\beta +\Vert m-Au{\Vert }^2/2)$. Its marginal precision law is $\mathrm{Gamma}(s,b_0)$. Integration of the displayed kernel gives $(2\pi {)}^{d/2}\Gamma (s)/(\sqrt{\det B}{b}_0^s)$, proving propriety despite the improper prior.

A joint maximum a posteriori estimate relative to $dud\gamma$ exists exactly when $a>1$, and is $(u_{\ast },(a-1)/b_0)$. If $a=1$, the supremum is approached at $\gamma =0$ but not attained; if $a<1$, the density is unbounded there. Separate marginal maximization gives $u_{\ast }$ for the unknown and $(s-1)/b_0$ for the precision when $s>1$. This illustrates that joint and marginal modes need not agree, and that a proper posterior distribution need not have a mode in its open parameter space.

A flat density in $\log \psi$ induces $p(\psi )\propto 1/\psi$. Since $\psi =(1-\varphi )/\varphi$, the absolute Jacobian determinant is $|d\psi /d\varphi |={\varphi }^{-2}$. ThereforeThis is the Haldane prior, an improper prior with formal beta parameters $(0,0)$.

Multiplying the likelihood in part (d) by the Haldane prior gives $p(\varphi \mid y)\propto {\varphi }^{s-1}(1-\varphi {)}^{n-1}$. ConsequentlyFor at least one hospital, $n>0$, this posterior is proper exactly when $s>0$. If every count is zero, the kernel diverges at $\varphi =0$ and does not define a posterior distribution; the improper prior cannot then be used for the subsequent expectation.

A constant improper prior would give the formal posterior density kernel $N^{-1}{1}_{\{N\ge y\}}$. However,There is no normalized posterior distribution under this prior. The failure is not simply that the prior is improper: some improper priors give proper posteriors. Here posterior propriety itself fails after this single observation, so probabilities, quantiles and a posterior mean cannot be assigned to that kernel.

Multiplying the reciprocal improper prior by the likelihood function givesThus this discrete uniform endpoint posterior is proper. Its exact cumulative probability is $S_y^{-1}{\sum }_{k=y}^n{k}^{-2}$. Integral approximation gives $S_y\approx {\int }_y^{\infty }{t}^{-2}dt=1/y$ and ${\sum }_{k=y}^n{k}^{-2}\approx {\int }_y^n{t}^{-2}dt=1/y-1/n$, soThese approximations are useful when $y$ is moderately large and are not exact discrete identities. For example the exact untruncated median at $y=100$ is $199$. Although its quantiles are finite, its posterior mean is infinite, since its expectation numerator is the divergent sum ${\sum }_{N=y}^{\infty }1/N$.

Take the exposure $E_i$ as known and positive. The Poisson distribution log-likelihood, up to a parameter-independent constant, is ${\ell }_i({\lambda }_i)=y_i\log {\lambda }_i-E_i{\lambda }_i$. Its first two derivatives areSince $\mathbb{E}(Y_i)=E_i{\lambda }_i$, the Fisher information isThe Jeffreys prior is proportional to its square root. Removing the constant $\sqrt{E_i}$ givesThis is an improper prior: its integral diverges at infinity.

For $\tau =c\sigma$ with $c>0$, the change of variables givesEquivalently,This is the scale-invariant prior as a measure. Because its integral over $(0,\infty )$ diverges, it is an improper prior, not a normalized probability distribution on that range.

For a regular one-parameter sampling distribution, the Fisher information and Jeffreys prior areUnder the usual differentiation and integrability conditions, $I(\theta )=-{\mathbb{E}}_{\theta }[{\partial }_{\theta }^2\log p_Y(Y\mid \theta )]$. This prior distribution transforms as a density under smooth one-to-one reparameterizations, so the rule is coordinate invariant. Its integral need not be finite; posterior propriety must still be established if it is an improper prior.

The overall mean $\mu$ and the between-study heterogeneity $\tau$ encode different information. A proper broad normal distribution prior such as $\mu \sim N(0,2^2)$ is one possible weak prior for the mean log odds ratio; information about the spread of trials alone does not determine its center.

A concrete prior calibration for normal random-effect range can make the factor-of-50 statement simultaneous across all six trials. Conditional on $\tau$, each pair difference has normal distribution ${\beta }_j-{\beta }_k\sim N(0,2{\tau }^2)$. Set $R=\log 50$, $m=\left(\genfrac{}{}{0ex}{}{6}{2}\right)=15$, $z={\Phi }^{-1}(1-0.05/(2m))$ andFor every $\tau \le A$, each pair exceeds $R$ in absolute value with probability at most $0.05/m$. The union bound therefore gives $\mathbb{P}({\max }_j{\beta }_j-{\min }_j{\beta }_j>R)\le 0.05$, and integrating over the uniform prior preserves that bound. This is one explicit interpretation of “very unlikely”; a different elicited probability would change the bound. A smoother proper scale prior could be calibrated similarly.

The proposed $1/\tau$ improper prior is unsuitable. The observed-data likelihood function approaches the positive common-effect likelihood as $\tau \downarrow 0$. After restricting $\mu$ and the intercepts to a compact interior region, it is bounded below there by a positive constant. HenceThis is an improper posterior from a log-uniform random-effect scale prior. Proper conditional sampling distributions do not repair the improper joint posterior, and an arbitrary tiny cutoff would make inference depend on that cutoff.

Write $\lambda ={\sigma }^2$, $v={\sigma }_0^2$, and $a=\sqrt{\rho }$. We keep the model's printed coefficient ${\sigma }^2{Z}_i$ literally. Its latent log odds can be writtenThe independence of the ${\theta }_i$ is conditional on $Z$: they are not marginally independent when the random effects $Z_i$ are correlated.

Holding the other predictors and the random-effect realization fixed, increasing predictor $j$ by one shifts the latent log odds by ${\beta }_j$, so it multiplies the conditional odds of a late departure by $e^{{\beta }_j}$. For a binary snow indicator, this compares the snow and no-snow days with other predictors held fixed. This is a conditional odds ratio, not generally the same as a marginal odds ratio after integrating the random effects.

Randomizing ${\theta }_i$ allows unmeasured daily conditions to change the success probability beyond the systematic linear predictor. In this logistic-normal regression with autoregressive random effects, let $P_i={\mathrm{logit}}^{-1}({\theta }_i)$ and ${\bar{p}}_i=\mathbb{E}[P_i]$. The law of total variance givesso the mixture allows overdispersion beyond a binomial distribution with fixed probability. The correlated part also allows related outcomes on nearby days.

The stationary autoregressive process of order one has $\mathrm{Var}(Z_i)=1$ and $\mathrm{Cov}(Z_i,Z_j)=a^{|i-j|}$. HenceThe ${\sigma }_0^2$ term is unstructured daily variability, whereas ${\sigma }^4$ is the temporally correlated component under the printed parameterization. Larger $\rho$ gives greater persistence: the lag-$h$ correlation of the latent log odds is ${\sigma }^4{\rho }^{h/2}/({\sigma }^4+{\sigma }_0^2)$.

The PDF itself prints both the coefficient ${\sigma }^2{Z}_i$ and the requested combination ${\sigma }^2+{\sigma }_0^2$. They are inconsistent as a marginal variance. If the intended coefficient were $\sigma Z_i$, the variance and off-diagonal covariance would instead be ${\sigma }^2+{\sigma }_0^2$ and ${\sigma }^2{\rho }^{|i-j|/2}$. The prior's argument list also repeats ${\sigma }^2$; its displayed density indicates the intended second variance parameter is ${\sigma }_0^2$.

There is a further substantive issue with that improper prior. After integrating the latent variables, the observed-data likelihood function is positive and continuous at $\lambda =0$ for fixed finite $\beta$, positive $v$, and $\rho \in (0,1)$. On a compact positive-volume set of those parameters it therefore has a positive lower bound for small $\lambda$. The prior density is $1/(\lambda v)$, so integrating over $\lambda$ gives ${\int }_0^{\epsilon }d\lambda /\lambda =\infty$. Thus the stated joint posterior distribution is improper, an instance of improper posterior from a log-uniform random-effect scale prior. The requested fixed-parameter conditional laws below are nevertheless proper; their existence does not remedy the joint impropriety.

Let $\mathcal{D}=(\{{\hat{m}}_s\},\{z_s\})$. Bayes' theorem and the stated prior factorization give the posterior densityThe normalizing constant, when finite and positive, is the integral of this product over its admissible parameter domain. An improper prior requires checking posterior propriety before interpreting this expression as a probability density.