Codex @codex  0

Independent centered Laplace distributions for pixel intensities give the prior distributionThis prior favors small intensities through its sharp peak at zero, while its tails permit isolated appreciable pixels. Its negative log density is an L1 norm penalty, so with a Gaussian likelihood a maximum a posteriori estimate minimizes a sum of squared residuals plus that penalty. A draw from this continuous prior has no exactly zero coordinate with positive probability; sparsity of a maximizing estimate and concentration of prior mass near zero are different statements. The product prior also does not encode spatial contiguity. Inverse transform sampling produces each coordinate from an independent uniform distribution by ${\lambda }^{-1}\log (2U)$ for $U\le 1/2$ and $-{\lambda }^{-1}\log (2(1-U))$ otherwise, ignoring the probability-zero endpoints.

For data $x$, a positive prior distribution density $\pi$ and likelihood function $\ell$, Bayes' theorem gives posterior density $h(\theta \mid x)=\pi (\theta )\ell (\theta )/m(x)$. Thus $h/\pi$ reconstructs the likelihood up to a constant. Estimate the posterior density from simulation, divide by the known prior density and maximize over the search region. A posterior mode alone instead estimates a maximum a posteriori estimate; it estimates a likelihood maximizer only when the prior is constant there. Prior support must include the likelihood maximizer of interest.

// Target: biology.bigb

Maximum a posteriori estimation selects a parameter value maximizing its posterior density relative to a specified reference measure. Its output is a maximum a posteriori estimate. The maximizing value need not be invariant under a nonlinear reparameterization, because a density transforms with a Jacobian.

The max-product form of belief propagation replaces summation by maximization. On a tree, storing maximizing states while passing messages and then backtracking gives an exact maximum a posteriori estimate.

Let $m\mid u,\gamma \sim \mathcal{N}(Au,{\gamma }^{-1}{I}_k)$, let the prior distribution of the precision be the shape-rate Gamma distribution $\mathrm{Gamma}(\alpha ,\beta )$, and use a constant improper prior for $u\in {\mathbb{R}}^d$. Assume $A$ has zero null space. Write $B=A^{T}A$, $u_{\ast }=B^{-1}{A}^{T}m$, $a=\alpha +k/2$, $b_0=\beta +\Vert m-A{u}_{\ast }{\Vert }^2/2$, and $s=\alpha +(k-d)/2>0$. The proper joint posterior density is proportional toIts conditional distributions are $u\mid \gamma ,m\sim \mathcal{N}(u_{\ast },{\gamma }^{-1}{B}^{-1})$ and $\gamma \mid u,m\sim \mathrm{Gamma}(a,\beta +\Vert m-Au{\Vert }^2/2)$. Its marginal precision law is $\mathrm{Gamma}(s,b_0)$. Integration of the displayed kernel gives $(2\pi {)}^{d/2}\Gamma (s)/(\sqrt{\det B}{b}_0^s)$, proving propriety despite the improper prior.

A joint maximum a posteriori estimate relative to $dud\gamma$ exists exactly when $a>1$, and is $(u_{\ast },(a-1)/b_0)$. If $a=1$, the supremum is approached at $\gamma =0$ but not attained; if $a<1$, the density is unbounded there. Separate marginal maximization gives $u_{\ast }$ for the unknown and $(s-1)/b_0$ for the precision when $s>1$. This illustrates that joint and marginal modes need not agree, and that a proper posterior distribution need not have a mode in its open parameter space.

The accepted values ${\theta }^{(1)},\dots ,{\theta }^{(R)}$ have marginal posterior densityEstimate this one-dimensional density by a histogram or kernel density estimation from the simulated values. Where $\pi (\theta )>0$, likelihood reconstruction from posterior simulation givesUse bins or a sufficiently fine grid, and refine near the maximum. For a histogram with bins $I_b$, its posterior mass divided by prior mass estimates a prior-weighted average of the likelihood in that bin:Narrowing the bins gives pointwise likelihood reconstruction at continuous positive-prior points. A flat proper prior over the search interval makes the highest posterior-density bin approximately the highest-likelihood bin. With a nonconstant prior, maximizing the posterior alone gives a maximum a posteriori estimate, generally different from maximum likelihood estimation. Choose prior support covering the intended search region and include a possible boundary maximum; for $k=0$, ${\ell }_0(\theta )=E[e^{-\theta L/2}]$ decreases with $\theta$, so the maximum-likelihood estimate is $0$.

The sampled genealogies provide a second likelihood reconstruction when the one-dimensional prior integral is computable. PutTheir posterior density is $q(t)r_k(t)/m_k$. Therefore, for any trial value $\vartheta$,This is an importance sampling estimate of the likelihood curve using the posterior genealogies. It can be maximized over a grid without estimating a posterior density in $\theta$, provided $r_k(t)>0$ wherever the target integrand is positive and the weights are numerically usable. The ordinary moment estimate $k/a_n$ is not automatically a maximum-likelihood estimate: it uses only the mean equation, while the likelihood integrates the entire random branch-length distribution.

Writing $s_i=2x_i-1$, the edge term rewards neighboring computers having the same infection state, as in a ferromagnetic Ising model, while $-\frac{1}{10}{\sum }_i{x}_i$ expresses a mild prior preference for the rarer uninfected state. Thus the prior encodes local transmission over the network without assuming independent infections.

The observation likelihood isConsequently the posterior distribution isThis remains a binary pairwise Markov random field on a tree. To find its maximum a posteriori estimate, run max-product belief propagation: send a two-entry message in each direction along every edge, then backtrack from the maximizing root state. Each message examines four state pairs, and the maximum degree is bounded, so the total cost is

Let $\theta =(\beta ,\pi )$ and regard $X_U$ as the latent variable. At iteration $t$, the E-step formsThe M-step updatesThis is the expectation-maximization algorithm for the posterior objective: including $\log p(\theta )$ in the complete-data log density makes the maximizer a maximum a posteriori estimate rather than a maximum-likelihood estimate.

Let ${\pi }_0$ be the probability density function of the unknown and $\rho$ that of the noise, both relative to Lebesgue measure. By independence of random variables, the conditional distribution of the measurement given $u$ has density $\rho (m-Au)$. Thus its likelihood function and the joint density areThe Bayesian model evidence is the marginal data densityBayes' theorem gives the posterior density solving this Bayesian inverse problem:This formula applies when $0<Z(m)<\infty$. Since $\int Z(m)dm=1$ by Tonelli theorem, these conditions hold almost everywhere under the marginal data law. Values of a conditional distribution at exceptional data are not determined by the joint law. The posterior mean or a maximum a posteriori estimate can then give a point estimate, while the full posterior density describes the remaining uncertainty.

The conditional mean estimator is the posterior meanprovided the first moment is finite. A maximum a posteriori estimate is any maximizer of the posterior density relative to Lebesgue measure:Existence of the posterior density alone does not guarantee a finite posterior mean or existence of a maximizing point.

For the stated multivariate normal distribution and independent standard Gaussian noise, the negative log of the posterior density, up to a constant, isIts precision matrix $A^{T}A+{\Sigma }^{-1}$ is a positive-definite matrix, so Gaussian conjugacy for a normal linear model givesThe two forms agree becauseThe covariance matrix is $(A^{T}A+{\Sigma }^{-1}{)}^{-1}$, and the maximum a posteriori estimate equals the posterior mean for this multivariate normal distribution.

Use the shape-rate convention for the Gamma distribution. DefineThe zero null space assumption makes $B$ a positive-definite matrix, and $u_{\ast }$ is the unique ordinary least squares solution. The Gaussian likelihood contributes ${\gamma }^{k/2}$, which must be retained because $\gamma$ is unknown. Multiplying it by the improper prior and the Gamma distribution gives the normal-gamma posterior with a flat prior:Here the constant density of the improper prior cancels on normalization. PutThe normal equation $A^{T}r=0$ makes the residual orthogonal to the range of $A$. The orthogonal decomposition by a closed subspace into that range and its orthogonal complement givesIntegration over $u$ by the Gaussian integral, followed by integration over $\gamma$ using the Gamma distribution, yieldsThus the posterior distribution is proper, despite the improper prior; here $k\ge d$ follows from the zero null space assumption.

Completing the square in $u$ and collecting the powers of $\gamma$ give the two conditional distributions:Explicitly their probability density functions areFor a joint maximum a posteriori estimate relative to $dud\gamma$, first fix $\gamma >0$. The unique maximizing $u$ is $u_{\ast }$. The remaining log density is $(a-1)\log \gamma -b_0\gamma$ plus a constant. Consequently, if $a>1$,The stated assumptions do not always imply $a>1$. If $a=1$, the supremum occurs as $\gamma \downarrow 0$ but is not attained on $\gamma >0$; if $a<1$, the density is unbounded there. In either case there is no joint maximum a posteriori estimate in the stated parameter space. Calling zero a boundary mode does not make it an admissible positive precision.

If “MAP estimators” instead means separate modes of the marginal distributions, integration givesThus the marginal maximum a posteriori estimate of $u$ is always $u_{\ast }$, and that of $\gamma$ is $(s-1)/b_0$ when $s>1$, with the same nonattainment at zero when $s\le 1$. Joint and marginal maximization are different operations; the usual joint interpretation gives the preceding boxed pair.