Beta-binomial conjugacy

A beta prior $\mathrm{Beta}(\alpha ,\beta )$ combined with $s$ Bernoulli successes and $f$ failures gives posterior $\mathrm{Beta}(\alpha +s,\beta +f)$.

With a uniform prior distribution on a Bernoulli chance $\theta$, $n$ successes yield posterior density $(n+1){\theta }^n$. Integration gives next-trial success probability $(n+1)/(n+2)$, but the singleton $\theta =1$ retains probability zero. More generally, ordinary conditioning with a finite integrable likelihood and positive finite normalization constant preserves absolute continuity, so a continuous prior does not acquire a new atom at certainty from a finite sample.

For independent geometric random variables with success probability $p$, observing $d$ terminal events and $F$ preceding failed Bernoulli trials gives likelihood function proportional to $p^d(1-p{)}^F$. Multiplication by a beta distribution prior $\mathrm{Beta}(a,b)$ gives the posterior distribution $\mathrm{Beta}(a+d,b+F)$. With complete waiting times $x_1,\dots ,x_n$, take $d=n$ and $F={\sum }_i(x_i-1)$.

A waiting time that is right-censored after $c$ failed trials contributes $(1-p{)}^c$: increase $F$ by $c$ without increasing $d$. This requires the censoring mechanism to be ignorable, as with an externally fixed observation horizon. Counting a censored waiting time as an observed event changes the likelihood function and biases the update.

Mix a binomial distribution with a beta distribution for its success probability: $\Theta \sim \mathrm{Beta}(\alpha ,\beta )$ and $Y\mid \Theta \sim \mathrm{Binomial}(n,\Theta )$. Integrating their densities gives the displayed mass for $y=0,\dots ,n$. Its expectation is $n\alpha /(\alpha +\beta )$, and its variance is $n\alpha \beta (\alpha +\beta +n)/[(\alpha +\beta {)}^2(\alpha +\beta +1)]$. It is the posterior predictive distribution for repeated Bernoulli trials after a beta-prior update.

A uniform prior distribution on the success chance makes the number of successes in any fixed number $n$ of Bernoulli trials uniform on $0,\dots ,n$, because $\left(\genfrac{}{}{0ex}{}{n}{y}\right)\mathrm{B}(y+1,n-y+1)=1/(n+1)$. This is a statement about counts, not a uniform distribution over the $2^n$ ordered outcome sequences. It also makes predictions reversible between equally sized batches: their joint probability distribution is symmetric and their marginal count distributions are the same uniform distribution.

Conditional on the pooled number $s$ of successes in $m+n$ exchangeable random variables that are binary, all placements of those successes are equally likely under a common success-chance mixture. The count in a chosen batch of size $n$ therefore has a hypergeometric distribution, with mass $\left(\genfrac{}{}{0ex}{}{s}{y}\right)\left(\genfrac{}{}{0ex}{}{m+n-s}{n-y}\right)/\left(\genfrac{}{}{0ex}{}{m+n}{n}\right)$. Under the uniform-count predictive property, $s$ is uniform on $0,\dots ,m+n$. Dividing the joint pooled-and-batch probability by the first batch's uniform marginal gives the posterior predictive distribution, with factor $(m+1)/(m+n+1)$. Substituting $s=x+y$ in the hypergeometric expression does not give a single normalized hypergeometric mass as $y$ varies, since its population success total then varies too.

Draw $P\sim \mathrm{Beta}(\alpha ,\beta )$, $X\mid P\sim \mathrm{Binomial}(n,P)$ and $Q\mid X\sim \mathrm{Beta}(\alpha +X,\beta +n-X)$, with $\alpha ,\beta >0$ and integer $n\ge 0$. The joint density of $(P,X,Q)$ is symmetric in $P,Q$, making them exchangeable random variables with identical beta distribution marginal distributions. The posterior beta normalization constant depends on $X$ and must be retained. The correlation coefficient is $n/(n+\alpha +\beta )$: this constructs tunable dependence without altering either marginal.