 Information projection on three outcomes

Let the possible numerical outcomes be $-1,0,1$, and give the reference law $p$ uniform distribution on a finite set: $p=(1/3,1/3,1/3)$. We want a law $q$ with expected value $1/2$. Write the three probabilities in outcome order as $(q_{-},q_0,q_{+})$. The mean constraint is $q_{+}-q_{-}=1/2$, while normalization requires $q_{-}+q_0+q_{+}=1$.

These conditions leave a whole line of possible probability distributions:For example, $(0,1/2,1/2)$ and $(1/4,0,3/4)$ both have the required mean. The constraint alone does not choose between them. Information projection selects the point on this line with the smallest relative entropy from the reference.

Exponential tilting gives weights proportional to $e^{-\theta },1,e^{\theta }$. Set $t=e^{\theta }>0$ and multiply these weights by $t$ to obtain $q^{\ast }=(1,t,t^2)/(1+t+t^2)$. Enforcing the mean givesThe positive root is $t=(1+\sqrt{13})/2$, henceThe most favored outcome becomes $1$, but all three remain possible. This is an exact constructed example with a stipulated reference law and constraint, not a fitted distribution from measurements.