Solution (source code)

= Solution

Write $Y=g(S)$ and $Z=\min(S,M)$. Their common <expected value> is $c$. Expanding about the retention gives \b[the requested variance identity]
$$
\mathbb E[(Y-M)^2]-(M-c)^2
=\mathbb E[Y^2]-2Mc+M^2-(M^2-2Mc+c^2)
=\boxed{\operatorname{Var}(Y)}.
$$
The <stop loss variance minimization principle> follows from a pointwise comparison. For $0\leq x\leq M$, the constraint $0\leq g(x)\leq x$ implies
$$
|g(x)-M|=M-g(x)\geq M-x
=|\min(x,M)-M|.
$$
For $x>M$, the squared distance of $\min(x,M)=M$ from $M$ is zero, so the same squared-distance comparison is immediate. Therefore
$$
\mathbb E[(Y-M)^2]\geq\mathbb E[(Z-M)^2].
$$
Subtracting the same $(M-c)^2$ proves \b[optimality of the retained stop loss payout]:
$$
\boxed{\operatorname{Var}(g(S))\geq
\operatorname{Var}(\min(S,M)).}
$$
Since $Z$ is bounded, its <variance> is finite; if $\mathbb E[Y^2]=\infty$, the inequality remains valid with infinite <variance> on the left. When it is finite, equality requires $g(S)=\min(S,M)$ almost surely, because the pointwise squared-distance inequality is strict whenever the two payouts differ.