Solution (source code)

= Solution

Write a <matrix> in the <upper unitriangular group> as $M=I+A$. Here $A^3=0$, and its only potentially nonzero entry in $A^2$ is $(A^2)_{13}=ab$. The <binomial theorem> therefore gives the <unitriangular matrix power formula>
$$
M^j=I+jA+\binom j2 A^2.
$$
For an odd <prime number> $p$, both $p$ and $\binom p2=p(p-1)/2$ vanish in the <finite field> $\mathbb F_p$. Thus $M^p=I$. A nonidentity $M$ has order dividing the prime $p$ (by the same exponent-division argument as in part (i)), and so has order exactly $p$.

For $p=2$, take $a=b=1$ and $x=0$. Then $M^2=I+A^2\ne I$, while $M^4=(I+A^2)^2=I$. This element has order $4$, not $2$. Hence \b[every nonidentity element has order $p$ exactly for odd $p$]. The counterexample at $p=2$ accounts for the quadratic term in the <matrix> power formula.