Solution (source code)

= Solution

Use the standard implicit assumption that $p$ is prime, so $\mathbb F_p$ is a <finite field>. The matrices in question form $GL_2(\mathbb F_p)$, the <general linear group over a finite field>. Products stay in the set because <determinants> multiply, the identity has determinant one, and
$$
\begin{pmatrix}a&b\\c&d\end{pmatrix}^{-1}
=(ad-bc)^{-1}\begin{pmatrix}d&-b\\-c&a\end{pmatrix}
$$
has entries in the <finite field> and nonzero <determinant>. Together with associativity this verifies the <group> axioms. The multiplication of matrices with column <vectors> is a <group action> because $I v=v$ and $(gh)v=g(hv)$.

Let $C=\langle g\rangle$ be the order-$p$ <cyclic subgroup>. By the <orbit-stabilizer theorem>, its <group orbits> on $\mathbb F_p^2$ have size either one or $p$. If $f$ is the number of fixed <vectors>, counting all $p^2$ vectors gives $f\equiv p^2\equiv0\pmod p$. Since the zero <vector> is fixed, $f\geq1$ and hence $f\geq p$. There is a nonzero fixed <vector> $v$.

Extend $v$ to a <basis> $(v,w)$. In this basis $g$ has matrix $\begin{pmatrix}1&b\\0&d\end{pmatrix}$. From $g^p=I$ we get $d^p=1$; in the prime <finite field>, $d^p=d$, so $d=1$. Since $g$ has order $p>1$, it is not the identity, and $b\ne0$. Replace $w$ by $b^{-1}w$. Then $g(b^{-1}w)=v+b^{-1}w$, and in the new basis
$$
\boxed{g\sim\begin{pmatrix}1&1\\0&1\end{pmatrix}.}
$$
Thus all the <order-p matrices in GL2 over the prime field> form one <conjugacy class>. This proof also works at $p=2$. Primality is essential to the first assertion: for modulus four, $\operatorname{diag}(2,1)$ has nonzero determinant but no inverse, so the set defined using merely nonzero determinants would not be a <group>.