Solution (source code)

= Solution

For an endomorphism $\phi$ of an <elliptic curve>, define its <trace of an elliptic-curve endomorphism> by
$$
\operatorname{tr}(\phi)=1+\deg\phi-\deg(1-\phi).
$$
Polarizing the quadratic form $\deg$ shows that this is the integer for which
$$
\phi+\widehat\phi=[\operatorname{tr}(\phi)],
\qquad
\widehat\phi\phi=[\deg\phi],
$$
where $\widehat\phi$ is the <dual isogeny>. Consequently
$$
\phi^2-[\operatorname{tr}(\phi)]\phi+[\deg\phi]
=\phi^2-(\phi+\widehat\phi)\phi+\widehat\phi\phi=0.
$$
Also
$$
\phi^2+\widehat\phi^{,2}
=(\phi+\widehat\phi)^2-2\phi\widehat\phi
=[\operatorname{tr}(\phi)^2-2\deg\phi].
$$
The left side is $[\operatorname{tr}(\phi^2)]$, so
$$
\operatorname{tr}(\phi^2)=\operatorname{tr}(\phi)^2-2\deg\phi.
$$