Past exam of the computer science course of the University of Oxford / 2015 / Computer security / 1 / d / i / Solution

Avoid constructing the huge integers $c^{3^j}$ explicitly. Maintain $a_0=c$ and $a_j=a_{j-1}^3\mathrm{mod}n$, computing $g_j=\gcd (a_j-c,n)$ after each step.

In case (1), the nontrivial greatest common divisor $g_k$ is one of the prime factors $p,q$. Divide $n$ by it, compute $d$ with the extended Euclidean algorithm, and return $c^d\mathrm{mod}n$.

In case (2), $a_k=c(\mathrm{mod}n)$. The preceding value satisfies $a_{k-1}^3=c(\mathrm{mod}n)$. Cubing is a bijection on the residue classes modulo $n$, so there is only one such cube root, namely the original plaintext $m$. ThereforeFor $k=1$, the preceding value is $a_0=c$; a fixed point is its own plaintext. Keeping the previous iterate makes the recovery immediate once the RSA cycling attack succeeds.