Solution (source code)

= Solution

Put $\mu_J=GM_J$. At the <periapsis distance>, the radial <velocity> vanishes. Insert $E=mv_0^2/2$ and $J=mbv_0$ into <11/a>:
$$
\frac12v_0^2=\frac{b^2v_0^2}{2r_{\min}^2}-\frac{\mu_J}{r_{\min}},
\qquad v_0^2r_{\min}^2+2\mu_Jr_{\min}-b^2v_0^2=0.
$$
The positive root is
$$
\boxed{r_{\min}=\frac{\sqrt{\mu_J^2+b^2v_0^4}-\mu_J}{v_0^2}
=\sqrt{b^2+\left(\frac{\mu_J}{v_0^2}\right)^2}-\frac{\mu_J}{v_0^2}.}
$$
Rationalizing gives the numerically stable alternative $r_{\min}=b^2v_0^2/[\sqrt{\mu_J^2+b^2v_0^4}+\mu_J]$. Attraction makes $r_{\min}<b$. This is a point-planet <scattering> calculation; a noncolliding physical encounter must also have $r_{\min}$ outside Jupiter's surface. At $b=0$, the limiting formula gives a radial collision, not a flyby.