Solution (source code)

= Solution

Substituting part (b) gives
$$
\widehat g_N-g'(x_0)
=\frac{h}{2N}\sum_{i=1}^NZ_i g''(\xi_i)
+\frac1{Nh}\sum_{i=1}^NZ_i\varepsilon_i
=A+B.
$$
The curvature bound gives $|A|\leq hM/2$. Conditional on the $Z_i$ and the evaluation points, $B$ has mean zero, so the cross term has expectation zero. Independence of the <normal random variables> $\varepsilon_i$ gives
$$
\mathbb E B^2=\frac1{N^2h^2}\sum_{i=1}^N\mathbb E\varepsilon_i^2
=\frac{\sigma^2}{Nh^2}.
$$
Consequently
$$
\boxed{\mathbb E|\widehat g_N-g'(x_0)|^2
\leq\frac{h^2M^2}{4}+\frac{\sigma^2}{Nh^2}}.
$$

The two terms are balanced by
$$
h_N^2=\frac{2\sigma}{M\sqrt N},
\qquad
h_N=\left(\frac{2\sigma}{M}\right)^{1/2}N^{-1/4}.
$$
At this value both terms equal $\sigma M/(2\sqrt N)$, and hence
$$
\boxed{\mathbb E|\widehat g_N-g'(x_0)|^2\leq\frac{\sigma M}{\sqrt N}}.
$$