= Solution
Evaluating the negative <functional gradient> at $x_j$ yields
$$
\boxed{\frac{dg_t(x_j)}{dt}=\sum_{i=1}^nk(x_j,x_i)(y_i-g_t(x_i))}.
$$
Thus the evaluation <vector> $v(t)=(g_t(x_1),\ldots,g_t(x_n))^T$ follows
$$
\boxed{\dot v=K(y-v),\qquad K_{ij}=k(x_i,x_j)=H^*_{ij}}.
$$
This is exactly the limiting prediction dynamics in part (ii). If $v(0)=u(0)$, subtracting the two systems gives $\frac d{dt}(v-u)=-H^*(v-u)$ with zero initial value, so $v(t)=u(t)$ for every $t$.
\b[An infinitely wide <neural network> in this regime follows the same training-prediction dynamics as <kernel regression> with its <neural tangent kernel> and matching initial predictions.] For this squared-error <loss function>, the relevant algorithm is <gradient flow> in the <Reproducing kernel Hilbert space>. Matching predictions at arbitrary additional inputs also requires the corresponding cross-input <neural tangent kernel> to freeze and the initial predictions there to match. A finite-width <neural network> can change its <feature map> during training, so the equivalence depends on the assumptions rather than holding for every architecture and parameterization.
Back to article page
