 Solution

ID: past-exam-of-the-advanced-topics-in-statistical-machine-learning-course-of-the-university-of-oxford/2025/2/c/iii/solution

For

h

H_{k}

\frac{d}{d ϵ} L (g + ϵ h)_{ϵ = 0} = \sum_{i} (g (x_{i}) - y_{i}) h (x_{i}) .

h (x_{i}) = ⟨ k (x_{i}, \cdot), h ⟩_{H_{k}}

. Hence

\frac{d}{d ϵ} L (g + ϵ h)_{ϵ = 0} = ⟨ \sum_{i} (g (x_{i}) - y_{i}) k (x_{i}, \cdot), h ⟩_{H_{k}} .

Each kernel section belongs to the Reproducing kernel Hilbert space, so this finite sum is an admissible element. Uniqueness of a vector with these inner products gives the functional gradient

\nabla_{g} L (g) = i = 1 \sum n (g (x_{i}) - y_{i}) k (x_{i}, \cdot) .

 New to topics? Read the docs here!