Expand at a <matrix> increment $H$:
$$
(A+H)^t(A+H)=A^tA+H^tA+A^tH+H^tH.
$$
The last term is $O(\|H\|^2)$ in a <matrix norm>, so it contributes no <derivative>. Hence
$$
\boxed{Ds_A(H)=H^tA+A^tH.}
$$
This is a linear operator on <matrix> increments, not a scalar entrywise <derivative>.

Solution

Past exam of the options I course of the University of Oxford / 2004 / F / F1 / a / Solution

 Solution (source code)