Solution
= Solution
The <variation-of-constants formula> gives, when $a\ne0$,
$$
X_t=e^{-at}x+\frac ba(1-e^{-at})
+\sigma\int_0^te^{-a(t-u)}dB_u.
$$
Therefore, with $m=\min(s,t)$,
$$
\begin{aligned}
\operatorname{cov}(X_t,X_s)
&=\sigma^2\int_0^me^{-a(t-u)}e^{-a(s-u)}du\\
&=\boxed{\frac{\sigma^2}{2a}
\left(e^{-a|t-s|}-e^{-a(t+s)}\right)}.
\end{aligned}
$$
If $a=0$, then $X_t=x+bt+\sigma B_t$ and
$$
\boxed{\operatorname{cov}(X_t,X_s)=\sigma^2\min(t,s).}
$$