Solution (source code)

= Solution

The <characteristic polynomial> of $M$ is
$$
\det(\lambda I-M)=(\lambda-6)(\lambda-3)+2=(\lambda-4)(\lambda-5).
$$
The corresponding <eigenvectors> may be chosen as $v_4=(1,2)^T$ and $v_5=(1,1)^T$. They are linearly independent, so write $\boldsymbol x=q_4(t)v_4+q_5(t)v_5$. Substitution in the system of <ordinary differential equations> gives
$$
(q_4''+4q_4)v_4+(q_5''+5q_5)v_5=0.
$$
Independence of the <eigenvectors> yields two decoupled <harmonic oscillator equations>. Their four integration constants give the general real solution
$$
\boxed{\boldsymbol x(t)=
\begin{pmatrix}1\\2\end{pmatrix}(A\cos2t+B\sin2t)
+\begin{pmatrix}1\\1\end{pmatrix}(C\cos(\sqrt5t)+D\sin(\sqrt5t)).}
$$
Both components can be differentiated directly to verify $\boldsymbol x''+M\boldsymbol x=0$. Orthogonality of the <eigenvectors> is unnecessary: diagonalizing this nonsymmetric <matrix> requires an invertible <eigenbasis>, not an orthonormal one.