Solution (source code)

= Solution

First solve the supplied equations as a linear system, with $\omega_0^2=g/l$ and $\alpha>0$. A <normal mode> $\mathbf x=\mathbf a e^{i\omega t}$ obeys
$$
\frac{\omega^2}{\omega_0^2}\mathbf a=D\mathbf a,\qquad
D=\begin{pmatrix}1+2\alpha&-\alpha\\-\alpha&\alpha\end{pmatrix}.
$$
Its <characteristic polynomial> is $\lambda^2-(1+3\alpha)\lambda+\alpha(1+\alpha)$. Hence
$$
\boxed{\omega_\pm^2=\frac g{2l}\left[1+3\alpha\pm\sqrt{1+2\alpha+5\alpha^2}\right].}
$$
Both are positive, since the <trace> and <determinant> of $D$ are positive.

There is a physical discrepancy in the source. The <horizontal-coordinate mass matrix of an unequal-mass double pendulum> follows from the stated horizontal <displacements>, which give
$$
T_2=\frac m2\dot x_1^2+\frac{\alpha m}{2}\dot x_2^2,\qquad
V_2=\frac{mg}{2l}\left[x_1^2+\alpha x_1^2+\alpha(x_2-x_1)^2\right].
$$
The <Euler-Lagrange equations> reproduce the supplied upper equation, but give $\ddot x_2=-(g/l)(x_2-x_1)$, without its extra $\alpha$. The true <normal modes of a double pendulum> consequently have $\omega_\pm^2=(g/l)[1+\alpha\pm\sqrt{\alpha(1+\alpha)}]$. The boxed answer and the remaining parts below concern the explicitly supplied linear system. The two models agree at $\alpha=1$, but not for a general mass ratio.