Solution (source code)

= Solution

Taking downward distance as positive, the <Lagrangian> is
$$
\boxed{
L
=\frac12(3m)\dot z_1^2+\frac12(2m)\dot z_2^2
-\frac k2(z_1-l)^2
-\frac k2(z_2-z_1-l)^2
+3mgz_1+2mgz_2
}.
$$

At equilibrium the lower spring supports only the lower mass, while the upper spring supports both masses:
$$
k(z_2-z_1-l)=2mg,
\qquad
k(z_1-l)=5mg.
$$
Hence
$$
\boxed{
z_1^{(0)}=l+\frac{5mg}{k},
\qquad
z_2^{(0)}=2l+\frac{7mg}{k}
}.
$$

Let $q_i=z_i-z_i^{(0)}$. The linear terms cancel by the equilibrium equations, and the quadratic Lagrangian is
$$
L
=\frac12\dot{\mathbf q}^{\,T}T\dot{\mathbf q}
-\frac12\mathbf q^TV\mathbf q+\text{constant},
$$
with the <small-oscillation mass and stiffness matrices>
$$
\boxed{
T=m\begin{pmatrix}3&0\\0&2\end{pmatrix},
\qquad
V=k\begin{pmatrix}2&-1\\-1&1\end{pmatrix}
}.
$$

The <generalized eigenvalue problem for small oscillations> is
$$
(V-\omega^2T)\mathbf a=0.
$$
Its characteristic equation is
$$
\det(V-\omega^2T)
=k^2-7km\omega^2+6m^2\omega^4=0,
$$
so
$$
\boxed{
\omega_1=\sqrt{\frac{k}{6m}},
\qquad
\omega_2=\sqrt{\frac{k}{m}}
}.
$$
Corresponding displacement eigenvectors are
$$
\boxed{
\mathbf a_1=(2,3)^T,
\qquad
\mathbf a_2=(1,-1)^T
}.
$$

They are not orthogonal in the ordinary Euclidean inner product. They are orthogonal in the kinetic-energy, or mass-matrix, inner product:
$$
\boxed{
\mathbf a_1^TT\mathbf a_2
=(2,3)
m\begin{pmatrix}3&0\\0&2\end{pmatrix}
\binom{1}{-1}
=0
}.
$$
This is <mass-matrix orthogonality of normal modes>.