Solution (source code)

= Solution

For $n=1$, the singular set is the point $\{0\}$, a zero-dimensional <submanifold>. For $n\geq2$, every elementary matrix $E_{ij}$ is singular and the smooth curve $t\mapsto tE_{ij}$ lies in $X$ through zero. If $X$ were a <submanifold>, all $E_{ij}$ would belong to $T_0X$ and span $M(n)$. Its dimension at zero would therefore be $n^2$, making $X$ locally open there. But arbitrarily small matrices $tI$ are invertible, a contradiction. Thus
$$
\boxed{X\text{ is a submanifold exactly when }n=1.}
$$
The failure is at the zero matrix; fixed-rank pieces of the singular set can separately be smooth <submanifolds>.