 Solution

For $n=1$, the singular set is the point $\{0\}$, a zero-dimensional submanifold. For $n\ge 2$, every elementary matrix $E_{ij}$ is singular and the smooth curve $t\mapsto t{E}_{ij}$ lies in $X$ through zero. If $X$ were a submanifold, all $E_{ij}$ would belong to $T_{0}X$ and span $M(n)$. Its dimension at zero would therefore be $n^2$, making $X$ locally open there. But arbitrarily small matrices $tI$ are invertible, a contradiction. ThusThe failure is at the zero matrix; fixed-rank pieces of the singular set can separately be smooth submanifolds.