Affine null cone (${\mathcal{N}}_G(V)=\{v:0\in \overline{Gv}\}$)

The affine null cone is the set of vectors whose orbit closure contains the origin. For a reductive algebraic group it is also the common zero locus of all positive-degree homogeneous invariants. A nonzero vector in this set represents an unstable projective point. The Hilbert-Mumford criterion for the affine null cone detects membership by a algebraic one-parameter subgroup with only positive active weights.

For conjugation of $n$ by $n$ complex matrices by the special linear group, the affine null cone consists exactly of the nilpotent matrices. If a conjugate tends to zero along an algebraic one-parameter subgroup, its constant characteristic polynomial must be $T^n$. Conversely, put a nilpotent matrix in strictly upper triangular form and conjugate it by $\mathrm{diag}(t^{n-1},t^{n-3},\dots ,t^{1-n})$. Every active matrix entry has positive weight and tends to zero. The Hilbert-Mumford criterion for projective stability therefore identifies the semistable locus of the projective conjugation representation as the complement of the projectivized nilpotent cone.