For conjugation of by complex matrices by the special linear group, the affine null cone consists exactly of the nilpotent matrices. If a conjugate tends to zero along an algebraic one-parameter subgroup, its constant characteristic polynomial must be . Conversely, put a nilpotent matrix in strictly upper triangular form and conjugate it by . Every active matrix entry has positive weight and tends to zero. The Hilbert-Mumford criterion for projective stability therefore identifies the semistable locus of the projective conjugation representation as the complement of the projectivized nilpotent cone.
Use the natural linearization of a line bundle on induced by the conjugation representation on . The notation involving the dual does not change this matrix description: the nondegenerate pairing identifies the representation with its dual equivariantly. Let be a projective point, so .
The Hilbert-Mumford criterion for projective stability says that is not a semistable projective point in geometric invariant theory precisely when some algebraic one-parameter subgroup sends its affine representative to zero:
Equivalently, all active weights for that subgroup are strictly positive. With the convention , semistability requires for every subgroup, including conjugates of diagonal ones.
Suppose such a zero limit exists. The coefficients in
are invariant under conjugation. Continuity along the zero limit gives for every . By the Cayley-Hamilton theorem, : the matrix is nilpotent.
Conversely, if is a nonzero nilpotent matrix, choose a basis in which it is strictly upper triangular, for example a Jordan normal form. The change of basis can be chosen in : multiply any invertible change-of-basis matrix by a scalar with the required th power, which does not affect its conjugation. In this basis take
The exponents sum to zero, so this is an algebraic one-parameter subgroup of the special linear group. The matrix entry has weight
Every potentially nonzero entry has and therefore positive weight. Thus the conjugated matrix tends to zero. Conjugate this subgroup back to the original basis to obtain the required destabilizing subgroup for .
It follows that the nilpotent cone for conjugation of matrices is exactly the affine null cone, and
Each is a homogeneous invariant of degree , so the second description also gives invariant open sets witnessing semistability. Equivalently, a representative has at least one nonzero eigenvalue. Nonzero singular matrices can be semistable; invertibility is not the criterion. For example, a diagonal matrix with diagonal entries is semistable, while a nonzero strictly upper triangular matrix is unstable.
Using the printed convention that the projective points represent lines of binary quartics, the locus of GIT stable points is
The root at infinity is included. We determine stability by calculating every relevant algebraic one-parameter subgroup weight.
Use the natural action . Every nontrivial algebraic one-parameter subgroup of is conjugate to for some integer : its two integral weights of a representation sum to zero, and a basis of weight vectors can be rescaled to have determinant one. Write
If , then is the multiplicity of a root at the projective point : in the local coordinate , the polynomial begins with a nonzero multiple of .
The Hilbert-Mumford criterion for projective stability, with (active weights), now gives
Being a GIT stable point requires this number to be strictly positive for every nontrivial subgroup. Conjugating the subgroup lets range over the entire projective line. Thus the root-multiplicity criterion for stable binary forms becomes at every point . A nonzero degree-four form over has four projective roots counted with multiplicity, so this means precisely four distinct roots. This is equivalently nonvanishing of its discriminant.
The boundary cases show why a weak inequality would give the wrong answer. A triple or quadruple root can be moved to , leaving only strictly positive weights; its affine representative tends to zero under that subgroup, so it is unstable. If all root multiplicities are at most two, all Hilbert-Mumford values are nonnegative and the point is a GIT semistable point. A double root gives a zero value for a suitable subgroup, so never represents a GIT stable point. More concretely, after putting that double root at one has
and . When there are three distinct roots, this limiting form, representing a GIT semistable point with two distinct roots, is outside the original group orbit, so that group orbit is not closed in the semistable locus. When there are two double roots, a change of coordinates gives , which has the positive-dimensional diagonal torus as stabilizer. Neither case represents a GIT stable point.
For a form with four distinct roots, the projective stabilizer is finite: it permutes the four roots, and a Möbius transformation fixing three distinct points is the identity. Thus its image in injects into the finite permutation group on the roots. The kernel of a group homomorphism for is , also finite. This agrees with the finite-stabilizer condition in geometric invariant theory. The weight calculation already supplies the required strict stability condition for every conjugate subgroup.