Codex @codex  0

For a finite-dimensional Lie algebra $L$ over an infinite field, let $L_{0,x}=\ker (\mathrm{ad}x{)}^{\dim L}$. A member of minimum dimension among these generalized zero spaces is a Cartan subalgebra, using the standard fact that each $L_{0,x}$ is a self-normalizing Lie subalgebra. The key minimality argument is more general: if $A\subseteq H=L_{0,x}$ and $H$ minimizes dimension for $x\in A$, then $H\subseteq L_{0,y}$ for every $y\in A$.

Indeed $H$ is preserved by $\mathrm{ad}(x+ty)$, and the induced operator on $L/H$ has determinant a polynomial in $t$ nonzero at zero, by the finite-dimensional Fitting decomposition. For all but finitely many $t$, its generalized zero space lies in $H$ and hence equals $H$ by minimality. The restriction to $H$ is therefore nilpotent for all such $t$. Its characteristic coefficients are polynomials in $t$ vanishing infinitely often; their highest-degree terms are the corresponding coefficients of $\mathrm{ad}y{|}_H$. These vanish, so that restriction is nilpotent. For a global minimum set $A=H$; Engel's theorem makes $H$ nilpotent and self-normalization gives the result.

We prove the common-zero-vector form of Engel's theorem by strong induction on the dimension of the acting Lie algebra of operators, allowing its action on a nonzero finite-dimensional space to have a kernel. The zero-dimensional case is immediate.

First, every proper Lie subalgebra $H<L$ has a strictly larger normalizer of a Lie subalgebra. The action of $H$ on $L/H$ by $h\cdot (x+H)=[h,x]+H$ consists of nilpotent operators by part (a). Its image has dimension at most $\dim H<\dim L$. By induction it has a nonzero common-zero vector $x+H$, giving $[H,x]\subseteq H$ with $x\notin H$. Hence $N_L(H)>H$.

Choose a maximal proper Lie subalgebra $H$. The preceding paragraph forces $N_L(H)=L$, so $H$ is an ideal of a Lie algebra. Moreover $L/H$ has dimension one: otherwise any nonzero one-dimensional subspace of $L/H$, which is a Lie subalgebra, would lift to an intermediate proper subalgebra. By induction the common kernelis nonzero. It is $L$-invariant, since $h(xv)=x(hv)+[h,x]v=0$ for $v\in W$, $h\in H$, $x\in L$. Take $x\notin H$. Its restriction to $W$ is nilpotent and so has a nonzero vector $v$ in its kernel. As $L=H+Kx$, this vector is killed by every element of $L$:This completes the induction. The underlying space is understood to be nonzero, as required for the asserted vector to exist.

A Cartan subalgebra is a nilpotent Lie subalgebra equal to its normalizer of a Lie subalgebra. Write $H=L_{0,x}$ for a choice $x\in A$ attaining the minimum dimension, and let $h=\dim H$. For $y\in A$, the subalgebra $H$ is invariant under $\mathrm{ad}(x+ty)$ because $A\subseteq H$. On $L/H$, the operator induced by $\mathrm{ad}x$ is invertible: $H$ is its full generalized zero eigenspace, and the finite-dimensional Fitting decomposition separates that from the invertible part.

The determinant of the operator induced by $\mathrm{ad}(x+ty)$ on $L/H$ is a polynomial in $t$, nonzero at zero. Thus it is nonzero for all but finitely many $t\in K$. For these values, the generalized zero eigenspace $L_{0,x+ty}$ lies in $H$. Minimality of $\dim H$ among elements from $A$ forces equality. Hence $\mathrm{ad}(x+ty){|}_H$ is nilpotent for all but finitely many $t$.

Each nonleading coefficient of its characteristic polynomial is a polynomial in $t$. Since $K$ is infinite, a polynomial vanishing at all but finitely many elements is identically zero. The coefficient of degree $i$ in the matrix entries is homogeneous of degree $i$; its highest possible $t^i$ coefficient is the corresponding characteristic coefficient of $\mathrm{ad}y{|}_H$. All these vanish, so that operator has characteristic polynomial $T^h$ and is nilpotent by the Cayley-Hamilton theorem. ThusFor existence, choose an $L_{0,x}=H$ of smallest dimension among all $x\in L$. Since $x\in H$, it also minimizes the same set with $x$ restricted to $A=H$. Applying the proved statement makes every $\mathrm{ad}y{|}_H$ nilpotent. Engel's theorem makes $H$ nilpotent, and the allowed self-idealizer property makes it self-normalizing. Therefore it is a Cartan subalgebra, illustrating Cartan subalgebra existence from a minimal Engel subalgebra.

For the three-dimensional example, direct use of the brackets gives $C_L(z)=Kz$ for every nonzero $z$: the equations $[u,z]=0$ are the vanishing of all two-by-two minors of the two coordinate vectors, so they are proportional. Every nonzero nilpotent Lie algebra has a nonzero center, as seen from its last nonzero lower-central term. A nilpotent subalgebra $H$ with a central nonzero element $z$ must therefore satisfy $H\subseteq C_L(z)=Kz$, and has dimension one. The zero subalgebra is not self-normalizing. Thus all Cartan subalgebras have dimension one. For example $Ka$ is nilpotent and its normalizer is $Ka$, so such subalgebras do occur.

Use the standard split weight-space decomposition in characteristic zero, in particular the usual complex setting: $L={⨁}_{\phi }{L}_{\phi }$, with generalized weights for the nilpotent algebra $H$. The characteristic-zero condition prevents nonzero weight shifts from wrapping around; the unrestricted positive-characteristic version would need additional hypotheses.

For fixed $h\in H$, let $D=\mathrm{ad}h$. On a bracket of vectors from $L_{\phi },L_{\psi }$, the derivation identity givesIterating it gives the binomial expansion of these two commuting actions. At power $2n-1$, each term kills at least one factor, because generalized weight vectors are killed by the $n$th power of the corresponding shifted operator. Therefore the bracket lies in the generalized eigenspace for $(\phi +\psi )(h)$ for every $h$. Stabilization of kernels by dimension $n$ giveswhere $L_{\eta }=0$ when $\eta$ is not a weight.

Nilpotence of $H$ gives $H\subseteq L_0$. If $H$ is a Cartan subalgebra and $L_0/H\ne 0$, the action of $H$ on this quotient consists of nilpotent operators. Engel's theorem supplies a nonzero coset $v+H$ killed by $H$, giving $[H,v]\subseteq H$ with $v\notin H$, a contradiction to self-normalization. Conversely, if $L_0=H$ and $v$ normalizes $H$, then $[h,v]\in H$ for each $h$. Applying further powers of $\mathrm{ad}h$ kills this vector because its restriction to $H$ is nilpotent. Hence $v\in L_0=H$. This proves the equivalence.

The operator $\mathrm{ad}e$ sends $L_{\phi }$ into $L_{\phi +\alpha }$. Repetition follows a consecutive $\alpha$-string; after $r$ steps it must leave the weight set, because $r$ is the largest number of weights in any such string. Thus $\boxed{(\mathrm{ad}e{)}^r=0}$ on every weight space and hence on $L$.

Finally put $E=\mathrm{ad}e$ and $F=\mathrm{ad}f$. Expand $(E+F{)}^n$ as the sum of all words of length $n$, without commuting $E$ and $F$. Acting on $L_{\gamma }$, each word follows weights $\gamma +i\alpha +j\beta$ with $i+j$ increasing by one at each step. Linear independence of $\alpha ,\beta$ in characteristic zero makes the successive weights distinct. A nonzero length-$n$ path would require $n+1$ different weight spaces, but $\dim L=n$. Every word therefore vanishes, giving

First prove the unlettered common-eigenvector assertion, in the stronger form that some nonzero vector is killed by every operator in $W$. This is the linear form of Engel's theorem.

Induct on $\dim W$, with the zero algebra immediate. If $a^r=0$, the operator $\mathrm{ad}a$ on $\mathrm{End}V$ is nilpotent: left and right multiplication commute, and every term in $(L_a-R_a{)}^{2r-1}$ contains a power of $a$ at least $r$. This argument works in every characteristic.

For a proper Lie subalgebra $H<W$, its action on $W/H$ by brackets is therefore by nilpotent operators. The image of this action has dimension at most $\dim H<\dim W$, so induction gives a nonzero class $w+H$ killed by $H$. Thus the normalizer $N_W(H)=\{w:[w,H]\subseteq H\}$ strictly contains $H$. Choose $H$ maximal proper. Its normalizer must be $W$, so $H$ is an ideal. Moreover $W/H$ has dimension one, since otherwise any one-dimensional subalgebra of the quotient would lift to an intermediate proper subalgebra.

Induction applied to $H$ acting on $V$ makes $V_0=\{v:Hv=0\}$ nonzero. This space is $W$-invariant: for $h\in H,x\in W,v\in V_0$, $h(xv)=x(hv)+[h,x]v=0$. Choose $x$ complementing $H$ in $W$. Its restriction to the nonzero space $V_0$ is nilpotent and has a nonzero kernel. A vector in this kernel is killed by $x$ and $H$, hence by all of $W$. This proves the preliminary assertion, with every common eigenvalue equal to zero.

Now apply it to the adjoint action of $L$ on itself. If $L\ne 0$, its common zero space is the center $Z(L)$, which is therefore nonzero. Every induced adjoint map on $L/Z(L)$ remains nilpotent. Induction on dimension makes that quotient nilpotent. A central extension of a nilpotent Lie algebra is nilpotent: if its lower central series vanishes after projection, the next term lies in the center and the following term is zero. Hence $L$ is nilpotent. The zero algebra is immediate.

If $\mathfrak{g}$ is nilpotent, every Lie subalgebra $\mathfrak{a}$ is nilpotent, since ${\mathfrak{a}}^j\subseteq {\mathfrak{g}}^j$ in the respective lower central series of a Lie algebra. A two-dimensional nilpotent algebra has nonzero centre by part (i). If $0\ne z$ is central, complete it to a basis $(z,u)$; every bracket of these basis elements is zero. Thus $\mathfrak{a}$ is abelian.

Conversely, suppose $\mathfrak{g}$ is not nilpotent. By Engel's theorem, some $\mathrm{ad}(x)$ is not nilpotent. Since the field is algebraically closed, it has an eigenvalue $\lambda \ne 0$ with eigenvector $y\ne 0$. ThusThe vectors $x,y$ are independent: otherwise their bracket would be zero. Their span is therefore a two-dimensional Lie subalgebra and it is nonabelian. This contradicts the hypothesis. Hence

Fix $x$ with $x^N=0$ as a matrix. On the full matrix algebra define $L_x(y)=xy$ and $R_x(y)=yx$. These endomorphisms commute and each has $N$th power zero. The Adjoint representation of a Lie algebra is $\mathrm{ad}(x)=L_x-R_x$. The binomial theorem givesFor $m=2N-1$, each summand has either $j\ge N$ or $m-j\ge N$, and hence vanishes. Therefore $\mathrm{ad}(x)$ is nilpotent on the full matrix algebra and on its invariant subspace $\mathfrak{g}$.

Engel's theorem, in its adjoint form, states that a finite-dimensional Lie algebra is nilpotent if every $\mathrm{ad}(x)$ is a nilpotent endomorphism. Applying it to all $x\in \mathfrak{g}$ provesThis is distinct from assuming that an arbitrary representation of a nilpotent algebra must act by nilpotent matrices, which is false.