Use the real Lie-algebra convention of the question, with a gauge covariant derivative . The gauge field strength is its curvature:
Write . To first order in ,
Substitute . The mixed second derivatives of cancel. The terms containing first derivatives of cancel in pairs. The remaining terms are
The Jacobi identity combines the last two into . Consequently
The transformation is homogeneous even though the connection transformation contains an inhomogeneous derivative term.
For a finite-dimensional Lie algebra, define its Adjoint representation by and its Killing form by
It is a symmetric bilinear form by cyclicity of the trace. The Jacobi identity gives . Set , , . Then
This proves the invariant bilinear form on a Lie algebra property, without assuming simplicity or nondegeneracy.
For definiteness use the Minkowski metric and take a real compact semisimple Lie algebra as the gauge algebra. Its positive internal metric is . A Killing-form Yang-Mills Lagrangian with the coupling absorbed into the connection is
The spacetime metric is unchanged by internal gauge transformations. Hence
Thus gauge invariance follows directly from invariance of the Killing form. Other overall conventions are possible, but the energy sign must be checked rather than inferred from a prefactor in isolation.
For physical kinetic terms the internal form must be real, nondegenerate and positive definite after choosing the overall sign. If and , the above convention has Lagrangian and physical energy density
The canonical Hamiltonian density also contains the nondynamical multiplier and a spatial divergence. With and , integration by parts using the invariant bilinear form on a Lie algebra gives
The Gauss law constraint in gauge theory sets . If the boundary flux vanishes, or the appropriate boundary contribution is included, the integrated physical Hamiltonian is the positive energy displayed above.
An indefinite internal form would give gauge-field polarizations with opposite kinetic signs. A degenerate form would fail to supply a kinetic term for some directions. The compactness criterion from the Killing form says that negative-definiteness of the Killing form of a real finite-dimensional algebra is equivalent to compact semisimplicity; thus the pure Killing-form construction selects compact semisimple real forms. One cannot use a complex-bilinear Killing form on arbitrary complex field components as if it were a positive Hermitian metric.
This does not prohibit Abelian gauge theories. A compact Abelian factor has zero Killing form, so it needs a separately chosen positive invariant bilinear form on a Lie algebra, rather than the Killing form. More generally an algebra with a positive invariant metric on a Lie algebra is compact reductive, namely a direct sum of a compact semisimple algebra and an Abelian center. A noncompact group can also share the same compact Lie algebra through global covering choices in an Abelian factor; positivity is a statement about the algebra and internal metric, not by itself a classification of global gauge-group topology. Quantum matter anomalies and global restrictions would require additional input; no matter content is specified here.
For the finite matrix transformation, let and regard as a multiplication operator. The identity gives
Taking commutators of these differential operators cancels the adjacent multiplication operators , yielding
This argument keeps the derivatives acting on test fields and avoids treating as just a matrix.
There is a normalization issue in the printed last paragraph. The standard Killing form defined through the adjoint trace on is
not simply . For example, with and , the defining trace is , whereas the adjoint trace is . The printed trace formula can be used as a rescaled invariant form, with the constant absorbed into the gauge coupling; it has exactly the invariance needed here. For either normalization,
by cyclicity. This proves finite Yang-Mills gauge transformation invariance, with the healthy sign chosen for anti-Hermitian gauge fields. The normalization discrepancy is not a failure of gauge invariance.
Finally let . Since is traceless and skew-Hermitian, this exponential lies in . Expanding the finite formula gives
Thus the stated infinitesimal transformation is the derivative of the finite transformation at the identity. It describes transformations in the identity component; arbitrary global or large gauge transformations need not be generated by one globally defined infinitesimal parameter.