Solution (source code)

= Solution

Let $R$ be rotation through $2\pi/3$ about $z$, and let $S$ be rotation through $\pi$ about $x$. These generate the order-six <dihedral group> with relations
$$
R^3=S^2=I,\qquad SRS=R^{-1},
$$
namely $\{I,R,R^2,S,RS,R^2S\}$. We specify its order explicitly because dihedral notation has two conventions. It is a proper finite <subgroup> of the full axial rotation <group>.

To see that it is sufficient, write a general <matrix> in blocks $T=\begin{pmatrix}B&u\\v^T&c\end{pmatrix}$. Commutation with $R$ forces $u=v=0$, since a nontrivial planar $120$-degree rotation fixes no nonzero planar <vector>. Its planar part is $\cos(2\pi/3)I+\sin(2\pi/3)J$, with nonzero sine, so $B$ must commute with $J$ and has the form $aI+bJ$. Commutation with $S$ again kills $b$. Hence this six-element <subgroup> has precisely the required invariant <tensors>.

It is also smallest by order. <Groups> of orders $2,3,5$ are cyclic: a nonidentity element has order dividing the prime <group> order by <Lagrange theorem>. For a cyclic rotation <subgroup>, the nonzero antisymmetric <tensor> representing <cross product> with its rotation-axis direction is invariant, so it cannot force the symmetric form above. The trivial <subgroup> plainly imposes no constraint. A <group> of order four is either cyclic or has three nonidentity elements of order two; in the latter case they commute, since $(AB)^{-1}=AB$ and also equals $BA$. Commuting distinct half-turns in three dimensions have perpendicular axes: conjugation by one half-turn must preserve the other's axis, and a distinct preserved axis lies in its perpendicular plane. In the specified axial <group>, their three axes are consequently $z$ and two perpendicular horizontal directions. In coordinates along them every <diagonal matrix> is invariant, including one with unequal horizontal entries. Such a <group> again does not force the required form.

Thus \b[the smallest <subgroup> has six elements], generated by the axial $120$-degree rotation and one horizontal half-turn. It is the <smallest axial rotation group forcing second-rank transverse isotropy>; conjugating the horizontal axis gives equally valid choices.