Past exam of the mathematics course of the University of Cambridge / 2017 / ii / Paper 1 / 18G / b

Use the conjugation relations to move every occurrence of $c$ to the right and the commuting generators $a,b$ into order. Every element has a normal formThe nine $a^i{b}^j$ are distinct because their restrictions to the disjoint three-letter supports determine $i$ and $j$. Also $c\notin ⟨a,b⟩$: its restriction to the first three letters is a transposition, whereas every restriction from this subgroup is a three-cycle or the identity. Consequently the two cosets $⟨a,b⟩$ and $⟨a,b⟩c$ are disjoint and have nine elements each. Thus $|E|=18$.

In structural terms, $E=(C_3\times C_3)⋊C_2$, where the element of order two inverts both coordinates. This is a generalized dihedral group. The normal-form count also proves that the listed relations are a complete presentation, rather than merely some relations holding in the permutation group.

Let $A,B$ be the stated diagonal matrices and $C$ the interchange matrix. Since ${\epsilon }^3={\eta }^3=1$, direct matrix multiplication givesThe complete presentation established by the preceding normal-form argument therefore defines a group homomorphism $\rho :E\to {\mathrm{GL}}_2(\mathbb{C})$. More explicitly, setMultiplication of normal forms uses $c{a}^i{b}^j=a^{-i}{b}^{-j}c$, and the matrices obey exactly the same rule, so this is well-defined and multiplicative. It is the required complex group representation.

On the normal subgroup $N=⟨a,b⟩$, the first diagonal entry is the linear character $\chi (a^i{b}^j)={\epsilon }^i{\eta }^j$. Its image has order at most three, although $|N|=9$, so its kernel of a group homomorphism has order at least three. The second entry is the inverse character and has the same kernel of a group homomorphism. ThusElements of the other coset have off-diagonal image and cannot be the identity. If $(\epsilon ,\eta )\ne (1,1)$, the kernel of a group homomorphism has order exactly three; otherwise it is all of $N$.

When both roots equal one, only $C$ acts nontrivially and its eigenlines spanned by $(1,1)$ and $(1,-1)$ are invariant, so the representation is reducible. Otherwise one of $A,B$ has two distinct eigenvalues. Any invariant line would be one of its coordinate eigenlines, but $C$ interchanges these two lines. Hence no invariant line exists, and