Past exam of the mathematics course of the University of Cambridge / 2019 / ii / Paper 3 / 19I (Representation Theory)

The Mackey's theorem, or Mackey restriction formula, says that if $H,K\le G$, $W$ is a complex $K$-representation, and $D$ is a set of representatives for the double cosets $H\backslash G/K$, thenHere ${}^{x}W$ is the representation of the conjugate subgroup $xK{x}^{-1}$ defined by

For the proof, use the group-algebra modelThe partition of $G$ into double cosets gives the direct-sum decomposition of $(H,K)$-bimodulesPut $L_x=H\cap xK{x}^{-1}$. For each $x$, defineIf $\ell =xk{x}^{-1}\in L_x$, thenso $T_x$ is well defined. It is visibly $H$-equivariant and surjective, and choosing representatives for $H/L_x$ shows that both sides have dimension $[H:L_x]\dim W$; hence it is an isomorphism. Summing the $T_x$ proves the formula.

The Frobenius reciprocity theorem states that for an $H$-representation $U$ and a $G$-representation $V$,For the corresponding character of a representation, this is

On the orbit $G/H$, an element $g\in G$ fixes the coset $xH$ exactly whenThe character of a representation of the corresponding permutation representation therefore isThis is the permutation character of a coset action.

The vectoris fixed by every element of $G$, so $\mathbb{C}{1}_X\cong {\mathbb{C}}_G$ is a trivial representation. The coefficient-sum subspaceis also $G$-invariant, and every vector has a unique decomposition into a constant vector and an element of $V$. HenceHere $V$ is the augmentation subrepresentation of a permutation representation.

Let $D$ represent $H\backslash G/H$. Combining Frobenius reciprocity with the Mackey's theorem givesThus

Work with the selected transitive orbit $X\cong G/H$. By Maschke's theorem, decompose its permutation character aswhere the ${\chi }_j$ are distinct nontrivial irreducible characters. Character orthogonality givesConsequently the augmentation summand $V$ is irreducible exactly when this inner product equals $2$.

The double cosets $H\backslash G/H$ are the $H$-orbits on $G/H$. Because $G$ acts transitively on $X$, these in turn correspond to the $G$-orbits on $X\times X$: move the first coordinate to the base point $H$, after which its stabilizer is $H$. One orbit is the diagonal. There are exactly two orbits precisely when $H$ is transitive on $X\setminus \{H\}$, equivalently when the action is two-transitive. Part (ii) therefore provesThis is the irreducible augmentation criterion for a transitive group action.

Finally, transitivity implies that the fixed-vector space $\mathbb{C}[X{]}^G$ consists only of constant vectors and has dimension one. If the nonzero representation $V$ were a trivial representation, it would contribute another fixed vector, a contradiction. Thus $V$ is not the trivial representation.