 Solution

Maschke's theorem says that if $H$ is a finite group and the characteristic of a field $F$ does not divide $|H|$, then every finite-dimensional $F$-representation of $H$ is semisimple.

To prove it, let $W$ be an invariant subspace of a representation $V$, and choose any linear projection $P:V\to W$. Average its conjugates:The scalar $|H{|}^{-1}$ exists by the characteristic assumption. Reindexing the sum shows that $\overline{P}$ commutes with every $\rho (g)$. Since each summand restricts to the identity on $W$, so does $\overline{P}$; hence $\overline{P}$ is an equivariant projection onto $W$. Its kernel is an invariant complement, and induction on $\dim V$ decomposes $V$ into irreducible representations.

The isometry group of $\mathbb{Z}$ is the infinite dihedral groupLet $\rho$ be a nonfaithful finite-dimensional complex representation. Its nontrivial normal kernel contains $t^m$ for some $m>0$, by the stated hint, so $\rho$ factors through the finite quotientMaschke's theorem decomposes this quotient representation into irreducibles. To bound their dimensions, choose a $t$-eigenvector $v$ with eigenvalue $\lambda$ in an irreducible constituent. The relation givesso $\mathrm{span}\{v,sv\}$ is invariant under both generators. Irreducibility makes it the whole constituent, whose dimension is therefore at most two. This proves the result on nonfaithful representations of the infinite dihedral group.

For the requested representation of $(\mathbb{Z},+)$, let its generator act on the polynomials of degree at most one by translation:In the basis $(1,x)$,This is a nontrivial Jordan block, so it is not diagonalizable. A direct sum of one-dimensional subrepresentations would make $T$ diagonalizable, proving that this nonsemisimple translation representation of the infinite cyclic group has the required property.

Finally, on $V=\mathbb{C}[x{]}_{\le 2}$ defineThen $S^2=I$ and $STS=T^{-1}$, so these operators define a quadratic-polynomial representation of the infinite dihedral group. In the basis $(1,x,x^2)$ they areHere $(T-I{)}^2\ne 0$ but $(T-I{)}^3=0$, so $T$ consists of one size-three Jordan block. If $V$ were a direct sum of subrepresentations of dimensions at most two, then $T$ would be block diagonal with Jordan blocks of sizes at most two, a contradiction. Thus this representation has the required failure of decomposition.