 Solution

The group order $|G|$ is the number of elements of the finite group. A normal subgroup $H\le G$ satisfies $gH{g}^{-1}=H$ for every $g\in G$, equivalently its left and right cosets agree.

For the square, let $R$ be counterclockwise rotation by $\pi /2$, and $s$ reflection in the horizontal axis. The dihedral group isThe first four elements rotate by $0,90,180,270$ degrees. The other four are Euclidean reflections in the horizontal axis, the line $y=x$, the vertical axis and the line $y=-x$, respectively, for a square centred at the origin with sides parallel to the axes.

The eight nontrivial proper subgroups areTo check completeness, Lagrange's theorem allows proper orders one, two and four. The order-two subgroups are generated by the five involutions displayed. An order-four subgroup containing $R$ or $R^3$ is the rotation subgroup. Otherwise it must contain reflections; their product is a rotation, which must be $R^2$, so its reflections differ by that half-turn, giving the two listed Klein subgroups. The trivial subgroup $\{I\}$ is an additional proper subgroup. Thus the PDF's count of eight uses “proper” to exclude the trivial subgroup; with the usual inclusive definition there are nine proper subgroups.

A normal subgroup of order two is central: conjugation fixes its identity and must also fix its only other member. The half-turn $R^2$ commutes with every square symmetry, whereas the reflection subgroups are exchanged by conjugation. Hence the unique order-two normal subgroup isFor the general polygon, write $D_n=\{R^j,R^{j}s:0\le j<n\}$ with $R^n=s^2=I$ and $sRs=R^{-1}$. A rotation $R^j$ commutes with $s$ exactly when $R^j=R^{-j}$, or $2j\equiv 0(\mathrm{mod}n)$. A reflection cannot commute with $R$ when $n\ge 3$, becauseTherefore the center of a group here is $\{I\}$ for odd $n$, and $\{I,R^{n/2}\}$ for even $n$. The order-two normal subgroups correspond exactly to nonidentity central involutions, provingThis agrees with the involutions in an even dihedral group and uses the convention $|D_n|=2n$ throughout.