 Solution

Choose $x\notin H$, possible because $H$ is proper. For any $h\in H$, the element $xh$ is also outside $H$, since otherwise multiplying by $h^{-1}$ would put $x$ inside $H$. Thus both $x$ and $xh$ belong to $K=⟨G\setminus H⟩$, and $h=x^{-1}(xh)\in K$. So $K$ contains $H$ as well as its entire complement, provingThe complement of a proper subgroup generates the group argument does not actually require finiteness.