The operator annihilates one field quantum and excites the atom. Its adjoint de-excites the atom and creates one field quantum. These are the two exchange processes in the Jaynes-Cummings model.
Unitary time evolution requires the Hamiltonian to be Hermitian. Since the common coefficient is real, the two terms must be adjoints with equal coefficients. Therefore