 Solution

Take $fg=f\circ g$. The assertion can fail on an infinite set. On $X={\mathbb{N}}_0$, set $g(n)=n+1$, $f(0)=0$, and $f(n)=n-1$ for $n\ge 1$. Then $f(g(n))=n$ for every $n$, but $g(f(0))=1\ne 0$. Thus a right inverse need not be a left inverse.

On a finite set it is always true. From $fg={\mathrm{id}}_X$, $g$ is an injection: equality of two $g$-values gives equality after applying $f$. An injective self-map of a finite set is a bijection. Composing $fg={\mathrm{id}}_X$ with $g^{-1}$ shows $f=g^{-1}$, and hence $gf={\mathrm{id}}_X$. This includes the empty set.