 Gaussian error linear unit

The Gaussian error linear unit activation is $\mathrm{GELU}(x)=x\Phi (x)$, where $\Phi$ is the standard normal distribution function.