Activation function
= Activation function
{title2=$z=g(x)$}
{wiki}
An <activation function> converts a unit's weighted input to its output. A <sigmoid function>, hyperbolic tangent or rectifier can give nonlinear features. Replacing all hidden <activation functions> by the identity collapses consecutive affine layers into a single affine map, removing the representational benefit of those hidden layers.