Residual connection
= Residual connection
{title2=$z\mapsto z+F(z)$}
A connection that adds a sublayer's input to its output, using a projection when dimensions require it. Its derivative is $I+DF(z)$, so the derivative has a direct identity contribution in addition to the sublayer's contribution. This supports propagation through a deep network but does not itself guarantee bounded <gradients>.