A connection that adds a sublayer's input to its output, using a projection when dimensions require it. Its derivative is , so the derivative has a direct identity contribution in addition to the sublayer's contribution. This supports propagation through a deep network but does not itself guarantee bounded gradients.
New to topics? Read the docs here!