= Neural network channel
A <neural network channel> indexes a component or response type at each position of a neural network's data grid. In a two-dimensional <convolutional neural network>, the channel axis is distinct from the two spatial axes. For example, an image represented with the <RGB color model> has red, green and blue input channels, while an intermediate <neural network feature map> can have many learned response channels.<Conv2d in the PyTorch documentation>[Opening equation, input/output channel counts and spatial dimensions]{ref}
The arrangement of axes must be stated. In the documented convention $(N,C,H,W)$, $N$ counts examples, $C$ channels, $H$ rows and $W$ columns. Thus two three-channel images, each four pixels high and five pixels wide, have shape $(2,3,4,5)$ and contain
$$
2\cdot3\cdot4\cdot5=120
$$
scalar entries. Changing the number of examples does not change the number of channels in each example.<Conv2d in the PyTorch documentation>[Shape, batched input convention]{ref}
A dense <convolutional layer> can mix all input channels into each output channel. For the preceding input, use eight output channels, three-by-three filters, <convolution stride> one and no padding. The output shape is
$$
(2,8,4-3+1,5-3+1)=(2,8,2,3),
$$
with $96$ scalar entries. Its eight filters use $8(3)(3)(3)=216$ weights, plus eight biases if enabled, giving $224$ parameters. The leading example count does not multiply this parameter count: the same weights process each example.<Conv2d in the PyTorch documentation>[Opening channel-summation equation; Shape; Variables, dense weight and bias dimensions with groups=1]{ref}
The eight output channels are not eight new primary colors. Their values are determined by learned combinations of the preceding channels and spatial neighborhoods. Individual channels can be useful for interpretation, but a channel's index does not by itself identify a human-named feature or make its response a probability.
A <pointwise convolution> makes the channel operation especially clear: at every spatial point it maps one channel <vector> to another using the same <matrix> of weights. By changing that matrix, it can reduce or increase the channel count while preserving the grid dimensions under unit-stride, unpadded conditions. Keeping channel, spatial and example axes separate prevents both shape errors and misleading interpretations.
Back to article page
