= Convolutional layer
A <convolutional layer> applies shared local filters across an input grid to produce output <neural network channels>. Each filter combines the input channels and nearby positions, with the same learned weights reused at different output positions. This <weight sharing> distinguishes it from assigning unrelated weights to every grid location.<Deep Learning by Ian Goodfellow and others>[Chapter 9, §9.2, sparse interactions and parameter sharing]{ref}
For real-valued input $X$, dense channel connectivity, no padding and <convolution stride> one, write the affine output as
$$
Z_{o,i,j}=b_o+
\sum_{c=1}^{C_{\mathrm{in}}}
\sum_{u=0}^{k_h-1}\sum_{v=0}^{k_w-1}
K_{o,c,u,v}X_{c,i+u,j+v}.
$$
Here $o$ selects the output channel, $c$ an input channel, $K$ the filter weights and $b_o$ a <neural network bias>. The spatial indices stay in range. This unflipped-filter formula is the <cross-correlation> convention used by the Conv2d implementation, despite its convolution name. Mathematical <discrete convolution> reverses the kernel's displacement indices; state the convention when comparing formulas.<Conv2d in the PyTorch documentation>[Opening equation, valid two-dimensional cross-correlation and channel summation]{ref}
For one input channel, take
$$
X=\begin{pmatrix}1&2&3\\4&5&6\\7&8&9\end{pmatrix},
\qquad
K=\begin{pmatrix}1&0\\0&2\end{pmatrix},
\qquad b=-15.
$$
The first patch gives $1+2(5)-15=-4$. Applying the same weights everywhere gives
$$
Z=\begin{pmatrix}-4&-1\\5&8\end{pmatrix}.
$$
A following <rectified linear unit> produces $\begin{pmatrix}0&0\\5&8\end{pmatrix}$. The affine filter operation and the subsequent <activation function> are separate steps, although an architectural description sometimes groups them together.
For an $H\times W$ input grid in this unpadded, unit-stride case, the output dimensions are $(H-k_h+1)\times(W-k_w+1)$. There are $k_hk_wC_{\mathrm{in}}$ weights for each output channel and one bias, so the total parameter count is
$$
C_{\mathrm{out}}(k_hk_wC_{\mathrm{in}}+1).
$$
It does not grow with $H$ or $W$; the number of filter applications does.<Conv2d in the PyTorch documentation>[Shape; Variables, weight and bias dimensions, groups=1]{ref}
The <neural network receptive field> identifies which input positions can influence an output. Local filters restrict that dependency, while successive layers can combine information from a larger region. <Convolution stride> and <zero padding> alter the spatial sampling and boundary behavior; the explicit formulas above assume neither subsampling nor padding.
Back to article page
