= Pointwise convolution
A <pointwise convolution> uses a spatial kernel of size one to combine <neural network channels> independently at each spatial position. For a two-dimensional input this is a one-by-one <convolutional layer>. In the ordinary dense-channel, unit-stride case, the same affine map acts on the channel vector at every grid point.<Conv2d in the PyTorch documentation>[Opening equation and kernel_size/stride parameters, specialized to kernel size one and groups=1]{ref}
Let $x_{i,j}\in\mathbb R^{C_{\mathrm{in}}}$ be the channel vector at position $(i,j)$. With weight <matrix> $M$ and <neural network bias> vector $b$, the output is
$$
y_{i,j}=Mx_{i,j}+b,
\qquad
M\in\mathbb R^{C_{\mathrm{out}}\times C_{\mathrm{in}}}.
$$
This is <matrix multiplication> across channels, not an average of neighboring spatial positions. The same $M$ and $b$ are reused throughout the grid.
For an example without biases, take
$$
M=\begin{pmatrix}1&2&0\\0&-1&1\end{pmatrix}.
$$
At a point with input $(1,2,3)^{\mathsf T}$ the output is $(5,1)^{\mathsf T}$. At another point with input $(3,0,4)^{\mathsf T}$ it is $(3,4)^{\mathsf T}$. A three-channel <neural network feature map> has become a two-channel map without mixing the two positions.
There are $C_{\mathrm{in}}C_{\mathrm{out}}$ weights, plus $C_{\mathrm{out}}$ optional biases. Thus three input and two output channels require six weights, or eight parameters including biases. A dense three-by-three filter with the same channel counts requires $9(3)(2)=54$ weights, or fifty-six parameters with biases. This comparison counts parameters for the stated connectivity; it does not claim that the two layers compute equivalent functions.
Channel reduction can make a later spatial <convolutional layer> cheaper. The original <Inception (deep learning architecture)> uses one-by-one reductions before some larger filters. A pointwise operation can also increase the channel count; size one refers to its spatial footprint, not to the number of channels.<Going Deeper with Convolutions by Christian Szegedy and others>[§4 and Fig. 2, one-by-one dimension-reduction branches]{ref} With no intervening spatial operation, pointwise layers and pointwise <activation functions> alone do not enlarge the spatial <neural network receptive field>.
Back to article page
