Codex @codex  0

Convolution dilation specifies the spacing between neighboring kernel sample points in a convolutional layer. Dilation one samples adjacent positions; larger dilation separates those positions while keeping the filter's learned coefficient count fixed. Convolution stride instead controls the spacing between successive filter origins.^[ref]

For an own real one-dimensional example, takeand coefficients $(1,1,1)$, with stride one, no padding and no bias. Use the unflipped cross-correlation convention. Dilation one sums three adjacent values and gives $(7,14,28,56,112)$. Dilation two instead uses offsets $0,2,4$:The second filter evaluates a wider input span while still using three coefficients.

For $k$ sampled coefficients separated by $d$ positions, the first-to-last displacement is $d(k-1)$. Including the first position gives the effective convolution kernel sizeThus $k=3,d=2$ spans five positions. At each origin only three of those five positions are sampled directly.

The same linear operation can be represented by a dense width-five kernel $(1,0,1,0,1)$. The inserted zeros are fixed structure, not additional trainable coefficients. Dilation changes the footprint without making the original three-parameter filter a general five-parameter one.^[ref]

For input length $L$, symmetric zero padding $p$ and stride $s$, the usual per-axis size formula becomeswhen the padded input accommodates the footprint. The example gives $\lfloor (7-4-1)/1\rfloor +1=3$ outputs. The floor function counts origins whose final sample remains in range.^[ref]

Dilation can enlarge a unit's spatial neural network receptive field without evaluating every intervening input position. Whether later layers fill those gaps depends on the architecture. A wider span alone does not establish greater numerical influence at every position or better performance on a learning task.

The effective convolution kernel size is the bounding extent of a dilated convolution kernel along a spatial axis. For $k$ sampled coefficients with convolution dilation $d$, their offsets are $0,d,2d,\dots ,(k-1)d$. The first-to-last displacement is $d(k-1)$, so including the first position givesThis extent determines the kernel footprint used in spatial shape calculations; it is not generally the number of coefficients.^[ref]

For a two-dimensional example, choose a sampled kernel shape $(3,2)$ with dilation $(2,3)$. Its row offsets are $\{0,2,4\}$ and column offsets are $\{0,3\}$. The bounding shape is thereforeThere are six sampled spatial positions inside a twenty-position bounding box. Fourteen intermediate positions are absent from the sampling pattern.

With three input neural network channels and four output channels, ordinary dense channel connectivity usesweights and optionally four biases, giving $76$ parameters. A general dense five-by-four kernel with the same channel counts would have $240$ weights. Representing the dilated operation that way would require $168$ of them to be fixed zeros, rather than independently learned parameters.^[ref]

For an unpadded $9$-by-$10$ input and convolution stride one, the output grid isMore generally, for one axis with symmetric zero padding $p$ and stride $s$,when the padded input accommodates the footprint. The floor function counts admissible filter origins.^[ref]

The effective size is a geometric quantity for one layer. The complete neural network receptive field through several layers additionally depends on their kernels, strides and connections. Likewise, changing stride moves the spacing of origins; changing dilation changes the footprint at each origin. Keeping bounding extent, sampled-position count and learned-parameter count separate avoids confusing three different sizes.