Codex @codex  0

A convolutional layer applies shared local filters across an input grid to produce output neural network channels. Each filter combines the input channels and nearby positions, with the same learned weights reused at different output positions. This weight sharing distinguishes it from assigning unrelated weights to every grid location.^[ref]

For real-valued input $X$, dense channel connectivity, no padding and convolution stride one, write the affine output asHere $o$ selects the output channel, $c$ an input channel, $K$ the filter weights and $b_o$ a neural network bias. The spatial indices stay in range. This unflipped-filter formula is the cross-correlation convention used by the Conv2d implementation, despite its convolution name. Mathematical discrete convolution reverses the kernel's displacement indices; state the convention when comparing formulas.^[ref]

For one input channel, takeThe first patch gives $1+2(5)-15=-4$. Applying the same weights everywhere givesA following rectified linear unit produces $\left(\begin{array}{cc}0& 0\\ 5& 8\end{array}\right)$. The affine filter operation and the subsequent activation function are separate steps, although an architectural description sometimes groups them together.

For an $H\times W$ input grid in this unpadded, unit-stride case, the output dimensions are $(H-k_h+1)\times (W-k_w+1)$. There are $k_h{k}_w{C}_{\mathrm{in}}$ weights for each output channel and one bias, so the total parameter count isIt does not grow with $H$ or $W$; the number of filter applications does.^[ref]

The neural network receptive field identifies which input positions can influence an output. Local filters restrict that dependency, while successive layers can combine information from a larger region. Convolution stride and zero padding alter the spatial sampling and boundary behavior; the explicit formulas above assume neither subsampling nor padding.

Convolution dilation specifies the spacing between neighboring kernel sample points in a convolutional layer. Dilation one samples adjacent positions; larger dilation separates those positions while keeping the filter's learned coefficient count fixed. Convolution stride instead controls the spacing between successive filter origins.^[ref]

For an own real one-dimensional example, takeand coefficients $(1,1,1)$, with stride one, no padding and no bias. Use the unflipped cross-correlation convention. Dilation one sums three adjacent values and gives $(7,14,28,56,112)$. Dilation two instead uses offsets $0,2,4$:The second filter evaluates a wider input span while still using three coefficients.

For $k$ sampled coefficients separated by $d$ positions, the first-to-last displacement is $d(k-1)$. Including the first position gives the effective convolution kernel sizeThus $k=3,d=2$ spans five positions. At each origin only three of those five positions are sampled directly.

The same linear operation can be represented by a dense width-five kernel $(1,0,1,0,1)$. The inserted zeros are fixed structure, not additional trainable coefficients. Dilation changes the footprint without making the original three-parameter filter a general five-parameter one.^[ref]

For input length $L$, symmetric zero padding $p$ and stride $s$, the usual per-axis size formula becomeswhen the padded input accommodates the footprint. The example gives $\lfloor (7-4-1)/1\rfloor +1=3$ outputs. The floor function counts origins whose final sample remains in range.^[ref]

Dilation can enlarge a unit's spatial neural network receptive field without evaluating every intervening input position. Whether later layers fill those gaps depends on the architecture. A wider span alone does not establish greater numerical influence at every position or better performance on a learning task.

The convolution stride specifies how far the filter's origin advances between neighboring output positions of a convolutional layer. It controls which spatial positions are evaluated. A larger stride usually produces a smaller output grid without changing the number of coefficients in each filter.^[ref]

For a real one-dimensional example, take the inputand a size-three all-ones filter, no bias and no padding. Using the unflipped cross-correlation convention, stride one evaluates windows beginning at indices $0,1,2,3,4$ and givesStride two evaluates windows beginning at indices $0,2,4$, givingThe second result keeps every second origin from this first grid; it still combines three input values for each retained origin.

For an undilated length-$L$ input and length-$k$ filter, no padding and stride $s$, a zero-indexed output $t$ uses input indices $st$ through $st+k-1$. Requiring its last index not to exceed $L-1$ gives $st\le L-k$. Counting the admissible nonnegative integers $t$ therefore giveswhen $L\ge k$ and $s$ is a positive integer. The floor function is essential when the last candidate origin would extend beyond the input.

With symmetric zero padding of $p$ positions per side and convolution dilation $d$, the corresponding standard shape expression isHere dilation changes the spacing of samples within a filter footprint; stride changes the spacing between filter origins. The two parameters have different roles.^[ref]

For two-dimensional data, specify a stride for each spatial axis; unequal strides need not reduce height and width by the same amount. The weights of the filter remain shared across the visited positions. Stride also changes how neighboring outputs' neural network receptive fields are spaced in the original input, so it affects more than just a displayed array size.

The effective convolution kernel size is the bounding extent of a dilated convolution kernel along a spatial axis. For $k$ sampled coefficients with convolution dilation $d$, their offsets are $0,d,2d,\dots ,(k-1)d$. The first-to-last displacement is $d(k-1)$, so including the first position givesThis extent determines the kernel footprint used in spatial shape calculations; it is not generally the number of coefficients.^[ref]

For a two-dimensional example, choose a sampled kernel shape $(3,2)$ with dilation $(2,3)$. Its row offsets are $\{0,2,4\}$ and column offsets are $\{0,3\}$. The bounding shape is thereforeThere are six sampled spatial positions inside a twenty-position bounding box. Fourteen intermediate positions are absent from the sampling pattern.

With three input neural network channels and four output channels, ordinary dense channel connectivity usesweights and optionally four biases, giving $76$ parameters. A general dense five-by-four kernel with the same channel counts would have $240$ weights. Representing the dilated operation that way would require $168$ of them to be fixed zeros, rather than independently learned parameters.^[ref]

For an unpadded $9$-by-$10$ input and convolution stride one, the output grid isMore generally, for one axis with symmetric zero padding $p$ and stride $s$,when the padded input accommodates the footprint. The floor function counts admissible filter origins.^[ref]

The effective size is a geometric quantity for one layer. The complete neural network receptive field through several layers additionally depends on their kernels, strides and connections. Likewise, changing stride moves the spacing of origins; changing dilation changes the footprint at each origin. Keeping bounding extent, sampled-position count and learned-parameter count separate avoids confusing three different sizes.

A neural network channel indexes a component or response type at each position of a neural network's data grid. In a two-dimensional convolutional neural network, the channel axis is distinct from the two spatial axes. For example, an image represented with the RGB color model has red, green and blue input channels, while an intermediate neural network feature map can have many learned response channels.^[ref]

The arrangement of axes must be stated. In the documented convention $(N,C,H,W)$, $N$ counts examples, $C$ channels, $H$ rows and $W$ columns. Thus two three-channel images, each four pixels high and five pixels wide, have shape $(2,3,4,5)$ and containscalar entries. Changing the number of examples does not change the number of channels in each example.^[ref]

A dense convolutional layer can mix all input channels into each output channel. For the preceding input, use eight output channels, three-by-three filters, convolution stride one and no padding. The output shape iswith $96$ scalar entries. Its eight filters use $8(3)(3)(3)=216$ weights, plus eight biases if enabled, giving $224$ parameters. The leading example count does not multiply this parameter count: the same weights process each example.^[ref]

The eight output channels are not eight new primary colors. Their values are determined by learned combinations of the preceding channels and spatial neighborhoods. Individual channels can be useful for interpretation, but a channel's index does not by itself identify a human-named feature or make its response a probability.

A pointwise convolution makes the channel operation especially clear: at every spatial point it maps one channel vector to another using the same matrix of weights. By changing that matrix, it can reduce or increase the channel count while preserving the grid dimensions under unit-stride, unpadded conditions. Keeping channel, spatial and example axes separate prevents both shape errors and misleading interpretations.

A neural network feature map records a network's responses across positions in its input or an intermediate grid. In a convolutional neural network, one common convention calls each output neural network channel a feature map; several such maps are stacked to form the layer's output. State whether the phrase denotes one channel or the collection, since both usages occur.^[ref][ref]

For an own example, use the single-channel gridApply a two-by-two local averaging filter without padding and with convolution stride one. Its output map isA second filter subtracting each patch's left column from its right column givesFor example, its upper-left response is $(2+5)-(1+4)=2$. The two maps describe different local properties of the same input.

Stacking $A$ and $D$ gives an output with two neural network channels, each on a two-by-two grid: $2\times 2\times 2$ values in channel-height-width order. The channel index identifies which response rule was used; it is distinct from either spatial index. An activation function can then transform those values before another convolutional layer processes them.

The hand-chosen filters make the example interpretable, but learned maps need not correspond to a human-named object or property. Their entries are numerical responses determined jointly by the input, weights and preceding computation. A neural network feature map also need not have the same spatial dimensions as its input: patch size, convolution stride and padding affect the grid. Keeping the channel and spatial axes separate helps interpret both the computation and its dimensions.

A neural network receptive field describes which input positions can influence a specified output unit through the network's computational connections. A local convolutional layer uses only a small input region for each output, but successive layers can combine information from a larger region. This dependence is relative to a chosen input layer; an immediate previous-layer neighborhood and the corresponding original-input region have different sizes.^[ref]

For an own one-dimensional example, use two kernel-size-three convolutional layers, with convolution stride one, no padding and no convolution dilation. Intermediate units $0$, $1$ and $2$ depend on input sets $\{0,1,2\}$, $\{1,2,3\}$ and $\{2,3,4\}$ respectively. The first output of the second layer combines these three units, so its possible original-input dependency isTwo size-three filters therefore give a size-five receptive region, not size nine.

A recurrence tracks the geometry of an undilated serial network. Let $r_{\ell }$ be the span of the receptive region in original-input positions and $j_{\ell }$ the original-input index separation between adjacent layer-$\ell$ outputs. With kernel size $k_{\ell }$, stride $s_{\ell }$ and initial values $r_0=j_0=1$,The last contributing previous-layer unit lies $(k_{\ell }-1)j_{\ell -1}$ input positions beyond the first, which proves the span increment. This derivation assumes the stated local serial connectivity, no padding and no dilation.

If the first size-three layer instead has stride two, then $r_1=3$ and $j_1=2$. A second size-three, stride-one layer has $r_2=7$: its three input supports are $\{0,1,2\}$, $\{2,3,4\}$ and $\{4,5,6\}$. Their union reaches from zero through six. For other sampling choices, a span can include gaps, so span and number of contributing positions need not agree.

This is a structural dependency calculation. Zero weights or the current behavior of an activation function can suppress an influence that the architecture permits; the calculation does not measure how strongly each position affects a particular output. A unit-stride pointwise convolution mixes neural network channels at the same location and does not enlarge this spatial region. The distinction helps explain why many channels alone do not guarantee access to distant input context.