 Neural network receptive field

A neural network receptive field describes which input positions can influence a specified output unit through the network's computational connections. A local convolutional layer uses only a small input region for each output, but successive layers can combine information from a larger region. This dependence is relative to a chosen input layer; an immediate previous-layer neighborhood and the corresponding original-input region have different sizes.^[ref]

For an own one-dimensional example, use two kernel-size-three convolutional layers, with convolution stride one, no padding and no convolution dilation. Intermediate units $0$, $1$ and $2$ depend on input sets $\{0,1,2\}$, $\{1,2,3\}$ and $\{2,3,4\}$ respectively. The first output of the second layer combines these three units, so its possible original-input dependency isTwo size-three filters therefore give a size-five receptive region, not size nine.

A recurrence tracks the geometry of an undilated serial network. Let $r_{\ell }$ be the span of the receptive region in original-input positions and $j_{\ell }$ the original-input index separation between adjacent layer-$\ell$ outputs. With kernel size $k_{\ell }$, stride $s_{\ell }$ and initial values $r_0=j_0=1$,The last contributing previous-layer unit lies $(k_{\ell }-1)j_{\ell -1}$ input positions beyond the first, which proves the span increment. This derivation assumes the stated local serial connectivity, no padding and no dilation.

If the first size-three layer instead has stride two, then $r_1=3$ and $j_1=2$. A second size-three, stride-one layer has $r_2=7$: its three input supports are $\{0,1,2\}$, $\{2,3,4\}$ and $\{4,5,6\}$. Their union reaches from zero through six. For other sampling choices, a span can include gaps, so span and number of contributing positions need not agree.

This is a structural dependency calculation. Zero weights or the current behavior of an activation function can suppress an influence that the architecture permits; the calculation does not measure how strongly each position affects a particular output. A unit-stride pointwise convolution mixes neural network channels at the same location and does not enlarge this spatial region. The distinction helps explain why many channels alone do not guarantee access to distant input context.