= Neural network receptive field
A <neural network receptive field> describes which input positions can influence a specified output unit through the network's computational connections. A local <convolutional layer> uses only a small input region for each output, but successive layers can combine information from a larger region. This dependence is relative to a chosen input layer; an immediate previous-layer neighborhood and the corresponding original-input region have different sizes.<Deep Learning by Ian Goodfellow and others>[Chapter 9, §9.2, sparse interactions and Figure 9.4, growth of receptive fields in deeper layers]{ref}
For an own one-dimensional example, use two kernel-size-three <convolutional layers>, with <convolution stride> one, no padding and no <convolution dilation>. Intermediate units $0$, $1$ and $2$ depend on input sets $\{0,1,2\}$, $\{1,2,3\}$ and $\{2,3,4\}$ respectively. The first output of the second layer combines these three units, so its possible original-input dependency is
$$
\{0,1,2\}\cup\{1,2,3\}\cup\{2,3,4\}=\{0,1,2,3,4\}.
$$
Two size-three filters therefore give a size-five receptive region, not size nine.
A recurrence tracks the geometry of an undilated serial network. Let $r_\ell$ be the span of the receptive region in original-input positions and $j_\ell$ the original-input index separation between adjacent layer-$\ell$ outputs. With kernel size $k_\ell$, stride $s_\ell$ and initial values $r_0=j_0=1$,
$$
j_\ell=s_\ell j_{\ell-1},
\qquad
r_\ell=r_{\ell-1}+(k_\ell-1)j_{\ell-1}.
$$
The last contributing previous-layer unit lies $(k_\ell-1)j_{\ell-1}$ input positions beyond the first, which proves the span increment. This derivation assumes the stated local serial connectivity, no padding and no dilation.
If the first size-three layer instead has stride two, then $r_1=3$ and $j_1=2$. A second size-three, stride-one layer has $r_2=7$: its three input supports are $\{0,1,2\}$, $\{2,3,4\}$ and $\{4,5,6\}$. Their union reaches from zero through six. For other sampling choices, a span can include gaps, so span and number of contributing positions need not agree.
This is a structural dependency calculation. Zero weights or the current behavior of an <activation function> can suppress an influence that the architecture permits; the calculation does not measure how strongly each position affects a particular output. A unit-stride <pointwise convolution> mixes <neural network channels> at the same location and does not enlarge this spatial region. The distinction helps explain why many channels alone do not guarantee access to distant input context.
Back to article page
