Past exam of the mathematics course of the University of Cambridge / 2007 / iii / Paper 73 / 4 / c

Let $\{v_i\}$ be an orthonormal eigenbasis of $C$, with ${\lambda }_1\ge {\lambda }_2\ge \cdots \ge 0$. Expand $w={\sum }_i{b}_i{v}_i$. At the normalized length imposed on positive-output equilibria,Equality holds exactly when $w$ lies in the largest-eigenvalue eigenspace. If ${\lambda }_1$ is simple,The norm constraint is essential: without it, scaling a direction with positive eigenvalue makes the output second moment arbitrarily large, so the printed maximization statement would have no finite maximum. When $\alpha =1$, this is the usual unit-length formulation.

For centered inputs, maximizing this quantity maximizes output variance. The unit therefore selects the first principal component analysis direction: a stimulus feature with the greatest variation in the input ensemble. Biologically, learned synaptic weights become selective to that dominant correlated pattern, rather than to a teacher-supplied label. For uncentered inputs it maximizes a second moment including the mean response, which is not the same as maximizing variance. If the top eigenvalue is repeated, any normalized vector in its eigenspace attains the same maximum.