= Solution
The printed PDF has a \b[plus sign before the second logarithm]. Differentiating that expression literally, with $h_i=\sigma(a_i)$ and $\partial h_i/\partial w_j=h_i(1-h_i)z_{ij}$, gives
$$
\frac{\partial E_i}{\partial h_i}=-\frac{y_i}{h_i}-\frac{1-y_i}{1-h_i},\qquad
\boxed{\frac{\partial E_i}{\partial w_j}=-[y_i(1-h_i)+(1-y_i)h_i]z_{ij}}.
$$
For example, at $y_i=0$ it is $-h_i z_{ij}$, so minimizing this printed objective pushes the predicted <probability> of class one upwards rather than downwards; the objective is unbounded below as $h_i\to1$. It is therefore not the usual <binary cross-entropy>.
If the second sign was intended to be negative, the standard negative log-likelihood is $-y_i\log h_i-(1-y_i)\log(1-h_i)$ and the derivative simplifies instead to \b[$(h_i-y_i)z_{ij}$]. Stating both cases preserves the actual question's algebra while identifying the apparent sign error. The logarithms here are natural; another base multiplies the <gradients> by its constant conversion factor.
Back to article page
