 Solution

The printed PDF has a plus sign before the second logarithm. Differentiating that expression literally, with $h_i=\sigma (a_i)$ and $\partial h_i/\partial w_j=h_i(1-h_i)z_{ij}$, givesFor example, at $y_i=0$ it is $-h_i{z}_{ij}$, so minimizing this printed objective pushes the predicted probability of class one upwards rather than downwards; the objective is unbounded below as $h_i\to 1$. It is therefore not the usual binary cross-entropy.

If the second sign was intended to be negative, the standard negative log-likelihood is $-y_i\log h_i-(1-y_i)\log (1-h_i)$ and the derivative simplifies instead to $(h_i-y_i)z_{ij}$. Stating both cases preserves the actual question's algebra while identifying the apparent sign error. The logarithms here are natural; another base multiplies the gradients by its constant conversion factor.