 Solution

The claim fails for a general extension. If $k$ is an infinite field, all the infinitely many maximal ideals $(x-a)$ of $k[x]$ contract to $(0)$ in $k$.

It still fails for an integral extension. Take a finite field $k={\mathbb{F}}_q$ andEvery $b\in B$ satisfies the monic equation $b^q-b=0$, so $B$ is integral over the diagonal copy of $k$. The coordinate kernels are infinitely many distinct maximal ideals, all lying over $(0)$.

The claim is true for a module-finite ring extension. The primes above $\mathfrak{p}$ correspond to the primes of the fiber ringThis is a finite-dimensional algebra over the residue field $\kappa (\mathfrak{p})$, hence an Artinian ring, and an Artinian ring has only finitely many prime ideals.