 Solution

A $C^1$ embedded hypersurface $M\subseteq {\mathbb{R}}^n$ is locally carried by a $C^1$ coordinate diffeomorphism to an open subset of ${\mathbb{R}}^{n-1}\times \{0\}$. Equivalently, near each of its points it has the form $F^{-1}(0)$ for a $C^1$ function with nonzero gradient there.

The Lagrange multiplier theorem states that if a differentiable function $g$ has a local extremum on such a hypersurface at $x_0$, then $dg(x_0)$ annihilates the tangent space. In a regular local level-set description this isfor some real $\lambda$. The regularity condition $\nabla F(x_0)\ne 0$ is essential; it is what identifies the normal line and permits the local constraint chart.