 Solution

Take independent smooth variations $y+\epsilon \eta$ and $z+\epsilon \zeta$. Differentiating the functional and using integration by parts gives its first variationFirst choose variations vanishing at the endpoints. To use the stated form of the fundamental lemma of the calculus of variations, take $\eta =(x-a)(b-x)\chi$ with arbitrary smooth $\chi$, and similarly for $\zeta$. The lemma makes $(x-a)(b-x)(F_y-d{F}_{y^{\prime }}/dx)$ vanish, and hence makes the unweighted coefficient vanish in the interior; continuity extends this to the endpoints. Applying the same argument to $z$ gives the Euler-Lagrange equationsIf either endpoint value of $y$ or $z$ is free, the corresponding variation there is arbitrary. The remaining boundary term gives the natural boundary conditions for a free endpoint: $F_{y^{\prime }}=0$ for a free $y$ value and $F_{z^{\prime }}=0$ for a free $z$ value at that endpoint. With both coordinates free at both endpoints, all four conditions apply.

For the specified integrand these equations reduce toand the natural conditions are $y^{\prime }=z^{\prime }=0$ at each free endpoint. Eliminating $z$ givesIts four independent modes give the full general solutionAt zero, $y^{\prime }(0)=C+2D$ and $z^{\prime }(0)=2C+D$. Vanishing forces $C=D=0$, givingAt $x=\pi /6$ the two derivative conditions become $A/2+\sqrt{3}B=0$ and $A+\sqrt{3}B/2=0$. Their coefficient determinant is nonzero, so $A=B=0$. Thus the unique stationary extremal with all four natural boundary conditions is $y=z=0$. The word extremal here means a stationary point of the functional; the indefinite integrand does not require this point to be a minimum.