 Solution

Let $\eta$ be a smooth variation with $\eta (a)=\eta (b)=0$. Assuming the usual continuous second derivatives of $F$, stationarity of the functional under $y+t\eta$ gives its first variationIntegration by parts removes the derivative of $\eta$; its boundary term vanishes because the endpoint values are fixed. Thusfor every such variation. The fundamental lemma of the calculus of variations implies the Euler-Lagrange equation $F_y=(d/dx)F_{y^{\prime }}$.

For arc length, $F=\sqrt{1+(y^{\prime }{)}^2}$, soThe function $s\mapsto s/\sqrt{1+s^2}$ is strictly increasing; hence $y^{\prime }$ is constant. With endpoints $(a,A)$ and $(b,B)$, $a<b$, the extremal isIt is also the global shortest curve: integrating the unit displacement vector dotted with the curve velocity gives the endpoint distance, which is at most the integrated speed by Cauchy-Schwarz inequality; the straight segment attains equality. A vertical segment is described parametrically or after rotating axes, since it cannot be represented by $y(x)$ on a nontrivial horizontal interval.