 Solution

For fixed endpoint values, the Euler-Lagrange equation is $f_y-d{f}_{y^{\prime }}/dx=0$. Along a stationary curve, the chain rule givesWhen $f_x=0$, the Beltrami identity therefore takes the requested sign convention

By Fermat principle, the elapsed-time integrand for the given speed is $f=c_0^{-1}\sqrt{1-ky}\sqrt{1+(y^{\prime }{)}^2}$. ThusThe constant is negative for a finite regular ray. Squaring gives $(1-ky)/(1+(y^{\prime }{)}^2)=c_0^2{A}^2$. At a maximum height $y_0$, the slope is zero, so $c_0^2{A}^2=1-k{y}_0>0$. HenceSeparate variables on either side of the turning point. Integration gives the parabolic light rays in a reciprocal-square-root speed profileThe quadratic opens downwards, with its greatest height $y_0$ at $x=x_c$. The positive index condition holds throughout the layer because $kh<1$. Vertical rays are a separate zero-horizontal-momentum case, outside the regular graph representation $y(x)$; they are not returning rays of the type considered below.

Equal endpoint heights at $x=\pm x_0$ force $x_c=0$. Setting $y(\pm x_0)=0$ givesFor nonzero separation, a real returning ray requires $k{x}_0\le 1$ and has candidate turning heightsOnly choices satisfying $0<y_0<h$ are rays wholly within the specified layer. Depending on its thickness there may be two, one, or no admissible choices; at zero discriminant the two algebraic candidates coincide. The entry and exit points at $y=0$ are interpreted as limiting boundary points of the open medium.