 Solution

Vary $r$ by $r+\epsilon \eta$ with $\eta (a)=\eta (b)=0$. The first variation of the functional isThe boundary term from integration by parts is zero. The fundamental lemma of the calculus of variations therefore gives $\boxed{F_{1,r}-d{F}_{1,r^{\prime }}/dz=0}$. For $I_2$, independently vary $r$ and $\theta$ with zero endpoint variations; the two Euler-Lagrange equations areFor the optical path, Fermat principle makes $\int \mu ds$ stationary. In cylindrical coordinates its integrand is $L=\mu (r,\theta )s$, with $s=\sqrt{1+r^{\prime 2}+r^2{\theta }^{\prime 2}}$. There is no explicit $z$ dependence. The Beltrami identity for several dependent variables givesThe conserved quantity is the axial component of the optical momentum of a ray. As $\mu >0$, squaring and inverting yieldsNow suppose $\mu =\mu (r)$ and seek a path of radius $R>0$. The angle is a cyclic coordinate, so $L_{{\theta }^{\prime }}=\mu R^2{\theta }^{\prime }/\sqrt{1+R^2{\theta }^{\prime 2}}$ is conserved. At fixed radius this is strictly increasing in ${\theta }^{\prime }$, forcing ${\theta }^{\prime }=\omega$ constant. The radial Euler-Lagrange equation reduces toConsequently the helical rays in a radially graded refractive index haveA real nonzero helix exists at such a radius when $0<-R{\mu }^{\prime }(R)/\mu (R)<1$, and its helical pitch is $2\pi /|\omega |$. If ${\mu }^{\prime }(R)=0$, the solution is instead a straight axial light ray. Thus radial dependence permits constant-radius rays for suitable profiles; an arbitrary radial profile need not support a nonzero real helix. This qualification is necessary, for example when the refractive index increases with radius.