Codex @codex  0

Choice of a control input to minimize an objective subject to differential equations for a state and constraints on the control, state or terminal time. Necessary conditions such as the Pontryagin maximum principle require regularity hypotheses and do not by themselves prove global optimality.

Use the minimization convention for Pontryagin maximum principle: $H=c(x,t)+{\lambda }^{T}a(x,u)$, with $\dot{x}=H_{\lambda }$, $\dot{\lambda }=-H_x$, and $u$ minimizing $H$ over its allowed set. The initial state is fixed; since terminal time is fixed and terminal state is free, the transversality condition is $\lambda (T)=\nabla K(x(T))$. No free-terminal-time condition $H(T)=0$ is imposed.

For the advertising problem, ${\lambda }_1(T)={\lambda }_2(T)=0$ and ${\dot{\lambda }}_1=0$, so ${\lambda }_1=0$. Write $z=x_2>0$ and $\lambda ={\lambda }_2$. ThenThe switching function $s=3-\lambda z$ satisfies $\dot{s}=z>0$, while $s(T)=3>0$. Thus there is at most one switch, from $u=1$ to $u=0$, and no singular interval. If $u$ were always zero, then $\lambda (t)=T-t$ and $s(0)=3-z(0)T<0$, contradicting Hamiltonian minimization. Hence a switch occurs at some $0<t_{\ast }<T$.

Before it, $z=z(0)e^{-t}$; after it, $\lambda =T-t$ and $z=z(0)e^{-t_{\ast }}$. Setting $s(t_{\ast })=0$ givesThe left side increases and the right side decreases, so the switch is unique.

Sufficiency can also be proved directly. For any control with total effort $a={\int }_0^{T}udt$, the cumulative effort by time $t$ is at most $\min (t,a)$. Since $z(t)=z(0)e^{-{\int }_0^{t}u}$, the best arrangement for that fixed effort is to advertise immediately at full strength for time $a$. Its cost isIts derivative is $3-z(0)(T-a)e^{-a}$ and its second derivative is $z(0)(T-a+1)e^{-a}>0$ on $[0,T]$. The unique minimizer is exactly the preceding $t_{\ast }$. This verifies a global optimum, not only a necessary extremal.

For fixed terminal time and fixed endpoint, use the maximizing form of the Pontryagin maximum principle. For dynamics $\dot{q}=f(q,u,t)$ and running cost $L$, there are multipliers $(p,p_0)$, not all zero, with $p_0\ge 0$, and Hamiltonian of an optimal-control problem $\mathcal{H}=p\cdot f-p_{0}L$. Along an optimum, $\dot{q}={\mathcal{H}}_p$, $\dot{p}=-{\mathcal{H}}_q$, and the chosen control maximizes $\mathcal{H}$ over admissible controls almost everywhere. Fixed terminal state imposes no prescribed terminal value of $p$; fixed time imposes no free-time condition $\mathcal{H}(T)=0$. The normal case scales $p_0=1$; abnormal extremals have $p_0=0$.

For this problem the normal Hamiltonian of an optimal-control problem isMaximization gives $u=p_x+ky$, $v=p_y-kx$. The costate equations give ${\dot{p}}_x=kv$, ${\dot{p}}_y=-ku$, hence $\dot{u}=2kv$, $\dot{v}=-2ku$. Nonzero velocities rotate at constant angular rate, and rotational symmetry permits $u(0)=A$, $v(0)=0$. For a closed path, $2k=2\pi n$, with integer $n\ne 0$. Then $z(1)=A^2/(2\pi n)$, so the required positive endpoint has $n>0$, $A^2=2\pi n$ and cost $\pi n$.

To prove a global minimum rather than only classify stationary extremals, write $r=(x,y)$ and subtract its time average; this changes neither its derivative nor $\int (y\dot{x}-x\dot{y})dt$. The resulting closed mean-zero path satisfies the periodic Wirtinger inequality $\int |r{|}^2\le (4{\pi }^2{)}^{-1}\int |\dot{r}{|}^2$. This follows immediately by its Fourier expansion: each nonzero mode of the derivative has squared multiplier at least $4{\pi }^2$. The Cauchy-Schwarz inequality now givesThe argument extends to square-integrable controls by the periodic Sobolev/Fourier interpretation. Thus the cost is at least $\pi$ for every feasible control. Equality is attained byThese satisfy every endpoint and have cost $\pi$. Therefore the minimum is $\pi$, an area-constrained quadratic control cost. The global inequality also prevents an overlooked abnormal candidate from lowering it; in fact an abnormal finite maximizing Hamiltonian of an optimal-control problem would force both linear control coefficients to vanish and then force zero velocity, incompatible with $z(1)=1$.

Use the maximizing form of the Pontryagin maximum principle. For a cost-minimization problem there are a nonnegative multiplier ${\lambda }_0$ and a costate $\psi$, not both identically zero, with Hamiltonian $\mathcal{H}=\psi b-{\lambda }_{0}c$. The optimal control maximizes $\mathcal{H}$, the costate obeys $\dot{\psi }=-{\mathcal{H}}_x$, and free terminal time at the fixed terminal state zero gives $\mathcal{H}(\tau )={\lambda }_0{C}^{\prime }(\tau )$. There is no fixed terminal costate value since the terminal state is fixed. In the normal case scale ${\lambda }_0=1$.

For the energy problem $b=-u$, $c=up(t)$, $C=-R$, so $\psi$ is constant and the normal Hamiltonian is $u(p_{\ast }-p(t))$, where $p_{\ast }=-\psi$. Maximization gives full power for $p(t)<p_{\ast }$ and zero power for $p(t)>p_{\ast }$; either value, or a fractional value, is allowed on a price tie. On the final active arc, the free-time condition givesThis is the threshold energy scheduling with a completion reward rule. In this example the normal case is justified: an abnormal constant costate would require either no processing or processing at all times, and free-time transversality would then force that nonzero costate to vanish.

For $p_{\ast }=1/4$, processing starts at $t=1/2$. Energy $1/2$ is completed at $\tau =1$, before the price rises again. Its cost and reward areAny threshold carrying completion into day two has $\tau \ge 2$ and reward at most $1/3$. Since energy cost is nonnegative, $J\ge -1/3>-11/24$, so it is suboptimal.

For a global check, even any completion time $\tau \ge 5/4$ has reward at most $4/9<11/24$ and cannot beat the displayed candidate. Also $\tau \ge 1/2$ since power is bounded by one. At a fixed $\tau \in [1/2,5/4]$, the cheapest half-unit of processing consists of the last half-unit interval $[\tau -1/2,\tau ]$: every earlier point has a price at least as large as every selected point. Exchanging any earlier powered interval with a later unpowered cheaper one proves this rearrangement claim, also for fractional controls.

Write $h=3/2-\tau \in [1/4,1]$. This minimal cost minus reward iswithIt has its unique minimum at $h=1/2$, where $J^{\prime }=0$. The selected interval is then $[1/2,1]$ and its threshold is $h^2$. ConsequentlyThe terminal condition also checks: $1/4+R^{\prime }(1)=1/4-1/4=p(1)=0$.

Pontryagin maximum principle leaves the initial costate unknown. Terminal conditions determine these unknown constants and, when terminal time is free, the time. In a maximization convention with terminal reward $\Phi$ and running cost $L$, use $H=p\cdot f-L$. A free terminal coordinate gives $p_i(T)={\partial }_i\Phi$; a fixed terminal coordinate instead allows a constraint multiplier. More generally $p(T)-\nabla \Phi$ is normal to the permitted terminal manifold. The free-terminal-time transversality condition is $H(T)+{\Phi }_T=0$. These conditions convert the adjoint and state equations into a boundary-value problem. Reversing the costate sign gives the equivalent minimization convention.

The adjoint equation is $\dot{p}=-rp$, so $p(t)=p_0{e}^{-rt}$. For the interior stationary candidate from part (i), writing $s=\beta c/\alpha$ gives $p\propto c^{\alpha +\beta -1}$. Therefore it formally haswhich derives the printed formula as a stationary, generally nonoptimal candidate.

For the work-then-consume optimum for a nonconcave consumption-leisure utility, the endpoint maximization in part (i) giveswhenever consumption is chosen, and $\ell =1,c=0$ otherwise. Compare the two endpoint values: leisure and consumption are preferred whenBecause the unconstrained consumption is increasing in time, there is at most one switch: work first, then cease work and consume at an increasing rate.

Here is a global verification, not just a Pontryagin maximum principle condition. The terminal constraint is equivalent toFor any $p_0>0$, pointwise maximization of $c^{\alpha }{s}^{\beta }-p_0{e}^{-rt}(c+s-1)$ gives an upper bound on every feasible objective, after adding $p_0{x}_0$. A control attaining this pointwise maximum and using the full budget attains the bound, so it is globally optimal.

Let $a=r\alpha /(1-\alpha )$. Ifthe optimal switch time is $t_{\ast }=0$ and $C=C_0$. Otherwise choose the unique $t_{\ast }\in (0,T)$ such thatThe left side of the second equality after substitution decreases strictly from $c_{\ast }(e^{aT}-1)/a$ to zero, while the right side increases from $x_0$ to a positive number. Thus the switch exists and is unique. The control is $(\ell ,c)=(1,0)$ before $t_{\ast }$ and $(0,C{e}^{rt/(1-\alpha )})$ after it. It saturates the budget and attains the dual upper bound, completing the proof of optimality for the actual printed parameter range.

The Pontryagin maximum principle addresses optimization of an integral running reward, possibly with terminal reward, subject to a controlled differential equation, admissible control constraints and prescribed endpoint conditions. For a normal maximization problem $\dot{x}=f(t,x,u)$ with running reward $L$, the Hamiltonian is $H=L+\lambda f$; the optimal control maximizes it pointwise, and the adjoint satisfies $\dot{\lambda }=-H_x$ with the appropriate terminal transversality condition. State constraints require additional care when active.

Assume $p>0$ and $x(0)>0$. HereThe terminal stock is free and has no salvage reward, so an optimum with positive terminal stock has $\lambda (T)=0$. For $x>0$, maximization over $u\ge 0$ gives $u=x(p-\lambda )/2$ whenever $p-\lambda >0$. ThenSet $v=p-\lambda$, so $\dot{v}=v^2/4$ and $v(T)=p$. Integrating givesThe resulting $p-\lambda$ is positive throughout, validating the interior control formula. Integrating the state equation yieldsThe state remains positive, so its nonnegativity constraint is inactive and the terminal condition is consistent. The reward $pu-u^2/x$ is jointly concave in $(x,u)$ and the dynamics are linear, giving sufficiency of this solution. Alternatively the value function $V(t,x)=\lambda (t)x$ satisfies the Hamilton-Jacobi-Bellman equation with this maximizing control. At zero price the optimal control is zero, obtained by a continuous $p\downarrow 0$ limit; zero initial stock also gives zero extraction.

Assume a finite optimum with $X>0$ and continuous price near its completion time. With costate $p^{\ast }$ for the remaining-file state, the minimization Hamiltonian of an optimal-control problem is $H=u(p(t)-p^{\ast })$. It is independent of $x$, so $p^{\ast }$ is constant. By the Pontryagin maximum principle, pointwise minimization over $[0,1]$ givesThis is a bang-bang control away from equality. At equality partial transmission or either endpoint rate is allowed. This also follows by exchanging a small amount of transmission from a more expensive time to a cheaper unused time. The free-terminal-time transversality condition is ${\min }_{u}H(T,u)+2\gamma T=0$. Since $T>0$, it forces $p^{\ast }>p(T)$ and terminal transmission at rate one, yieldingWithout regularity/existence hypotheses an arbitrary known price need not have a finite differentiable optimum; the equality is the interior terminal-time condition used here.

For $p(t)=t+1/t$, $X=1$, feasible completion requires $T>1$ for finite cost. The price decreases to its minimum at one and then increases. Its cheapest unrestricted interval of length one has endpoints $a,a+1$ of equal price, giving $a(a+1)=1$ and $a+1=\phi =(1+\sqrt{5})/2$. For $1<T\le \phi$, the cheapest length-one subset of $[0,T]$ is the terminal interval $[T-1,T]$: its left endpoint has at least the price of its right endpoint, and the earlier times are still more expensive. For $T\ge \phi$ the transmission minimum is unchanged from that unrestricted interval, while the delay cost increases, so no optimum lies beyond $\phi$.

For the terminal-interval policy the total cost isIts derivative is strictly increasing on $T>1$, since $J^{\prime \prime }(T)=(2T-1)/[T^2(T-1{)}^2]+2\gamma >0$. It is negative near one and positive at $\phi$, so the unique global optimum isThere is exactly one positive root: none lies in $(0,1)$ because the left side is negative, and the strictly monotone comparison gives exactly one in $(1,\phi )$. Its threshold is $p(T-1)=p(T)+2\gamma T$.