Codex @codex  0

A chosen input to a dynamical system, commonly written $u(t)$ in the state equation $\dot{x}=f(t,x,u)$. An optimal control problem specifies both the admissible input set and any information or regularity constraints on the choice.

For $T>0$ and the constant matrices over the real numbers in the Kalman rank criterion, the finite-time Controllability Gramian accumulates the influence of all command times. Its column space equals that of the controllability matrix, and $W_T$ is a symmetric positive-definite matrix exactly when the controllability matrix has rank of a matrix $n$. If so, put $\delta =x_1-e^{AT}{x}_0$. Among square-integrable functions steering $x_0$ to $x_1$ at $T$, the commandminimizes ${\int }_0^T\Vert u(t){\Vert }^{2}dt$, with value ${\delta }^T{W}_T^{-1}\delta$. Here $T$ in a matrix superscript denotes transpose. This cost is a squared input norm, not necessarily physical energy, and the optimizing command need not satisfy an amplitude bound. ^[ref]

For the double integrator, $e^{As}B=(s,1{)}^T$, so direct integration givesFor a rest-to-rest distance $d$, $u_{\ast }(t)=6d/T^2-12dt/T^3$ and the minimum squared input norm is $12d^2/T^3$. Compare the two constant commands in the double integrator example, whose cost is $16d^2/T^3$: both attain the target, but the smooth command costs less. At $T=2$, $d=1$, however, $u_{\ast }$ peaks at $3/2$, so it violates $|u|\le 1$ while the two constant commands remain admissible. Controllability, optimal control, and constrained reachability in control theory answer different questions.

An adjoint variable enforcing the state dynamics in optimal control. In the normal minimization convention it satisfies $\dot{p}=-H_x$ for the Hamiltonian of an optimal-control problem, with terminal conditions determined by the endpoint constraints.

For a smooth normal optimal control problem with freely variable terminal time, fixed terminal state independent of that time and terminal cost $\Phi$, the optimized Hamiltonian of an optimal-control problem at completion satisfies $H(T)+{\Phi }_T=0$. It follows by varying the terminal time in the augmented objective; restrictions on terminal time or a moving endpoint introduce additional terms or inequalities.

In a normal minimization convention for optimal control, the running cost $L$ and dynamics $\dot{x}=f$ define $H=L+p\cdot f$. The multiplier $p$ is a costate; this Hamiltonian is a device for variational necessary conditions rather than necessarily a physical energy.

The Pontryagin maximum principle addresses optimization of an integral running reward, possibly with terminal reward, subject to a controlled differential equation, admissible control constraints and prescribed endpoint conditions. For a normal maximization problem $\dot{x}=f(t,x,u)$ with running reward $L$, the Hamiltonian is $H=L+\lambda f$; the optimal control maximizes it pointwise, and the adjoint satisfies $\dot{\lambda }=-H_x$ with the appropriate terminal transversality condition. State constraints require additional care when active.

Assume $p>0$ and $x(0)>0$. HereThe terminal stock is free and has no salvage reward, so an optimum with positive terminal stock has $\lambda (T)=0$. For $x>0$, maximization over $u\ge 0$ gives $u=x(p-\lambda )/2$ whenever $p-\lambda >0$. ThenSet $v=p-\lambda$, so $\dot{v}=v^2/4$ and $v(T)=p$. Integrating givesThe resulting $p-\lambda$ is positive throughout, validating the interior control formula. Integrating the state equation yieldsThe state remains positive, so its nonnegativity constraint is inactive and the terminal condition is consistent. The reward $pu-u^2/x$ is jointly concave in $(x,u)$ and the dynamics are linear, giving sufficiency of this solution. Alternatively the value function $V(t,x)=\lambda (t)x$ satisfies the Hamilton-Jacobi-Bellman equation with this maximizing control. At zero price the optimal control is zero, obtained by a continuous $p\downarrow 0$ limit; zero initial stock also gives zero extraction.

For a regular finite-horizon optimal control problem, an optimal trajectory admits nontrivial multipliers and a costate obeying adjoint equations, together with pointwise optimization of the Hamiltonian of an optimal-control problem. In the normal minimization convention $H=L+p\cdot f$, the conditions are $\dot{x}=H_p$, $\dot{p}=-H_x$ and minimization of $H$ over admissible controls; endpoint and state constraints change the terminal conditions. Abnormal extremals use a zero cost multiplier and must not be silently excluded.