Linear-quadratic optimal control (source code)

= Linear-quadratic optimal control

A standard finite-horizon linear-quadratic problem uses <linear> dynamics $\dot x=Ax+Bu$ and minimizes
$$
x(T)^TQ_fx(T)+\int_t^T[x(s)^TQx(s)+u(s)^TRu(s)]\,ds,
$$
where $Q,Q_f$ are symmetric <positive semidefinite matrices> and $R$ is <positive-definite>. The <control input> is unrestricted in Euclidean space. These hypotheses, rather than the word quadratic alone, make the control minimization well posed.

The <value function> has the <quadratic form> $V(t,x)=x^TP(t)x$. Substitution into the <Hamilton-Jacobi-Bellman equation> and completion of the control square give
$$
u_*=-R^{-1}B^TPx,\qquad-\dot P=A^TP+PA-PBR^{-1}B^TP+Q,\qquad P(T)=Q_f.
$$
Thus a backward matrix <Riccati equation> gives a linear <feedback control>. <Underactuated Robotics by Russ Tedrake>[Underactuated Robotics by Russ Tedrake, chapter8, Finite-horizon LQR]{ref}

For the scalar system $\dot x=u$, terminal cost zero and running cost $x^2+u^2$, the solution is $P(t)=\tanh(T-t)$ and $u_*=-\tanh(T-t)x$. Differentiation verifies $-\dot P=1-P^2$. Infinite-horizon formulas need additional stabilizability and detectability conditions; they do not follow from an arbitrary quadratic cost.