= Solution
<Completing the square> gives
$$
c(x,a)=(a+Q^{-1}Sx)^TQ(a+Q^{-1}Sx)+x^T(R-S^TQ^{-1}S)x.
$$
Since $Q$ is <positive-definite>, the unique minimizer is $\boxed{a=-Q^{-1}Sx}$ and the minimum is $\boxed{x^T(R-S^TQ^{-1}S)x}$.
The <Bellman equation> is $V(n,x)=x^T\Pi_0x$ and
$$
V(k,x)=\inf_a\{c(x,a)+\mathbb E[V(k+1,Ax+Ba+\epsilon_{k+1})]\}.
$$
Suppose $V(k+1,z)=z^T\Pi_rz+\gamma_{k+1}$, where $r=n-k-1$. Zero noise mean gives the expected quadratic $(Ax+Ba)^T\Pi_r(Ax+Ba)+\operatorname{tr}(\Pi_rN_{k+1})$. Complete the square again, with $Q_r=Q+B^T\Pi_rB$ and $S_r=S+B^T\Pi_rA$. This yields the <discrete Riccati recurrence>
$$
\boxed{\Pi_{r+1}=R+A^T\Pi_rA-S_r^TQ_r^{-1}S_r,\qquad K_r=-Q_r^{-1}S_r.}
$$
The matrices $Q_r$ are positive definite. The minimum of the nonnegative stage-plus-future quadratic is nonnegative, so each $\Pi_r$ is symmetric <positive semidefinite>. Starting from $\gamma_n=0$, the constants obey
$$
\boxed{\gamma_k=\gamma_{k+1}+\operatorname{tr}(\Pi_{n-k-1}N_{k+1})=\sum_{j=k}^{n-1}\operatorname{tr}(\Pi_{n-j-1}N_{j+1}).}
$$
Induction proves $V(k,x)=x^T\Pi_{n-k}x+\gamma_k$, with optimal <feedback control> $U_k=K_{n-k-1}X_k$.
Back to article page
