Past exam of the mathematics course of the University of Cambridge / 2008 / ii / Paper 3 / 28I / Solution

Completing the square givesSince $Q$ is positive-definite, the unique minimizer is $\boxed{a=-Q^{-1}Sx}$ and the minimum is $\boxed{x^T(R-S^T{Q}^{-1}S)x}$.

The Bellman equation is $V(n,x)=x^T{\Pi }_{0}x$ andSuppose $V(k+1,z)=z^T{\Pi }_{r}z+{\gamma }_{k+1}$, where $r=n-k-1$. Zero noise mean gives the expected quadratic $(Ax+Ba{)}^T{\Pi }_r(Ax+Ba)+\mathrm{tr}({\Pi }_r{N}_{k+1})$. Complete the square again, with $Q_r=Q+B^T{\Pi }_{r}B$ and $S_r=S+B^T{\Pi }_{r}A$. This yields the discrete Riccati recurrenceThe matrices $Q_r$ are positive definite. The minimum of the nonnegative stage-plus-future quadratic is nonnegative, so each ${\Pi }_r$ is symmetric positive semidefinite. Starting from ${\gamma }_n=0$, the constants obeyInduction proves $V(k,x)=x^T{\Pi }_{n-k}x+{\gamma }_k$, with optimal feedback control $U_k=K_{n-k-1}{X}_k$.