Past exam of the mathematics course of the University of Cambridge / 2018 / ii / Paper 3 / 30K (Optimisation and Control)

Suppose inductively thatThe observation innovation isso its conditional variance is $V_{t-1}+1$ and its conditional covariance with $x_{t-1}$ is $V_{t-1}$. The scalar static-state Kalman update therefore givesAdding the known control $u_{t-1}$ shifts the conditional mean but not the variance, giving exactlySince $V_0=1$, induction yieldsEvery update is an affine conditioning operation on jointly Gaussian random variables, so

At the terminal time,so $P_T=1$ and $d_T=V_T=1/(T+1)$.

Assume $F(W_t)=P_t{\hat{x}}_t^2+d_t$. Conditional on $W_{t-1}$, the random part of the next Kalman filter estimate is $h_t$ times the innovation. Its variance isTherefore the Bellman equation givesCompleting the square givesStarting from $P_T=1$ yieldsand hence

The minimizer in the quadratic expression from part (b) isRelabelling the time index,

If $\mathrm{Var}(x_0)$ or the observation-noise variances change, the Kalman filter gains and the sequence $V_t$ change. The control gain $K_t$ does not change, because the state dynamics and quadratic cost are unchanged. This is the separation principle: estimation uncertainty changes the additive constants $d_t$, while the optimal linear feedback gain is the full-information gain.