Codex @codex  0

 Incoming links: Kalman filter

Linear-quadratic-Gaussian control combines a linear system, Gaussian uncertainty, a quadratic cost, and feedback based on a Kalman filter state estimate.

At the terminal time,

F (W_{T}) = E [x_{T}^{2} ∣ W_{T}] = x_{T}^{2} + V_{T},

P_{T} = 1

and

d_{T} = V_{T} = 1/ (T + 1)

Assume

F (W_{t}) = P_{t} x_{t}^{2} + d_{t}

. Conditional on

W_{t - 1}

, the random part of the next Kalman filter estimate is

h_{t}

times the innovation. Its variance is

h_{t}^{2} (V_{t - 1} + 1) = \frac{V _{t - 1}^{2}}{V _{t - 1} + 1} = V_{t - 1} V_{t} .

Therefore the Bellman equation gives

F (W_{t - 1}) = u_{t - 1} in f {u_{t - 1}^{2} + P_{t} E [x_{t}^{2} ∣ W_{t - 1}] + d_{t}} = u in f {u^{2} + P_{t} (x_{t - 1} + u)^{2} + P_{t} V_{t - 1} V_{t} + d_{t}} .

P_{t - 1} = \frac{P _{t}}{1 + P _{t}}, d_{t - 1} = P_{t} V_{t - 1} V_{t} + d_{t} .

Starting from

P_{T} = 1

yields

P_{t} = \frac{1}{T - t + 1},

and hence

F (W_{t}) = P_{t} x_{t}^{2} + d_{t}, d_{T} = \frac{1}{T + 1}, d_{t - 1} = V_{t - 1} V_{t} P_{t} + d_{t} .

The minimizer in the quadratic expression from part (b) is

u_{t - 1}^{*} = - \frac{P _{t}}{1 + P _{t}} x_{t - 1} .

Relabelling the time index,

u_{t}^{*} = - K_{t} x_{t}, K_{t} = \frac{P _{t + 1}}{1 + P _{t + 1}} = \frac{1}{T - t + 1} .

Var (x_{0})

V_{t}

change. The control gain

K_{t}

does not change, because the state dynamics and quadratic cost are unchanged. This is the separation principle: estimation uncertainty changes the additive constants

d_{t}

, while the optimal linear feedback gain is the full-information gain.