= Solution
The stochastic objective is interpreted as its <expectation>; a literal pathwise minimization is generally impossible without knowing future noises. Let the value function from time $t$ be $V_t(x)=P_t x^2+d_t$, with $P_T=1,d_T=0$. Independence from past observations and zero contemporaneous <covariance> give
$$
\mathbb E[x_{t+1}^2\mid x_t=x,u_t=u]=V_\xi(Ax+u)^2+V_\epsilon.
$$
The <Bellman equation> is $V_t(x)=x^2+\min_u\{cu^2+P_{t+1}V_\xi(Ax+u)^2+P_{t+1}V_\epsilon+d_{t+1}\}$. Completing the square yields
$$
\boxed{u_t^*(x)=-\frac{A V_\xi P_{t+1}}{c+V_\xi P_{t+1}}x,\quad
P_t=1+\frac{A^2c V_\xi P_{t+1}}{c+V_\xi P_{t+1}},\quad
d_t=d_{t+1}+P_{t+1}V_\epsilon.}
$$
This feedback is optimal among all controls measurable from the specified past, not merely linear controls. It depends on $V_\xi$ and \b[does not depend on $V_\epsilon$], though the minimal cost does. There is no cost-relevant $u_T$ in the displayed objective.
The backward <Riccati recurrence> iterates $F(P)=1+A^2cV_\xi P/(c+V_\xi P)$ from $1$. It is increasing, maps into $[1,1+A^2c]$, and has exactly one positive <fixed point>. Consequently it converges to
$$
\boxed{P=\frac{-b+\sqrt{b^2+4cV_\xi}}{2V_\xi},\quad
b=c-V_\xi-A^2cV_\xi,\qquad
u^*(x)=-\frac{A V_\xi P}{c+V_\xi P}x.}
$$
As $T\to\infty$, the finite-horizon feedback at any fixed time tends to this feedback. The closed-loop multiplier is $A c/(c+V_\xi P)$, and its squared-mean factor $\alpha=V_\xi A^2c^2/(c+V_\xi P)^2$ is less than one, since $(P-1)/P=A^2cV_\xi/(c+V_\xi P)$ and $\alpha=(P-1)c/[P(c+V_\xi P)]$. Thus the second moment tends to $V_\epsilon/(1-\alpha)$. The finite-horizon value has $d_1=V_\epsilon\sum_{t=1}^{T-1}P_{t+1}$; its average converges by Cesaro summation. For a fixed initial state of finite second moment,
$$
\boxed{\lim_{T\to\infty}\frac1T\inf\mathbb E[\text{total cost}]=P V_\epsilon.}
$$
The same average is attained by the stationary feedback, by the stationary <Bellman equation> and bounded second moments.
Back to article page
