 Solution

The stochastic objective is interpreted as its expectation; a literal pathwise minimization is generally impossible without knowing future noises. Let the value function from time $t$ be $V_t(x)=P_t{x}^2+d_t$, with $P_T=1,d_T=0$. Independence from past observations and zero contemporaneous covariance giveThe Bellman equation is $V_t(x)=x^2+{\min }_u\{c{u}^2+P_{t+1}{V}_{\xi }(Ax+u{)}^2+P_{t+1}{V}_{ϵ}+d_{t+1}\}$. Completing the square yieldsThis feedback is optimal among all controls measurable from the specified past, not merely linear controls. It depends on $V_{\xi }$ and does not depend on $V_{ϵ}$, though the minimal cost does. There is no cost-relevant $u_T$ in the displayed objective.

The backward Riccati recurrence iterates $F(P)=1+A^{2}c{V}_{\xi }P/(c+V_{\xi }P)$ from $1$. It is increasing, maps into $[1,1+A^{2}c]$, and has exactly one positive fixed point. Consequently it converges toAs $T\to \infty$, the finite-horizon feedback at any fixed time tends to this feedback. The closed-loop multiplier is $Ac/(c+V_{\xi }P)$, and its squared-mean factor $\alpha =V_{\xi }{A}^2{c}^2/(c+V_{\xi }P{)}^2$ is less than one, since $(P-1)/P=A^{2}c{V}_{\xi }/(c+V_{\xi }P)$ and $\alpha =(P-1)c/[P(c+V_{\xi }P)]$. Thus the second moment tends to $V_{ϵ}/(1-\alpha )$. The finite-horizon value has $d_1=V_{ϵ}{\sum }_{t=1}^{T-1}{P}_{t+1}$; its average converges by Cesaro summation. For a fixed initial state of finite second moment,The same average is attained by the stationary feedback, by the stationary Bellman equation and bounded second moments.