Codex @codex  0

Causal state filtering estimates the current state of a dynamical system from observations up to the current time. Under quadratic loss, the conditional expectation is an optimal estimate. A Kalman filter obtains this estimate recursively for a linear Gaussian state-space model, without using future observations.

In a linear Gaussian state-space model with process noise covariance $Q$ and exact scalar observation $X_t=F{S}_t$, the predictive state covariance includes $Q$ even though there is no separate measurement noise. Here $R_t=G{P}_{t-1}{G}^{\top }+Q$ denotes predicted state covariance; it differs from the measurement-noise notation in the general Kalman filter equations. The observation innovation variance is $V_t=FG{P}_{t-1}{G}^{\top }{F}^{\top }+FQ{F}^{\top }$. Omitting the second term incorrectly removes the current process noise. If $V_t>0$, Gaussian conditional expectation gives the posterior covariance matrix $P_t=R_t-R_t{F}^{\top }F{R}_t/V_t$. This can be a singular matrix because the observed linear combination is then known exactly.

For a concrete example with two state components that are dimensionless quantities, take predicted mean $(10,0{)}^{\top }$, predicted covariance matrixand exact observation of the first component, $F=(1,0)$, $X=12$. The innovation is $2$, its variance is $4$, and the Kalman gain is $(1,1/2{)}^{\top }$. The posterior mean and posterior covariance matrix are thereforeThe observed component now has zero variance. The second component remains uncertain, but its posterior mean changes and its variance falls from $9$ to $8$ because its prediction error had nonzero correlation with that of the first. This is the mechanism by which measurements of one component can inform another.

In a linear Gaussian state-space model, let the predicted state mean and covariance be $m_t^{-},P_t^{-}$. The observation innovation process has conditional mean zero and covariance ${\Sigma }_t=A{P}_t^{-}{A}^{\top }+R$, with innovation ${ϵ}_t=y_t-A{m}_t^{-}$. The conditional multivariate normal distribution gives the predictive observation density. Multiplying these conditional densities by the chain rule gives the displayed likelihood function; equivalently the innovations are independent Gaussian vectors. The Kalman filter supplies $m_t^{-},P_t^{-}$ recursively, so one evaluates the exact observed-data likelihood function without integrating all latent states jointly.

For the Kalman filter model, the innovation is $e_t=y_t-H_t{m}_t^{-}$. Its conditional covariance matrix before the measurement isThis includes uncertainty in both the predicted state and the measurement noise. It is a covariance in observation coordinates; $P_t^{-}$ is a covariance in state coordinates, and need not even have the same dimensions.

The Kalman filter alternates a Kalman prediction step with a Kalman measurement update. It carries a state estimate forward using a state-space model, then adjusts it by the part of the new observation that was not predicted: the innovation. The Kalman gain determines the adjustment from the uncertainty of the prediction and the measurement noise. A noisy measurement should have less influence than an equally surprising precise one.

For a precise discrete-time statement, let the state and observation be vectors in real vector spaces obeying the linear state-space modelThe matrices and control inputs are known. Assume a Gaussian random vector $x_0$ with mean $m_0$ and covariance matrix $P_0$, and mutually independent centered Gaussian noises $w_t,v_t$, independent of $x_0$, with covariance matrices $Q_t,R_t$. Suppose $R_t$ is positive-definite, so the observation innovation covariance is invertible. Then the recursions below give the exact posterior mean $m_t=\mathbb{E}[x_t\mid y_1,\dots ,y_t]$ and posterior covariance matrix $P_t$, hence the full conditional multivariate normal distribution. They also minimize mean squared error. Without the Gaussian assumptions, suitable second-order assumptions still give best affine state estimates, but do not generally give an exact posterior distribution. ^[ref].

For an exact scalar example, suppose the predicted state has normal distribution $N(10,4)$ and a measurement reports $y=14$ in the same units. With measurement noise variance $R=1$, the Kalman gain is $4/(4+1)=4/5$ and the posterior distribution isWith the same prediction and observed value but measurement noise variance $R=16$, the Kalman gain becomes $1/5$ and the posterior distribution is instead $N(10.8,3.2)$. Both updates move toward the observation; the more precise measurement moves the estimate further and reduces uncertainty more.

The useful compression is exact under the stated model: the two objects $m_t,P_t$ contain everything from the past observations needed for the next update. Their size depends on the state dimension, not on the length of the observation history. This makes causal state filtering practical when observations arrive continually. It also gives Linear-quadratic-Gaussian control an estimate of the state on which to act.

Rudolf E. Kalman's 1960 paper organized linear filtering around state transitions, error covariance and recursive computation, connecting estimation with optimal regulation. Its introduction acknowledges earlier work by Wiener, Bode, Shannon and others; the useful historical claim is this particular formulation, rather than that estimation began in 1960. ^[ref].

Consider a local-level state-space model $x_t=x_{t-1}+w_t$, $y_t=x_t+v_t$, with mutually independent centered Gaussian noises of variances $Q=1/4$ and $R=1$. Start with $x_0\sim N(10,4)$, independently of these noises. Before the first observation, the Kalman prediction step gives $m_1^{-}=10$, $P_1^{-}=17/4$. If $y_1=14$, the Kalman gain is $17/21$, and the posterior mean and posterior variance becomeThe next Kalman prediction step keeps the mean $m_2^{-}=278/21$ but adds fresh process noise variance, giving $P_2^{-}=89/84$. If $y_2=12$, thenThus new uncertainty is introduced at each transition and reduced by each measurement. It does not shrink to zero merely because many measurements have arrived: the state continues to change.

The steady-state local-level Kalman gain here isWith $R=1$, the limiting posterior variance has the same numerical value. By contrast, the static-state model $Q=0$ with $P_0=4$ gives $P_t=1/(t+1/4)$ and $K_t=1/(t+1/4)$. That decreasing gain is appropriate for a state that truly remains constant. Applied to the changing random walk, it builds confidence in the wrong model. The choice of process noise is therefore a statement about state evolution, rather than just an instruction to make a plot look smoother.

Linear-quadratic-Gaussian control combines a linear system, Gaussian uncertainty, a quadratic cost, and feedback based on a Kalman filter state estimate.

For the three-coordinate state realization of an ARMA(1,1) process, the observation row and transition matrix areThe first row of $G{S}_{t-1}$ is $x_{t-1}$ by the previous-time recursion; the second row is zero until the new white noise value is added; and the third row copies ${\epsilon }_{t-1}$. Hence the state equation has noise covariance

We use the causal state model, in which the new driving noise is independent of the previous state. The Gaussian filtering with exact observations form of the Kalman filter alternates prediction through the state equation with conditioning on the next observation. Given the posterior expected value and covariance at time $t-1$, independence of the new white noise value gives predictive expected value $a_t=G{\hat{S}}_{t-1}$ and predictive covarianceProject through the observation row to getThe residual $v_t=x_t-{\hat{x}}_t$ is the innovation. Conditioning the joint Gaussian state-observation vector adjusts the expected value according to this residual and reduces the covariance. In this exact-observation model, the updates can be writtenThus $F{\hat{S}}_t=x_t$ and $F{P}_t{F}^{\top }=0$: the observed linear combination of the state is now known exactly, though other state combinations remain uncertain.

The PDF's stated prediction variance omits $FQ{F}^{\top }={\sigma }^2$. This omission cannot be used in a valid likelihood: for the AR(1) specialization below it would give zero prediction variance after the first observation. The required likelihood calculation uses the corrected innovation variance just derived.

For stationary initialization of an ARMA(1,1) state with $|\varphi |<1$, take ${\hat{S}}_0=0$ and the unconditional stationary covariance. The causal coefficients are $1$ at lag zero and $(\varphi +\theta ){\varphi }^{j-1}$ at lags $j\ge 1$, givingSince $S_0=(x_{-1},{\epsilon }_0,{\epsilon }_{-1}{)}^{\top }$, the cross-covariance with ${\epsilon }_0$ is zero and that with ${\epsilon }_{-1}$ is ${\sigma }^2$. ThereforeDirect substitution verifies $P_0=G{P}_0{G}^{\top }+Q$. If prior state information is available, instead use its conditional expected value and covariance. A diffuse initialization is another option when the initial state is genuinely unknown, but it changes the finite-sample likelihood and should not silently replace the stationary initial law.

By the Gaussian prediction step,The chain rule for probabilities for the joint probability density function therefore givesTaking minus twice its logarithm shows that maximum likelihood estimation is equivalent to minimizingwith each $V_t$ and ${\hat{x}}_t$ evaluated at the candidate parameters and the specified initialization. The corrected $+{\sigma }^2$ is essential in every $V_t$.

For the stationary causal AR(1) case, $\theta =0$ and $|\varphi |<1$. The first observation is $N(0,{\sigma }^2/(1-{\varphi }^2))$, whereas each later conditional observation is $\varphi x_{t-1}+{\epsilon }_t$. HenceThis also gives a direct counterexample to the uncorrected PDF variance: with $\theta =0$, $FG=\varphi F$ and $F{P}_{t-1}{F}^{\top }=0$ after observing $x_{t-1}$, so its printed expression would give $V_t=0$ for $t\ge 2$.

To compare the conditional and stationary AR1 likelihood estimators, putThen its negative twice log-likelihood isFor fixed $\varphi$, $\widehat{{\sigma }^2}=Q_T(\varphi )/T$. Writingthe exact interior score equation isThe stationary initial contribution is order one for a fixed interior parameter, while $C_T$ is order $T$. Discarding this lower-order contribution givesConditioning on $x_1$ makes this ratio the exact unconstrained conditional maximum likelihood estimate. Under the stationary initial law it is only the leading approximation: at an interior optimum,This approximation presumes a nondegenerate sample denominator and a true coefficient fixed away from the unit-root boundary; a finite-sample constrained optimum must respect $|\varphi |<1$.

The state-space model as printed supplies only uncorrelated white noise. That is insufficient for the claimed normal conditional distributions. For example, take $S_0$ normal, independent process noises $w_t$ taking values $\pm \sqrt{W}$ equally often, and independent Gaussian observation noises. With $G=1$ and no previous observations, $S_1=S_0+w_1$ is a two-component shifted mixture distribution, not normal for $W>0$. Its characteristic function is $e^{i\mu s-P_0{s}^2/2}\cos (\sqrt{W}s)$, which has zeros, whereas a normal characteristic function has none. Similarly a non-Gaussian observation noise prevents the asserted conditional normal law for $X_t$.

The intended scalar Gaussian Kalman recursion requires $w_t\sim N(0,W)$ and $v_t\sim N(0,V)$ independent of each other and of the previous state and observation history. Assume these Gaussian independence conditions, $V>0$, $W\ge 0$, and the given normal filtering law. Conditional on the past, $G{S}_{t-1}+w_t$ is a sum of independent normals, henceThe new observation noise is independent of $S_t$ and the past, so $X_t\mid S_t,{\mathcal{F}}_{t-1}\sim N(F{S}_t,V)$. Write $S_t=m_t+{\eta }_t$ and $X_t=F{m}_t+F{\eta }_t+v_t$. The independent normal pair $({\eta }_t,v_t)$ shows directly thatThe covariance $F{R}_t$ comes from the common ${\eta }_t$ term. Applying the conditional multivariate normal distribution formula with $X_t$ as the conditioned coordinate gives the Kalman filter updateThe final expression verifies nonnegativity of the posterior variance.

For $F=G=1$, defineThe recursions become ${\hat{S}}_t=(1-{\alpha }_t){\hat{S}}_{t-1}+{\alpha }_t{X}_t$ and $P_t={\alpha }_{t}V$. If $P_t\to P$, then ${\alpha }_t\to \alpha =P/V$ and, with $c=W/V$,The nonnegative solution is the steady-state local-level Kalman gain:For $c=0$ the limit is zero; for $c>0$ it lies strictly between zero and one. These are exact conditional-law results under the additional Gaussian assumptions, not consequences of the weaker printed noise description alone.

Use the standard linear Gaussian state-space model convention: the initial state is independent of both mutually independent iid noise sequences. This independence from the initial state is needed for the usual forecast recursion. Let ${\mathcal{Y}}_t=\sigma (y_1,\dots ,y_t)$ and write $m_{t-1}=x_{t-1}^{t-1}$ and $C_{t-1}=P_{t-1}^{t-1}$. Taking conditional expectations in the state equation givesTo justify the covariance, the forecast error is $\Phi (x_{t-1}-m_{t-1})+w_t$. Its two terms have zero cross-covariance because $w_t$ is independent of the previous state and observations. Their covariances therefore add. In this Gaussian model the conditional error covariance depends on the parameters and time but not on the realized data, so it also equals the unconditional forecast-error covariance used in the notation of the question.

For completeness, derive the filtering step rather than leave the algorithm unspecified. Set $m_t^{-}=x_t^{t-1}$ and $C_t^{-}=P_t^{t-1}$. Conditionally on the previous data, the observation has mean $A{m}_t^{-}$ and covarianceDefine the innovation process and gain byThe state forecast error and innovation are jointly normal, with cross-covariance $C_t^{-}{A}^{\top }$. The residual $x_t-m_t^{-}-K_t{ϵ}_t$ is uncorrelated with ${ϵ}_t$ by the definition of $K_t$, hence independent of it by joint normality. Its mean is zero and its covariance is $C_t^{-}-C_t^{-}{A}^{\top }{\Sigma }_t^{-1}A{C}_t^{-}$. This proves the conditional multivariate normal distribution updateInitialize $x_0^0={\mu }_0,P_0^0={\Sigma }_0$. For $t=1,\dots ,n$, predict using the first boxed pair, store that forecast, form ${ϵ}_t,{\Sigma }_t,K_t$, and update using the second boxed pair. The updated state then starts the next iteration. This is the Kalman filter, and supplies the entire forecast sequence in one forward pass.

The requested unconditional innovation moments areIn fact ${ϵ}_t\mid {\mathcal{Y}}_{t-1}\sim N_q(0,{\Sigma }_t)$, with ${\Sigma }_t$ deterministic for fixed parameters. Thus the innovation is independent of the past observations, and hence the innovations are mutually independent. Equivalently, orthogonality to the past together with joint normality gives this independence.

The chain rule for densities now yields the Gaussian innovation likelihood, treating the specified initial mean and covariance as known:Both ${ϵ}_t$ and ${\Sigma }_t$ are functions of the trial parameters, so the filtering recursion must be run again for each candidate $\Theta$. The ordinary density formula assumes ${\Sigma }_t$ positive definite; for example positive definite $R$ ensures this. Singular observation laws instead require densities on their Gaussian support.

Evaluate the log-likelihood numerically by the forward recursion, and maximize it over admissible $\Phi ,A,Q,R$, enforcing the covariance constraints. A Cholesky decomposition ${\Sigma }_t=L_t{L}_t^{\top }$ gives $\log |{\Sigma }_t|=2{\sum }_j\log (L_t{)}_{jj}$ and the quadratic term $\Vert L_t^{-1}{ϵ}_t{\Vert }^2$, avoiding an explicit matrix inverse. Covariance parameterizations or constrained optimization keep $Q,R$ positive semidefinite and the innovation covariance nonsingular. An attained maximizer is a maximum-likelihood estimator; in general this is a numerical optimization problem and neither uniqueness nor global convergence of a local optimizer is automatic.

Use the usual state-space assumption that the prior $X_0$ is independent of future process and observation noises. Suppose inductively $X_t\mid {\mathcal{Y}}_t\sim N({\hat{X}}_t,V_t)$. Prediction through the linear dynamics gives $X_{t+1}\mid {\mathcal{Y}}_t\sim N(A{\hat{X}}_t,M_t)$, where $M_t=A{V}_t{A}^T+{\Sigma }_{\epsilon }$. The pair $(X_{t+1},Y_{t+1})$ is conditionally multivariate normal, with means $A{\hat{X}}_t,CA{\hat{X}}_t$, cross covariance $M_t{C}^T$, and observation covariance $R_t={\Sigma }_{\eta }+C{M}_t{C}^T$. Applying part (a) gives the Kalman filterThis proves the inductive Gaussian assertion. The inverse requires nonsingular innovation covariance, as in the printed formula. Importantly, prediction uses $A{\hat{X}}_t$: the TeX conversion drops this $A$ in one place, whereas the original PDF includes it.

Write ${\sigma }^2={\Sigma }_{\eta }>0$ and assume $V_0>0$. The scalar Kalman filter reduces toTaking reciprocal variance and dividing the mean by variance givesThusSince the process noise is zero and $A=1$, $X_t=X_0$ and $Y_j=X_0+{\eta }_j$. The strong law of large numbers gives $t^{-1}{\sum }_j{Y}_j\to X_0$ almost surely. The prior's finite weight vanishes in the displayed formula, so both limits equal $X_0$ almost surely. A zero prior variance instead means $X_0$ is known, and the assertion follows directly without reciprocal-variance formulas.

At the terminal time,so $P_T=1$ and $d_T=V_T=1/(T+1)$.

Assume $F(W_t)=P_t{\hat{x}}_t^2+d_t$. Conditional on $W_{t-1}$, the random part of the next Kalman filter estimate is $h_t$ times the innovation. Its variance isTherefore the Bellman equation givesCompleting the square givesStarting from $P_T=1$ yieldsand hence

The minimizer in the quadratic expression from part (b) isRelabelling the time index,

If $\mathrm{Var}(x_0)$ or the observation-noise variances change, the Kalman filter gains and the sequence $V_t$ change. The control gain $K_t$ does not change, because the state dynamics and quadratic cost are unchanged. This is the separation principle: estimation uncertainty changes the additive constants $d_t$, while the optimal linear feedback gain is the full-information gain.

For the state-space model $S_t=G{S}_{t-1}+w_t$, $X_t=F{S}_t+v_t$, assume independent Gaussian process noise and measurement noise of variances $W,V$, independent of the prior state and previous observations. A normal distribution for the filtered state, with mean $m$ and variance $P$, predicts mean $Gm$ and variance $R=G^{2}P+W$. Here $R$ denotes predicted state variance, rather than the observation-noise covariance notation used in the vector equations above. Gaussian conditional expectation then gives posterior mean $Gm+K(X-FGm)$ and posterior variance $RV/(F^{2}R+V)$, where the Kalman gain is $K=FR/(F^{2}R+V)$.

Mere weak white noise assumptions do not imply normal posterior distributions: independent process noise with a Rademacher distribution makes the next state a mixture of two shifted normal distributions. The Kalman filter can still describe a best affine state estimate under appropriate second-order assumptions, but that is distinct from an exact Gaussian conditional law.

State estimation infers an unobserved state of a dynamical system from its observations and a model of its evolution. An observed position can help estimate an unobserved velocity; repeated noisy observations can help distinguish changes in the state from measurement noise. The result should describe both an estimate and its uncertainty. A Kalman filter does this exactly for a linear Gaussian state-space model, maintaining a conditional expectation and conditional covariance.

The time at which information is available matters. Causal state filtering estimates the present state from observations available now. State prediction estimates a future state from those same observations. Later observations can improve an estimate of a past state, but that is a different information set from the one a live controller had. Kalman explicitly distinguishes these estimation tasks in A New Approach to Linear Filtering and Prediction Problems (1960), section Optimal Estimates.