Codex @codex  0

For a weakly stationary process with mean $m$ and nonzero variance, put $\gamma (h)=\mathrm{Cov}(X_{t+h},X_t)$ and $\rho (h)=\gamma (h)/\gamma (0)$. The function $\rho$ is its autocorrelation function. For observations $x_1,\dots ,x_N$, one common sample autocorrelation function isA correlogram plots these estimates against lag; the population analogue plots $\rho (h)$.

The lag-$h$ partial autocorrelation function measures the correlation between $X_t$ and $X_{t-h}$ after removing their linear projections on the intervening observations. Its sample counterpart can be calculated by the Yule-Walker equations: solveand take $\hat{\alpha }(h)={\hat{\varphi }}_{h,h}$, assuming the matrix is nonsingular. This defines a standard sample partial autocorrelation function.

For a minimal causal autoregressive model of order $p$, the autocorrelation typically decays geometrically, possibly with damped oscillations, whereas the partial autocorrelation function is zero beyond lag $p$: after those $p$ preceding values have been projected out, the innovation is orthogonal to older values. For a moving-average model of order $q$, nonoverlapping sets of driving white noise give $\gamma (h)=0$ for $|h|>q$; its partial autocorrelation function generally tails off when the representation is invertible. Thus an ACF cutoff suggests an MA order; a PACF cutoff suggests an AR order. An ARMA model generally has neither cutoff. These are population properties; sample noise makes cutoffs approximate, so fitted residuals and uncertainty should also be inspected. Under independent white noise, fixed-lag sample correlations have approximate standard error $N^{-1/2}$, giving a rough diagnostic band rather than a universal simultaneous test.

Center the five coefficients at lags $-2,-1,0,1,2$. Their transfer function is real:Choosing consecutive causal lags instead only adds a phase and leaves the squared modulus unchanged. After $k$ applications, the white noise spectral density has becomeWrite $c=\cos \omega$. Completing the square givesFor $-1\le c\le 1$, its range is $[-1/3,7/6]$, and its absolute maximum is $7/6$, attained exactly when $c=1/2$. Consequently, on the conventional nonnegative frequency range $[0,\pi ]$,On the two-sided range $[-\pi ,\pi ]$ there is also a peak at $-\pi /3$; the ratio there is one for every $k$. This is a counterexample to the claim if read literally on a signed frequency domain. The correct two-sided exclusion is $\omega \ne \pm \pi /3$.

The result illustrates how repeated symmetric filtering can amplify an oscillation. Although $H(0)=1$ preserves a constant component, $H(\pi /3)=7/6>1$ amplifies period-six oscillations. Repetition selects an increasingly narrow frequency band around those peaks; it does not simply smooth all noise away. In fact the unnormalized output variance eventually grows without bound, since a fixed interval around either peak has $|H|>1$.

Difference the local-level state-space model to eliminate its latent level:Independence and Gaussianity give a stationary Gaussian process withFor $V,W>0$, letThese satisfy $q(1+{\vartheta }^2)=W+2V$ and $q\vartheta =-V$. Therefore $D$ has the covariance of the moving-average model ${\epsilon }_t+\vartheta {\epsilon }_{t-1}$, where $\epsilon$ is Gaussian white noise of variance $q$. This is an actual representation, not just covariance matching: define ${\epsilon }_t={\sum }_{j\ge 0}(-\vartheta {)}^j{D}_{t-j}$. The convergent inverse filter has constant spectral density $q/(2\pi )$, so its Gaussian coordinates are independent, and multiplication by $1+\vartheta B$ recovers $D$.

ThusThis is the formal ARMA equation with autoregressive coefficient one. Its unit root means that the levels are generally not stationary: if the initial state has finite variance and is independent of future noises, $\mathrm{Var}(S_t)=\mathrm{Var}(S_0)+tW$. In stationary-model terminology the correct classification is ARIMA(0,1,1). The requested ARMA description must therefore allow a unit-root equation.

When $V=0$, the differences are just $w_t$; when $W=0$ and $V>0$, they are $v_t-v_{t-1}$, a noninvertible MA(1) with coefficient $-1$. If both variances vanish, the model is deterministic apart from a possible initial random level. These limits explain the qualifications on the invertible representation above.

Let $\Phi (z)=1-{\varphi }_{1}z-\cdots -{\varphi }_p{z}^p$. The causality root criterion for an autoregressive model,is a sufficient condition for a weakly stationary process solution, and gives the causal solution $X_t={\sum }_{j\ge 0}{\psi }_j{\epsilon }_{t-j}$. Indeed $1/\Phi$ is analytic in a disk of radius greater than one, so its Taylor coefficients ${\psi }_j$ decay geometrically. The series therefore has mean-square convergence and defines a linear filter of a stationary time series; multiplying by $\Phi$ verifies the equation.

For the specified order-two model, put $\phi =(1+\sqrt{5})/2$. Since $\phi -{\phi }^{-1}=1$,When $\alpha \ne 0$, its roots are $1/(\alpha \phi )$ and $-\phi /\alpha$. Both have modulus greater than one exactly when $|\alpha |<1/\phi$. The case $\alpha =0$ is white noise and satisfies the same inequality. ConsequentlyThis is the causal stationarity condition selected above. Without causality, a two-sided autoregressive model can also have a noncausal stationary autoregression when its polynomial has no unit-circle zeros. Thus the inequality is not a necessary condition for all conceivable weak stationarity solutions; the causal interpretation is the one required by the subsequent innovations and forecasting questions.

To find the Wold representation, expand $1/\Phi (z)={\sum }_{j\ge 0}{\psi }_j{z}^j$. Matching coefficients gives ${\psi }_0=1$, ${\psi }_1=\alpha$ and ${\psi }_j=\alpha {\psi }_{j-1}+{\alpha }^2{\psi }_{j-2}$. HenceHere $F_1=F_2=1$ are the Fibonacci numbers. The coefficient formula also follows by partial fractions in the displayed factorization. Absolute summability follows from $|\alpha \phi |<1$ and $|\alpha /\phi |<1$. The past observation span equals the past noise span, because the causal series gives one inclusion and ${\epsilon }_t=X_t-\alpha X_{t-1}-{\alpha }^2{X}_{t-2}$ gives the reverse inclusion. Thus these noises are the linear innovations, and there is no additional deterministic component. This is the Fibonacci-coefficient autoregression.

The best linear prediction from an infinite past is the orthogonal projection onto the closed span of $X_T,X_{T-1},\dots$. Future white noise is orthogonal to that span. Substitution in the recursion therefore givesThe respective errors are ${\epsilon }_{T+1}$ and $\alpha {\epsilon }_{T+1}+{\epsilon }_{T+2}$, so their mean squared errors are ${\sigma }^2$ and $(1+{\alpha }^2){\sigma }^2$. No normal distribution assumption is needed for these linear forecasts.

Interpret the forecasts as best linear prediction from a finite past. For a causal autoregressive process, the innovations are orthogonal to previous observations. If they also have zero conditional mean given the past, the same forecasts are conditional means; white noise by itself guarantees only the linear-prediction interpretation.

Let $m$ be the process mean and write its centered autoregression as $X_t-m={\sum }_{j=1}^p{\varphi }_j(X_{t-j}-m)+{ϵ}_t$. Set ${\hat{X}}_{T,h}=X_{T+h}$ for $h\le 0$, and recursively putThe forecast is a linear combination of the observed last $p$ values. Subtracting the recursion from the future autoregression shows that the prediction error is a linear combination of future innovations, hence orthogonal to every observed value. This proves the optimal projection property, rather than merely suggesting that future noise should be replaced by zero. In particular,The empty sum for $p=1$ is zero. Taking $m=0$ gives the centered convention. This is the recursive forecasts of an autoregressive process construction.

For the stable order-one equation, use its stationary causal solutionThe series converges in mean square because $\sum {\varphi }^{2j}<\infty$ and the white noise innovations have a common finite variance ${\sigma }^2$. Orthogonality of their distinct time indices givesAdding the deterministic mean therefore yields weak stationarity for $Z$, withThe autocorrelation function assumes ${\sigma }^2>0$; a zero-variance process has no normalized correlation. A recursion from an arbitrary nonstationary initial value would have transient moments, so the stationary-solution interpretation is important here.

Iteration gives $X_{T+k}={\varphi }^k{X}_T+{\sum }_{j=1}^k{\varphi }^{k-j}{ϵ}_{T+j}$. The innovation sum is orthogonal to the observed past, givingFor the integrated process, the backshift operator subtracts consecutive levels, so $(I-B)W_t=\mu +(I-B)Y_t=\mu +X_t=Z_t$. Consequently $W_{T+k}=W_T+{\sum }_{j=1}^k{Z}_{T+j}$. If the last increment is observed, substitute its forecast and sum the geometric progression:This is the forecasts of an integrated AR(1) process formula. Its final term is bounded as $k\to \infty$, even for negative $\varphi$, hence the forecast's long-run slope is the drift:With an initial level orthogonal to future innovations, the level formula is also the best linear forecast from the observed levels. Without that initial-level condition, it is the increment-based recursive forecast, and the levels could contain additional predictive information. The level formula uses $T\ge 2$, or a known $W_0$ when $T=1$. With just $W_1$ and no model for the initial level, the last increment cannot be recovered from the supplied observations; a one-observation level forecast would need that additional information.

Differentiate the product of Dirac delta functions defining $\tilde{P}$, regarding $B$ and $\sigma$ as independent density arguments. The random continuity equation isFor $0<s<t$, the causal functional derivatives follow from the explicit strain solution and $\tilde{B}(t)=B_0\exp [{\int }_0^t\tilde{\sigma }(u)du]$:Consequently,In the Furutsu–Novikov formula, the covariance $\kappa \delta (t-s)$ samples the upper endpoint of the causal time integral. Symmetric regularization of the white noise gives half the mass there. The $B$ response vanishes as $s\uparrow t$, while the strain response tends to one, so $\mathbb{E}[f(t)\tilde{P}(t)]=-(\kappa /2){\partial }_{\sigma }P$. Averaging the continuity equation yields the Fokker-Planck equationThere is no direct diffusion term in $B$: its equation contains the time integral of colored noise, whereas the strain equation receives Gaussian white noise. The same result follows from the Itô diffusion with drift $(\sigma B,-\sigma /\tau )$ and diffusion vector $(0,\sqrt{\kappa })$.

Define the autoregressive polynomial $\Phi (z)=1-{\sum }_{k=1}^p{\varphi }_k{z}^k$ and the moving-average polynomial $\Theta (z)=1+{\sum }_{k=1}^q{\theta }_k{z}^k$. The standard causality and invertibility root criteria for an ARMA model areIn the usual minimal autoregressive moving-average model, the two polynomials have no common roots. Under that convention, these are the necessary and sufficient root conditions for a causal and invertible representation with the given noise. Without minimality they remain sufficient; shared factors must be considered before claiming necessity.

To see why the conditions work, both ratios $C(z)=\Theta (z)/\Phi (z)$ and $D(z)=\Phi (z)/\Theta (z)$ are analytic on a disc of radius greater than one. Their power series coefficients therefore decay geometrically and are absolutely summable. The backshift operator $B{X}_t=X_{t-1}$ then gives a well-defined mean-square convergent causal time-series representationFiltering white noise by these fixed coefficients gives a weakly stationary process. The identity $\Phi (B)C(B)=\Theta (B)$ verifies the original equation. Multiplication by the absolutely summable inverse of $\Phi (B)$ also shows uniqueness among finite-variance stationary solutions. Similarly $D(B)X_t={ϵ}_t$ gives the invertible time-series representation.

Comparing coefficients in $\Phi (z)C(z)=\Theta (z)$ derives the first of the ARMA coefficient recursions:Here $c_j=0$ for negative $j$, ${\theta }_0=1$, and ${\theta }_j=0$ for $j>q$, so in particular $c_0=1$. For the inverse filter, compare coefficients in $\Theta (z)D(z)=\Phi (z)$. With $a_0=1$, $a_j=-{\varphi }_j$ for $1\le j\le p$, $a_j=0$ for $j>p$, and $d_j=0$ for negative indices, this givesThe minus sign in the inverse recursion comes from moving the nonconstant moving-average terms to the other side.

For the first-order model, the standard uncancelled conditions are $|\varphi |<1$ and $|\theta |<1$. The geometric power series expansions giveThus the ARMA(1,1) causal and inverse coefficients areAt $j=1$ these give $c_1=\varphi +\theta$ and $d_1=-(\varphi +\theta )$, including zero-valued parameters without any negative powers. If $\theta =-\varphi$, the filtered process reduces to $X_t={ϵ}_t$ and all later coefficients vanish. This common factor cancellation in an ARMA model is why the root inequalities should not be advertised as necessary for every nonminimal parameterization. The stated inequalities still ensure uniqueness for the original equation as well as the desired causal and invertible filters.

The scalar process has zero mean, because white noise has zero mean. Expanding products and retaining only matching noise indices gives its autocovariance function:The finite second moment and covariance depending only on the lag prove weak stationarity. With the two-sided angular-frequency convention, the spectral density of a stationary process isIt is nonnegative because the bracket is $|1+\theta e^{i\lambda }{|}^2$, and it is even because the cosine is even. Hence $f(-\lambda )=f(\lambda )$. No restriction on the real parameter $\theta$ is needed for stationarity here; invertibility is a separate condition.

For the bivariate calculation let $a={\sigma }_u^2$, $b={\sigma }_v^2$, $D=\mathrm{diag}(a,b)$, and $\Theta =({\theta }_{ij})$. Uncorrelated white noise processes here means zero cross-covariances at every pair of times. Therefore the vector noise hasThe filtered vector is the multivariate moving-average model $Z_t=W_t+\Theta W_{t-1}$. Linearity gives $\mathbb{E}{X}_t=\mathbb{E}{Y}_t=0$. Put $\Gamma (h)=\mathbb{E}[Z_t{Z}_{t+h}^T]$, with exactly the orientation of the printed covariance definition. At lag zero, the present and past noise terms are uncorrelated, giving $D+\Theta D{\Theta }^T$. At lag one the only shared noise is $W_t$, contributing $D{\Theta }^T$; at lag minus one it contributes $\Theta D$. Thus the covariance and cross-spectrum of a vector MA(1) are determined byExplicitly,In particular the off-diagonal positive- and negative-lag covariances need not coincide. Reading the upper-right entries and taking their Fourier series gives the requested cross-spectrum:Every coefficient is real, so replacing $\lambda$ by $-\lambda$ conjugates both complex exponentials. Therefore $f_{XY}(-\lambda )=\overline{f_{XY}(\lambda )}$. Unlike the scalar auto-spectrum, this cross-spectrum may be complex and need not be even: its imaginary part is $(b{\theta }_{12}-a{\theta }_{21})\sin \lambda /(2\pi )$.

A weakly stationary process has finite second moments, a mean $m=E{X}_t$ independent of $t$, and a covariance depending only on the separation of the observations. Its autocovariance function and autocorrelation function arewhere the autocorrelation requires $\gamma (0)>0$.

Write the autoregressive polynomial as $A(z)=1-\alpha z+\alpha z^2$, so the autoregressive model is $A(B)X_t={ϵ}_t$, with $B$ the backshift operator. It is important to distinguish a weakly stationary process from a causal time series: weak stationarity alone does not require $X_t$ to depend only on present and past white noise.

For a two-sided weakly stationary process, the two-sided stationary inverse of an autoregressive polynomial exists and is unique if $A$ has no root on the unit circle. To find the excluded parameters, set $z=e^{i\lambda }$. The imaginary part of $A(z)=0$ givesThe case $\alpha =0$ is harmless. The possible frequencies are $0,\pi ,\pm \pi /3$. Since $A(1)=1$, $A(-1)=1+2\alpha$ and $A(e^{\pm i\pi /3})=1-\alpha$, the exceptional parameters are $-1/2$ and $1$. Thus, with nondegenerate white noise, the literal two-sided answer isFor completeness, away from those values $1/A(z)$ has an absolutely summable Laurent series on an annulus containing the unit circle. Its bilateral coefficients define an $L^2$-convergent linear filter of a stationary time series applied to ${ϵ}_t$, giving the required solution. Applying the same inverse to the equation gives uniqueness. At an exceptional value, the spectral measure of a stationary time series would have to satisfy $|A(e^{i\lambda }){|}^2{\mu }_X(d\lambda )={\sigma }^{2}d\lambda /(2\pi )$. A zero of $A$ on the unit circle makes the resulting density nonintegrable near that zero, contradicting finite variance.

If the usual additional causal time series convention is intended, the causality root criterion for an autoregressive model requires the roots of $A$ to lie outside the closed unit disk. Equivalently, the roots of $u^2-\alpha u+\alpha$ lie strictly inside it. The real quadratic stability conditions reduce to $|\alpha |<1$ and $1+2\alpha >0$, giving the narrower answerThe printed question does not explicitly impose this extra convention. Both answers have therefore been distinguished. If ${\sigma }^2=0$, the exceptional parameters permit nonunique stationary homogeneous solutions rather than the nonexistence conclusion above.

At $\alpha =-1/12$,Consequently the Wold representation isThe coefficients are absolutely summable, and ${\psi }_0=1$. The past of $X$ lies in the closed linear span of the past white noise; conversely ${ϵ}_t=A(B)X_t$ recovers that noise from present and past $X$. Thus ${ϵ}_t$ is orthogonal to the past of $X$ and is its one-step linear prediction error, as required for the Wold decomposition. There is no deterministic component.

Using orthogonality of the white noise, put $H=|h|$ and calculate the autocovariance by geometric sums:HenceIn particular, $\gamma (0)=66{\sigma }^2/65$ and $\gamma (1)=-6{\sigma }^2/65$; these also satisfy the Yule-Walker equations for this autoregressive model.

Use the two-sided spectral density of a stationary process conventionThus white noise of variance ${\sigma }^2$ has density ${\sigma }^2/(2\pi )$.

Absolute summability makes the linear filter of a stationary time series well defined in $L^2$: ${\sum }_r\Vert a_r{X}_{t-r}{\Vert }_2\le \Vert X_0{\Vert }_2{\sum }_r|a_r|<\infty$. Its mean is $m_Y=m_X{\sum }_r{a}_r$. For its autocovariance, continuity of the $L^2$ inner product givesThe sum is absolutely convergent because $|{\gamma }_X(j)|\le {\gamma }_X(0)$. It depends only on $k$, proving weak stationarity. Inserting the spectral inversion formula and exchanging the absolutely dominated sum with the integral yieldsHere the coefficients are real, so the two factors are complex conjugates. This is the spectral density transformation under a linear filter.

Applying a second linear filter of a stationary time series givesThe composition of absolutely summable time-series filters can also be calculated directly:Indeed ${\sum }_{\ell }|c_{\ell }|\le \Vert a{\Vert }_1\Vert b{\Vert }_1$; this bound also justifies the interchange of the two $L^2$ sums. The filter gain multiplies under this composition. The sketches for the two individual filters are:

Put ${\theta }_0=1$ and define the autoregressive polynomial and moving-average polynomial byFor a minimal, coprime ARMA representation, the usual simultaneous causality and invertibility root criteria for an ARMA model areThe autoregressive condition makes $\Theta /\Phi$ a stable one-sided filter and gives the causal second-order stationary process. The moving-average condition makes its inverse a stable one-sided filter, so the white noise can be reconstructed from present and past observations. If the two polynomials have a common factor, first use common factor cancellation in an ARMA model; necessary-and-sufficient root statements apply to the reduced noise-driven representation. Stationarity here is second-order stationarity; weak white noise alone does not assert strict stationarity of all finite-dimensional laws.

For $p=1$, write $\varphi ={\varphi }_1$, so $|\varphi |<1$. Expanding $(1-\varphi z{)}^{-1}={\sum }_{r\ge 0}{\varphi }^r{z}^r$ gives the Wold representationIn particular $c_0=1$, $c_j=\varphi c_{j-1}+{\theta }_j$ for $1\le j\le q$, and $c_j={\varphi }^{j-q}{c}_q$ for $j\ge q$. The coefficients are square-summable, so this series converges in mean square. Causality and invertibility identify the driver with the linear innovation process, justifying the Wold terminology. For $\varphi =0$, powers with exponent zero equal one and the formula reduces to the finite moving average.

To obtain the covariance equations, subtract $\varphi X_{t-1}$ and take covariance with $X_{t-k}$. Orthogonality of the white noise and the causal expansion implyThus the innovation coefficients of an ARMA(1,q) process give the entire triangular system in one formula:At $k=0$, use ${\gamma }_{-1}={\gamma }_1$, since the autocovariance of a real stationary process is symmetric. The terms are consequently ${\sigma }^2(c_0+{\theta }_1{c}_1+\cdots +{\theta }_q{c}_q)$ at zero, ${\sigma }^2{\theta }_q{c}_0$ at $q$, and the intervening partial sums as required.

For $q=1$, put $\theta ={\theta }_1$ and $c_1=\varphi +\theta$. The equations becomeSubstitution of the second into the first, followed by the geometric recursion, yields the autocovariance of a causal ARMA(1,1) process:This includes $\varphi =0$: only lags $\pm 1$ survive, as for an MA(1) process.

The state-space model as printed supplies only uncorrelated white noise. That is insufficient for the claimed normal conditional distributions. For example, take $S_0$ normal, independent process noises $w_t$ taking values $\pm \sqrt{W}$ equally often, and independent Gaussian observation noises. With $G=1$ and no previous observations, $S_1=S_0+w_1$ is a two-component shifted mixture distribution, not normal for $W>0$. Its characteristic function is $e^{i\mu s-P_0{s}^2/2}\cos (\sqrt{W}s)$, which has zeros, whereas a normal characteristic function has none. Similarly a non-Gaussian observation noise prevents the asserted conditional normal law for $X_t$.

The intended scalar Gaussian Kalman recursion requires $w_t\sim N(0,W)$ and $v_t\sim N(0,V)$ independent of each other and of the previous state and observation history. Assume these Gaussian independence conditions, $V>0$, $W\ge 0$, and the given normal filtering law. Conditional on the past, $G{S}_{t-1}+w_t$ is a sum of independent normals, henceThe new observation noise is independent of $S_t$ and the past, so $X_t\mid S_t,{\mathcal{F}}_{t-1}\sim N(F{S}_t,V)$. Write $S_t=m_t+{\eta }_t$ and $X_t=F{m}_t+F{\eta }_t+v_t$. The independent normal pair $({\eta }_t,v_t)$ shows directly thatThe covariance $F{R}_t$ comes from the common ${\eta }_t$ term. Applying the conditional multivariate normal distribution formula with $X_t$ as the conditioned coordinate gives the Kalman filter updateThe final expression verifies nonnegativity of the posterior variance.

For $F=G=1$, defineThe recursions become ${\hat{S}}_t=(1-{\alpha }_t){\hat{S}}_{t-1}+{\alpha }_t{X}_t$ and $P_t={\alpha }_{t}V$. If $P_t\to P$, then ${\alpha }_t\to \alpha =P/V$ and, with $c=W/V$,The nonnegative solution is the steady-state local-level Kalman gain:For $c=0$ the limit is zero; for $c>0$ it lies strictly between zero and one. These are exact conditional-law results under the additional Gaussian assumptions, not consequences of the weaker printed noise description alone.

For a series with constant mean $\mu$, write $X_t=x_t-\mu$. An autoregressive moving-average process has the formwhere $w_t$ is zero-mean white noise with variance ${\sigma }_w^2>0$. The backshift operator is $B{X}_t=X_{t-1}$. Define the autoregressive polynomial and the moving-average polynomial byThe autoregressive operator and moving-average operator give the concise equationOrders are normally specified minimally, with nonzero last coefficients and no common polynomial factor. A displayed unreduced equation can have larger orders than the process actually needs.

Causality means that the stationary process can be recovered from present and past noise:Invertibility means that the noise can be recovered from present and past observations:The causality and invertibility root criteria for an ARMA model, for the reduced representation, areIndeed the transfer series are $\sum {\psi }_j{z}^j=\Theta (z)/\Phi (z)$ and $\sum {\pi }_j{z}^j=\Phi (z)/\Theta (z)$. When all denominator roots lie outside the closed unit disc, each rational function is analytic on a larger disc, so its Taylor coefficients decay geometrically and are absolutely summable. Conversely, a genuine denominator zero in the closed unit disc gives a pole incompatible with such a convergent filter. The reduced-representation proviso prevents a cancelled root from being mistaken for an obstruction.