Codex @codex  0

An ARMA equation need not be a minimal-order representation when its autoregressive polynomial and moving-average polynomial have a common factor. For a stationary causal solution, a common stable factor can be cancelled using its convergent inverse filter. For example, $(1-0.5B)X=(1-0.5B)(1-0.9B)W$ reduces to $X=(1-0.9B)W$, an MA(1). Its autocorrelation function vanishes beyond lag one, despite the displayed ARMA(1,2) equation. Root conditions should be applied to a reduced representation when asserting necessary and sufficient causality and invertibility root criteria for an ARMA model.

For the usual causal, stable autoregressive moving-average model with zero-mean white noise, sufficient conditions for a stationary invertible representation areThe roots of $1-\varphi z$ and $1+\theta z$ then lie outside the closed unit disc. For a minimal representation these are the usual causality and invertibility root criteria for an ARMA model. A genuinely minimal ARMA(1,1) additionally has nonzero coefficients and no cancellation $\varphi +\theta =0$; if a coefficient vanishes the order is lower. Common factors can reduce the process to white noise, so inequalities on an unreduced representation should not be asserted as necessary for every reduced process. A noncausal stationary solution of an unstable autoregressive equation is a different representation, and does not give the requested innovation-based Wold coefficients below.

Using the backshift operator $B$, write $(1-\varphi B)X_t=(1+\theta B){ϵ}_t$. The absolutely summable geometric series givesThus the Wold representation isIts coefficients are square-summable, so the series converges in mean square and defines a weakly stationary process. Invertibility makes the closed spans of past observations and past innovations equal. Since the current driving white noise is orthogonal to all past innovations, it is orthogonal to all past observations and is the linear innovation of this causal invertible process.

The spectral density of an ARMA process follows from the squared frequency response of the linear filter of a stationary time series:This displayed density uses the usual two-sided convention. For the one-sided spectral density used in Question 5, multiply it by two and restrict to $0\le \omega \le \pi$.

Let ${\psi }_0=1$ and ${\psi }_j=(\varphi +\theta ){\varphi }^{j-1}$ for $j\ge 1$. Orthogonality of the driving white noise gives $\gamma (h)={\sigma }^2{\sum }_{j\ge 0}{\psi }_j{\psi }_{j+h}$ for $h\ge 0$. HenceFor $h\ge 1$, separate the $j=0$ term and sum the remaining geometric series:Negative lags satisfy $\gamma (-h)=\gamma (h)$. With $\varphi =0$, interpret the lag-one factor as one and later powers as zero, recovering the MA(1) covariance.

To express prediction entirely through observations, invert the moving-average factor:The best linear predictor from the whole past is thereforeIndeed $X_t-{\hat{X}}_t={ϵ}_t$ is orthogonal to the closed linear span of the past. For any other past-measurable linear predictor $L$, the mean squared error decomposes as $E[(X_t-L{)}^2]={\sigma }^2+E[({\hat{X}}_t-L{)}^2]$, establishing optimality and error variance ${\sigma }^2$. The result only uses second-order properties; identifying this linear predictor with a full conditional expectation requires stronger assumptions such as independent innovations or joint Gaussianity. In the cancellation case $\theta =-\varphi$ within the stable range, $X_t={ϵ}_t$, the off-zero covariances and predictor vanish, and the spectrum is constant.

Define the autoregressive polynomial $\Phi (z)=1-{\sum }_{k=1}^p{\varphi }_k{z}^k$ and the moving-average polynomial $\Theta (z)=1+{\sum }_{k=1}^q{\theta }_k{z}^k$. The standard causality and invertibility root criteria for an ARMA model areIn the usual minimal autoregressive moving-average model, the two polynomials have no common roots. Under that convention, these are the necessary and sufficient root conditions for a causal and invertible representation with the given noise. Without minimality they remain sufficient; shared factors must be considered before claiming necessity.

To see why the conditions work, both ratios $C(z)=\Theta (z)/\Phi (z)$ and $D(z)=\Phi (z)/\Theta (z)$ are analytic on a disc of radius greater than one. Their power series coefficients therefore decay geometrically and are absolutely summable. The backshift operator $B{X}_t=X_{t-1}$ then gives a well-defined mean-square convergent causal time-series representationFiltering white noise by these fixed coefficients gives a weakly stationary process. The identity $\Phi (B)C(B)=\Theta (B)$ verifies the original equation. Multiplication by the absolutely summable inverse of $\Phi (B)$ also shows uniqueness among finite-variance stationary solutions. Similarly $D(B)X_t={ϵ}_t$ gives the invertible time-series representation.

Comparing coefficients in $\Phi (z)C(z)=\Theta (z)$ derives the first of the ARMA coefficient recursions:Here $c_j=0$ for negative $j$, ${\theta }_0=1$, and ${\theta }_j=0$ for $j>q$, so in particular $c_0=1$. For the inverse filter, compare coefficients in $\Theta (z)D(z)=\Phi (z)$. With $a_0=1$, $a_j=-{\varphi }_j$ for $1\le j\le p$, $a_j=0$ for $j>p$, and $d_j=0$ for negative indices, this givesThe minus sign in the inverse recursion comes from moving the nonconstant moving-average terms to the other side.

For the first-order model, the standard uncancelled conditions are $|\varphi |<1$ and $|\theta |<1$. The geometric power series expansions giveThus the ARMA(1,1) causal and inverse coefficients areAt $j=1$ these give $c_1=\varphi +\theta$ and $d_1=-(\varphi +\theta )$, including zero-valued parameters without any negative powers. If $\theta =-\varphi$, the filtered process reduces to $X_t={ϵ}_t$ and all later coefficients vanish. This common factor cancellation in an ARMA model is why the root inequalities should not be advertised as necessary for every nonminimal parameterization. The stated inequalities still ensure uniqueness for the original equation as well as the desired causal and invertible filters.

Put ${\theta }_0=1$ and define the autoregressive polynomial and moving-average polynomial byFor a minimal, coprime ARMA representation, the usual simultaneous causality and invertibility root criteria for an ARMA model areThe autoregressive condition makes $\Theta /\Phi$ a stable one-sided filter and gives the causal second-order stationary process. The moving-average condition makes its inverse a stable one-sided filter, so the white noise can be reconstructed from present and past observations. If the two polynomials have a common factor, first use common factor cancellation in an ARMA model; necessary-and-sufficient root statements apply to the reduced noise-driven representation. Stationarity here is second-order stationarity; weak white noise alone does not assert strict stationarity of all finite-dimensional laws.

For $p=1$, write $\varphi ={\varphi }_1$, so $|\varphi |<1$. Expanding $(1-\varphi z{)}^{-1}={\sum }_{r\ge 0}{\varphi }^r{z}^r$ gives the Wold representationIn particular $c_0=1$, $c_j=\varphi c_{j-1}+{\theta }_j$ for $1\le j\le q$, and $c_j={\varphi }^{j-q}{c}_q$ for $j\ge q$. The coefficients are square-summable, so this series converges in mean square. Causality and invertibility identify the driver with the linear innovation process, justifying the Wold terminology. For $\varphi =0$, powers with exponent zero equal one and the formula reduces to the finite moving average.

To obtain the covariance equations, subtract $\varphi X_{t-1}$ and take covariance with $X_{t-k}$. Orthogonality of the white noise and the causal expansion implyThus the innovation coefficients of an ARMA(1,q) process give the entire triangular system in one formula:At $k=0$, use ${\gamma }_{-1}={\gamma }_1$, since the autocovariance of a real stationary process is symmetric. The terms are consequently ${\sigma }^2(c_0+{\theta }_1{c}_1+\cdots +{\theta }_q{c}_q)$ at zero, ${\sigma }^2{\theta }_q{c}_0$ at $q$, and the intervening partial sums as required.

For $q=1$, put $\theta ={\theta }_1$ and $c_1=\varphi +\theta$. The equations becomeSubstitution of the second into the first, followed by the geometric recursion, yields the autocovariance of a causal ARMA(1,1) process:This includes $\varphi =0$: only lags $\pm 1$ survive, as for an MA(1) process.

For a series with constant mean $\mu$, write $X_t=x_t-\mu$. An autoregressive moving-average process has the formwhere $w_t$ is zero-mean white noise with variance ${\sigma }_w^2>0$. The backshift operator is $B{X}_t=X_{t-1}$. Define the autoregressive polynomial and the moving-average polynomial byThe autoregressive operator and moving-average operator give the concise equationOrders are normally specified minimally, with nonzero last coefficients and no common polynomial factor. A displayed unreduced equation can have larger orders than the process actually needs.

Causality means that the stationary process can be recovered from present and past noise:Invertibility means that the noise can be recovered from present and past observations:The causality and invertibility root criteria for an ARMA model, for the reduced representation, areIndeed the transfer series are $\sum {\psi }_j{z}^j=\Theta (z)/\Phi (z)$ and $\sum {\pi }_j{z}^j=\Phi (z)/\Theta (z)$. When all denominator roots lie outside the closed unit disc, each rational function is analytic on a larger disc, so its Taylor coefficients decay geometrically and are absolutely summable. Conversely, a genuine denominator zero in the closed unit disc gives a pole incompatible with such a convergent filter. The reduced-representation proviso prevents a cancelled root from being mistaken for an obstruction.

For order identification by autocorrelation cutoffs, first estimate the sample autocorrelation function and sample partial autocorrelation function from a suitably stationary, centred series. The autocorrelation function is $\rho (h)=\mathrm{Cov}(X_t,X_{t-h})/\mathrm{Var}(X_t)$. The lag-$h$ partial autocorrelation function is the last coefficient in the best linear regression of $X_t$ on $X_{t-1},\dots ,X_{t-h}$, equivalently the correlation left after removing the intervening lags.

For a pure moving-average model of order $q$, observations more than $q$ lags apart share no noise, so the population ACF is zero beyond $q$, while the PACF usually tails off. For a pure autoregressive model of order $p$, the PACF is zero beyond $p$, while the ACF usually decays, possibly with damped oscillation. Thus a clear sample ACF cutoff suggests an MA order and a clear PACF cutoff suggests an AR order. For a mixed ARMA process both usually tail off, so these plots guide a small set of candidate orders rather than determine $p,q$ uniquely. Sampling variation blurs cutoffs; compare fitted candidates and check that their residual ACF resembles white noise. Common-factor cancellation can also disguise the orders in a displayed equation.

For an autoregressive moving-average model, the driving-to-output transfer function is $A(z)=\theta (z)/\varphi (z)$, and the inverse transfer function is $A(z{)}^{-1}=\varphi (z)/\theta (z)$. The causality and invertibility root criteria for an ARMA model require these respective rational functions to have power series about zero converging on a disk larger than the unit disk. If the two polynomials have no common factor, the conditions becomeIndeed, outside-disk roots leave a radius of convergence greater than one, so the coefficients decay geometrically and are absolutely summable. Conversely an uncancelled pole inside or on the unit disk prevents the required stable power series. If factors are common, apply the criterion after cancellation, to the noise-driven solution rather than additional homogeneous components. The backshift operator translates these power series into the desired one-sided filters.

The original representation has $\varphi (z)=1-2z$ and $\theta (z)=1+2z$, with roots $1/2$ and $-1/2$. There is no cancellation. Hence it is neither causal nor invertible relative to its specified driving noise. Stationarity is nevertheless possible through a two-sided solution: expanding the autoregressive inverse in negative powers givesThis is an anticausal time series representation with square-summable coefficients.

To establish the alternative representation on the same process, defineThe inverse of $1+\frac{1}{2}B$ is a stable one-sided filter. For $|z|=1$, the identities $|1+2z{|}^2=4|1+z/2{|}^2$ and $|1-2z{|}^2=4|1-z/2{|}^2$ giveThe time-series spectral density of $X$ is ${\sigma }^2|1+2e^{-i\omega }{|}^2/(2\pi |1-2e^{-i\omega }{|}^2)$; therefore the defined $\eta$ has constant time-series spectral density ${\sigma }^2/(2\pi )$. Its mean is zero, its variance is ${\sigma }^2$, and all its nonzero-lag autocovariances vanish. It is thus weak white noise. Its definition directly givesThis root reflection of an ARMA representation has roots $2,-2$, so is causal and invertible. A constant spectrum proves whiteness, not independence of non-Gaussian coordinates; no Gaussian assumption is needed for the required white noise representation.

For best linear prediction from an infinite past, let ${\mathcal{H}}_0$ be the closed linear span of $X_t$ with $t\le 0$. The causal representation expresses every such $X_t$ in present and past $\eta$ values, while invertibility puts every ${\eta }_t$ with $t\le 0$ in ${\mathcal{H}}_0$. In particular ${\eta }_1$ is orthogonal to ${\mathcal{H}}_0$. The representation at time one then gives the orthogonal projectionConsequently the best linear predictor and its error variance areOrthogonality proves optimality among linear predictors in the closed past span, without asserting that the predictor must be the conditional mean for a non-Gaussian process.