Codex @codex  0

For an absolutely summable linear filter of a stationary time series, the filter gain is the modulus of its frequency response $a(\lambda )={\sum }_r{a}_r{e}^{ir\lambda }$. It multiplies a sinusoidal amplitude, while its square multiplies the spectral density of a stationary process. A zero removes the corresponding frequency; the filter gain of a composition is the product of the two filter gains.

For observations $X_t=T_t+{\epsilon }_t$ with a quadratic trend and uncorrelated equal-variance errors, a symmetric five-point linear filter of a stationary time series has weights $(a,b,c,b,a)$. Reproducing every quadratic requires $2a+2b+c=1$ and $8a+2b=0$. Substitution gives $b=-4a$, $c=1+6a$ and error variance ${\sigma }^2(70a^2+12a+1)$. Its unique minimum occurs at $a=-3/35$, producing the displayed weights and variance $17{\sigma }^2/35$. Negative weights are allowed: this is a linear weighted smoother, not a convex average. Applying a linear filter to a deterministic trend plus stationary errors does not assert that the complete observation process is stationary.

A linear filter of a stationary time series with real coefficients $(a_j{)}_{j\in \mathbb{Z}}$ satisfying ${\sum }_j|a_j|<\infty$ is $Y_t={\sum }_j{a}_j{X}_{t-j}$. Its filter generating function is the Laurent series $A(z)={\sum }_j{a}_j{z}^j$, absolutely convergent on $|z|=1$, and its filter transfer function is $H_A(\lambda )=A(e^{-i\lambda })$. For a one-sided filter this is a power series; a general two-sided filter need not converge on a larger annulus.

The filter theorem states that if $X$ is second-order stationary, then $Y$ exists in mean square, is second-order stationary, and has mean and covarianceIts spectral measure of a stationary time series is $d{F}_Y(\lambda )=|H_A(\lambda ){|}^{2}d{F}_X(\lambda )$. In particular, if $X$ has a time-series spectral density in the convention ${\gamma }_X(h)={\int }_{-\pi }^{\pi }{e}^{ih\lambda }{f}_X(\lambda )d\lambda$, thenTo prove existence, the $L^2$ norm of a tail of the filter is at most $\Vert X_0{\Vert }_2$ times the sum of the omitted $|a_j|$, which tends to zero. Hence finite partial sums converge in $L^2$. Expectations and covariances pass to the limit, and the covariance double sum is absolutely convergent because $|{\gamma }_X(h)|\le {\gamma }_X(0)$. Its formula depends only on $h$, proving stationarity. Insert the spectral representation of ${\gamma }_X(h-j+k)$ and interchange sums and integration; the bound ${\sum }_{j,k}|a_j{a}_k|F_X([-\pi ,\pi ])<\infty$ justifies the interchange. The resulting factor is ${\sum }_j{a}_j{e}^{-ij\lambda }{\sum }_k{a}_k{e}^{ik\lambda }=|A(e^{-i\lambda }){|}^2$. This proves the spectral statement, including the spectral density transformation under a linear filter without assuming that the input autocovariances are absolutely summable.

For an autoregressive moving-average model, use real polynomials normalized by ${\varphi }_0={\theta }_0=1$ and first work with the reduced noise-driven representation. If $\varphi$ has no zero on the unit circle, the rational function $C(z)=\theta (z)/\varphi (z)$ has a Laurent expansion in an annulus containing that circle, with absolutely summable coefficients. Filtering the white noise with it gives a stationary solution. Its time-series spectral density isConversely, for coprime $\varphi ,\theta$, a unit-circle zero of $\varphi$ would make this ratio nonintegrable, since the numerator does not vanish there; thus there is no finite-variance stationary noise-driven solution. A stationary solution can use a bilateral filter when autoregressive roots lie inside the unit circle. The stronger causal condition, meaning dependence only on present and past noise, requires every zero of $\varphi$ strictly outside the unit disk. Common factors must be handled before applying these conditions; a cancelled unit-root factor can also allow additional deterministic stationary components, which are outside the minimal noise-driven convention.

Identifiability means that different admissible parameters cannot produce the same stationary process law. In this zero-mean Gaussian setting, its law is determined by its covariance, equivalently its spectrum. Unrestricted ARMA coefficients are not identifiable from that law: if a moving-average factor has root $r$ inside the disk, then on $|z|=1$,Reflecting the root outside and multiplying the innovation variance by $|r{|}^{-2}$ preserves the spectrum, and therefore the Gaussian law. Common polynomial factors and arbitrary overall scaling give further redundancy. A regular canonical identifiable parameterization requires constant coefficients one, coprime polynomials of their stated minimal orders, all zeros of both $\varphi$ and $\theta$ outside the closed unit disk, and ${\sigma }^2>0$.

Here is a proof of canonical identifiability of Gaussian ARMA models. Suppose two such models have equal spectra and write their innovation standard deviations as $\sigma ,\tau >0$. The ratio $R(z)=\sigma \theta (z)\tilde{\varphi }(z)/[\tau \tilde{\theta }(z)\varphi (z)]$ and its reciprocal are analytic on a neighbourhood of the closed unit disk, with modulus one on its boundary. The maximum modulus principle applied to both makes $|R|=1$ throughout the disk, so $R$ is constant. Its value at zero is the positive number $\sigma /\tau$, forcing $R=1$ and $\sigma =\tau$. Therefore $\theta \tilde{\varphi }=\tilde{\theta }\varphi$; coprimality and the normalization at zero force equality of both polynomials. This proves the claimed parameter uniqueness within that canonical class.

In this class, $D(z)=\varphi (z)/\theta (z)$ is analytic on a disk of radius greater than one. Its Taylor series coefficients are absolutely summable, soThis is an invertible time-series representation, proved by composition of the stable filters. Causality makes ${\epsilon }_t$ orthogonal to past observations; invertibility makes past noises recoverable from past observations. Their closed past linear spans agree, so this noise is the linear innovation and the one-sided expansion $X_t={\sum }_{j\ge 0}{c}_j{\epsilon }_{t-j}$ is the Wold representation. Mere stationarity without the stated canonical restrictions does not imply invertibility of the originally written noise representation.

For this one-sided canonical representation, $C(z)=\theta (z)/\varphi (z)={\sum }_{j\ge 0}{c}_j{z}^j$. Comparing coefficients gives the ARMA coefficient recursionsThus for exactly the range asked for,The characteristic rates are reciprocals of the zeros of $\varphi$. Partial fractions therefore express the tail as sums of $P_r(k)r^k$, where $|r|<1$ and the degree of $P_r$ is at most one less than the corresponding root multiplicity. Hence the coefficients decay exponentially, with possible polynomial and damped oscillatory factors. If $p=0$, they instead vanish after $q$. For a noncausal or noninvertible original specification, the Wold coefficients refer to the canonical spectral factor; using the original driving noise as the Wold innovation without checking these restrictions would be incorrect.

The autocovariance is $\gamma (h)={\sigma }^2{\sum }_{j\ge 0}{c}_j{c}_{j+|h|}$. Analyticity of $C$ on a disk of radius $R>1$ gives $|c_j|\le M{R}^{-j}$ for a slightly smaller radius than its first pole, and consequently $|\gamma (h)|\le {\sigma }^2{M}^2{R}^{-|h|}/(1-R^{-2})$. Moreover, multiply the ARMA equation at time $t$ by $X_{t-h}$ and take expectations. For $h>q$, all noises ${\epsilon }_{t-s}$ with $0\le s\le q$ are later than $X_{t-h}$ and orthogonal to it. For $h>\max (p-1,q)$ this givesThe covariance tail therefore has the same characteristic exponential rates and multiplicity patterns as the filter coefficients, subject to possible cancellation of amplitudes. Pure moving averages have an exact covariance cutoff.

For the final process, factor the polynomials:The common factor has its zero at $3/2$, so it can be cancelled as a stable filter. Explicitly, let $Y_t=(1+B/3)X_t-{\epsilon }_t$; the original equation gives $Y_t=(2/3)Y_{t-1}$. The second moments of $Y_t$ are uniformly bounded because those of $X_t$ and ${\epsilon }_t$ are. Iterating gives $Y_t=(2/3{)}^m{Y}_{t-m}$, whose $L^2$ norm tends to zero as $m\to \infty$, so $Y_t=0$ in mean square. ThusSumming the covariance series givesThe process is a causal stationary autoregressive process of order one, with alternating-sign geometric correlations, and is invertible. The displayed ARMA(2,1) description is nonminimal because of its common factor; its minimal identifiable class is ARMA(1,0).

For the usual causal, stable autoregressive moving-average model with zero-mean white noise, sufficient conditions for a stationary invertible representation areThe roots of $1-\varphi z$ and $1+\theta z$ then lie outside the closed unit disc. For a minimal representation these are the usual causality and invertibility root criteria for an ARMA model. A genuinely minimal ARMA(1,1) additionally has nonzero coefficients and no cancellation $\varphi +\theta =0$; if a coefficient vanishes the order is lower. Common factors can reduce the process to white noise, so inequalities on an unreduced representation should not be asserted as necessary for every reduced process. A noncausal stationary solution of an unstable autoregressive equation is a different representation, and does not give the requested innovation-based Wold coefficients below.

Using the backshift operator $B$, write $(1-\varphi B)X_t=(1+\theta B){ϵ}_t$. The absolutely summable geometric series givesThus the Wold representation isIts coefficients are square-summable, so the series converges in mean square and defines a weakly stationary process. Invertibility makes the closed spans of past observations and past innovations equal. Since the current driving white noise is orthogonal to all past innovations, it is orthogonal to all past observations and is the linear innovation of this causal invertible process.

The spectral density of an ARMA process follows from the squared frequency response of the linear filter of a stationary time series:This displayed density uses the usual two-sided convention. For the one-sided spectral density used in Question 5, multiply it by two and restrict to $0\le \omega \le \pi$.

Let ${\psi }_0=1$ and ${\psi }_j=(\varphi +\theta ){\varphi }^{j-1}$ for $j\ge 1$. Orthogonality of the driving white noise gives $\gamma (h)={\sigma }^2{\sum }_{j\ge 0}{\psi }_j{\psi }_{j+h}$ for $h\ge 0$. HenceFor $h\ge 1$, separate the $j=0$ term and sum the remaining geometric series:Negative lags satisfy $\gamma (-h)=\gamma (h)$. With $\varphi =0$, interpret the lag-one factor as one and later powers as zero, recovering the MA(1) covariance.

To express prediction entirely through observations, invert the moving-average factor:The best linear predictor from the whole past is thereforeIndeed $X_t-{\hat{X}}_t={ϵ}_t$ is orthogonal to the closed linear span of the past. For any other past-measurable linear predictor $L$, the mean squared error decomposes as $E[(X_t-L{)}^2]={\sigma }^2+E[({\hat{X}}_t-L{)}^2]$, establishing optimality and error variance ${\sigma }^2$. The result only uses second-order properties; identifying this linear predictor with a full conditional expectation requires stronger assumptions such as independent innovations or joint Gaussianity. In the cancellation case $\theta =-\varphi$ within the stable range, $X_t={ϵ}_t$, the off-zero covariances and predictor vanish, and the spectrum is constant.

Let $\Phi (z)=1-{\varphi }_{1}z-\cdots -{\varphi }_p{z}^p$. The causality root criterion for an autoregressive model,is a sufficient condition for a weakly stationary process solution, and gives the causal solution $X_t={\sum }_{j\ge 0}{\psi }_j{\epsilon }_{t-j}$. Indeed $1/\Phi$ is analytic in a disk of radius greater than one, so its Taylor coefficients ${\psi }_j$ decay geometrically. The series therefore has mean-square convergence and defines a linear filter of a stationary time series; multiplying by $\Phi$ verifies the equation.

For the specified order-two model, put $\phi =(1+\sqrt{5})/2$. Since $\phi -{\phi }^{-1}=1$,When $\alpha \ne 0$, its roots are $1/(\alpha \phi )$ and $-\phi /\alpha$. Both have modulus greater than one exactly when $|\alpha |<1/\phi$. The case $\alpha =0$ is white noise and satisfies the same inequality. ConsequentlyThis is the causal stationarity condition selected above. Without causality, a two-sided autoregressive model can also have a noncausal stationary autoregression when its polynomial has no unit-circle zeros. Thus the inequality is not a necessary condition for all conceivable weak stationarity solutions; the causal interpretation is the one required by the subsequent innovations and forecasting questions.

To find the Wold representation, expand $1/\Phi (z)={\sum }_{j\ge 0}{\psi }_j{z}^j$. Matching coefficients gives ${\psi }_0=1$, ${\psi }_1=\alpha$ and ${\psi }_j=\alpha {\psi }_{j-1}+{\alpha }^2{\psi }_{j-2}$. HenceHere $F_1=F_2=1$ are the Fibonacci numbers. The coefficient formula also follows by partial fractions in the displayed factorization. Absolute summability follows from $|\alpha \phi |<1$ and $|\alpha /\phi |<1$. The past observation span equals the past noise span, because the causal series gives one inclusion and ${\epsilon }_t=X_t-\alpha X_{t-1}-{\alpha }^2{X}_{t-2}$ gives the reverse inclusion. Thus these noises are the linear innovations, and there is no additional deterministic component. This is the Fibonacci-coefficient autoregression.

The best linear prediction from an infinite past is the orthogonal projection onto the closed span of $X_T,X_{T-1},\dots$. Future white noise is orthogonal to that span. Substitution in the recursion therefore givesThe respective errors are ${\epsilon }_{T+1}$ and $\alpha {\epsilon }_{T+1}+{\epsilon }_{T+2}$, so their mean squared errors are ${\sigma }^2$ and $(1+{\alpha }^2){\sigma }^2$. No normal distribution assumption is needed for these linear forecasts.

A weakly stationary process has finite second moments, a mean $m=E{X}_t$ independent of $t$, and a covariance depending only on the separation of the observations. Its autocovariance function and autocorrelation function arewhere the autocorrelation requires $\gamma (0)>0$.

Write the autoregressive polynomial as $A(z)=1-\alpha z+\alpha z^2$, so the autoregressive model is $A(B)X_t={ϵ}_t$, with $B$ the backshift operator. It is important to distinguish a weakly stationary process from a causal time series: weak stationarity alone does not require $X_t$ to depend only on present and past white noise.

For a two-sided weakly stationary process, the two-sided stationary inverse of an autoregressive polynomial exists and is unique if $A$ has no root on the unit circle. To find the excluded parameters, set $z=e^{i\lambda }$. The imaginary part of $A(z)=0$ givesThe case $\alpha =0$ is harmless. The possible frequencies are $0,\pi ,\pm \pi /3$. Since $A(1)=1$, $A(-1)=1+2\alpha$ and $A(e^{\pm i\pi /3})=1-\alpha$, the exceptional parameters are $-1/2$ and $1$. Thus, with nondegenerate white noise, the literal two-sided answer isFor completeness, away from those values $1/A(z)$ has an absolutely summable Laurent series on an annulus containing the unit circle. Its bilateral coefficients define an $L^2$-convergent linear filter of a stationary time series applied to ${ϵ}_t$, giving the required solution. Applying the same inverse to the equation gives uniqueness. At an exceptional value, the spectral measure of a stationary time series would have to satisfy $|A(e^{i\lambda }){|}^2{\mu }_X(d\lambda )={\sigma }^{2}d\lambda /(2\pi )$. A zero of $A$ on the unit circle makes the resulting density nonintegrable near that zero, contradicting finite variance.

If the usual additional causal time series convention is intended, the causality root criterion for an autoregressive model requires the roots of $A$ to lie outside the closed unit disk. Equivalently, the roots of $u^2-\alpha u+\alpha$ lie strictly inside it. The real quadratic stability conditions reduce to $|\alpha |<1$ and $1+2\alpha >0$, giving the narrower answerThe printed question does not explicitly impose this extra convention. Both answers have therefore been distinguished. If ${\sigma }^2=0$, the exceptional parameters permit nonunique stationary homogeneous solutions rather than the nonexistence conclusion above.

At $\alpha =-1/12$,Consequently the Wold representation isThe coefficients are absolutely summable, and ${\psi }_0=1$. The past of $X$ lies in the closed linear span of the past white noise; conversely ${ϵ}_t=A(B)X_t$ recovers that noise from present and past $X$. Thus ${ϵ}_t$ is orthogonal to the past of $X$ and is its one-step linear prediction error, as required for the Wold decomposition. There is no deterministic component.

Using orthogonality of the white noise, put $H=|h|$ and calculate the autocovariance by geometric sums:HenceIn particular, $\gamma (0)=66{\sigma }^2/65$ and $\gamma (1)=-6{\sigma }^2/65$; these also satisfy the Yule-Walker equations for this autoregressive model.

Use the two-sided spectral density of a stationary process conventionThus white noise of variance ${\sigma }^2$ has density ${\sigma }^2/(2\pi )$.

Absolute summability makes the linear filter of a stationary time series well defined in $L^2$: ${\sum }_r\Vert a_r{X}_{t-r}{\Vert }_2\le \Vert X_0{\Vert }_2{\sum }_r|a_r|<\infty$. Its mean is $m_Y=m_X{\sum }_r{a}_r$. For its autocovariance, continuity of the $L^2$ inner product givesThe sum is absolutely convergent because $|{\gamma }_X(j)|\le {\gamma }_X(0)$. It depends only on $k$, proving weak stationarity. Inserting the spectral inversion formula and exchanging the absolutely dominated sum with the integral yieldsHere the coefficients are real, so the two factors are complex conjugates. This is the spectral density transformation under a linear filter.

Applying a second linear filter of a stationary time series givesThe composition of absolutely summable time-series filters can also be calculated directly:Indeed ${\sum }_{\ell }|c_{\ell }|\le \Vert a{\Vert }_1\Vert b{\Vert }_1$; this bound also justifies the interchange of the two $L^2$ sums. The filter gain multiplies under this composition. The sketches for the two individual filters are:

The series defining the linear filter of a stationary time series converges in $L^2$, because the norm of a tail is at most $\Vert X_0{\Vert }_2$ times the corresponding tail sum of $|a_s|$. Its mean is the original mean times ${\sum }_s{a}_s$. Its covariance is time independent and equalsUnder the summability assumption in (i),We may therefore interchange all the sums. Setting $h=k-s+u$ in the Fourier series givesThe second factor is the conjugate of the first because the coefficients are real. HenceThis spectral density transformation under a linear filter also holds whenever the original spectral measure has a density, without covariance summability: insert its integral representation into the covariance double sum, justified by absolute summability of the filter coefficients.

If an autoregressive model has white noise of positive variance, a unique weakly stationary process solving its equation on all integer times exists precisely when its autoregressive polynomial has no root on the unit circle. In that case $1/A(z)$ has an absolutely summable Laurent series on an annulus containing the unit circle; its coefficients define a bilateral linear filter of a stationary time series. Applying that inverse also proves uniqueness among bounded second-moment solutions. A root on the unit circle forces the candidate spectral density of a stationary process to have a nonintegrable singularity. The stronger causality root criterion for an autoregressive model requires every root outside the closed unit disk.

A second-order stationary time series decomposes into a deterministic component and a one-sided square-summable linear filter of a stationary time series of its mutually orthogonal linear innovations. Normalize $c_0=1$. For a causal invertible ARMA process with no deterministic component, the driving white noise is this innovation process: observations and innovations span the same closed past subspace, while the next innovation is orthogonal to it.