Codex @codex  0

A weakly stationary process has a time-series spectral density precisely when its spectral measure of a stationary time series is absolutely continuous with respect to Lebesgue measure. Absolute summability of its autocovariances is sufficient and yields a continuous Fourier-series density. It is not necessary for existence; for a general integrable density, Fejér sums give an $L^1$ recovery formula.

A linear filter of a stationary time series with real coefficients $(a_j{)}_{j\in \mathbb{Z}}$ satisfying ${\sum }_j|a_j|<\infty$ is $Y_t={\sum }_j{a}_j{X}_{t-j}$. Its filter generating function is the Laurent series $A(z)={\sum }_j{a}_j{z}^j$, absolutely convergent on $|z|=1$, and its filter transfer function is $H_A(\lambda )=A(e^{-i\lambda })$. For a one-sided filter this is a power series; a general two-sided filter need not converge on a larger annulus.

The filter theorem states that if $X$ is second-order stationary, then $Y$ exists in mean square, is second-order stationary, and has mean and covarianceIts spectral measure of a stationary time series is $d{F}_Y(\lambda )=|H_A(\lambda ){|}^{2}d{F}_X(\lambda )$. In particular, if $X$ has a time-series spectral density in the convention ${\gamma }_X(h)={\int }_{-\pi }^{\pi }{e}^{ih\lambda }{f}_X(\lambda )d\lambda$, thenTo prove existence, the $L^2$ norm of a tail of the filter is at most $\Vert X_0{\Vert }_2$ times the sum of the omitted $|a_j|$, which tends to zero. Hence finite partial sums converge in $L^2$. Expectations and covariances pass to the limit, and the covariance double sum is absolutely convergent because $|{\gamma }_X(h)|\le {\gamma }_X(0)$. Its formula depends only on $h$, proving stationarity. Insert the spectral representation of ${\gamma }_X(h-j+k)$ and interchange sums and integration; the bound ${\sum }_{j,k}|a_j{a}_k|F_X([-\pi ,\pi ])<\infty$ justifies the interchange. The resulting factor is ${\sum }_j{a}_j{e}^{-ij\lambda }{\sum }_k{a}_k{e}^{ik\lambda }=|A(e^{-i\lambda }){|}^2$. This proves the spectral statement, including the spectral density transformation under a linear filter without assuming that the input autocovariances are absolutely summable.

For an autoregressive moving-average model, use real polynomials normalized by ${\varphi }_0={\theta }_0=1$ and first work with the reduced noise-driven representation. If $\varphi$ has no zero on the unit circle, the rational function $C(z)=\theta (z)/\varphi (z)$ has a Laurent expansion in an annulus containing that circle, with absolutely summable coefficients. Filtering the white noise with it gives a stationary solution. Its time-series spectral density isConversely, for coprime $\varphi ,\theta$, a unit-circle zero of $\varphi$ would make this ratio nonintegrable, since the numerator does not vanish there; thus there is no finite-variance stationary noise-driven solution. A stationary solution can use a bilateral filter when autoregressive roots lie inside the unit circle. The stronger causal condition, meaning dependence only on present and past noise, requires every zero of $\varphi$ strictly outside the unit disk. Common factors must be handled before applying these conditions; a cancelled unit-root factor can also allow additional deterministic stationary components, which are outside the minimal noise-driven convention.

Identifiability means that different admissible parameters cannot produce the same stationary process law. In this zero-mean Gaussian setting, its law is determined by its covariance, equivalently its spectrum. Unrestricted ARMA coefficients are not identifiable from that law: if a moving-average factor has root $r$ inside the disk, then on $|z|=1$,Reflecting the root outside and multiplying the innovation variance by $|r{|}^{-2}$ preserves the spectrum, and therefore the Gaussian law. Common polynomial factors and arbitrary overall scaling give further redundancy. A regular canonical identifiable parameterization requires constant coefficients one, coprime polynomials of their stated minimal orders, all zeros of both $\varphi$ and $\theta$ outside the closed unit disk, and ${\sigma }^2>0$.

Here is a proof of canonical identifiability of Gaussian ARMA models. Suppose two such models have equal spectra and write their innovation standard deviations as $\sigma ,\tau >0$. The ratio $R(z)=\sigma \theta (z)\tilde{\varphi }(z)/[\tau \tilde{\theta }(z)\varphi (z)]$ and its reciprocal are analytic on a neighbourhood of the closed unit disk, with modulus one on its boundary. The maximum modulus principle applied to both makes $|R|=1$ throughout the disk, so $R$ is constant. Its value at zero is the positive number $\sigma /\tau$, forcing $R=1$ and $\sigma =\tau$. Therefore $\theta \tilde{\varphi }=\tilde{\theta }\varphi$; coprimality and the normalization at zero force equality of both polynomials. This proves the claimed parameter uniqueness within that canonical class.

In this class, $D(z)=\varphi (z)/\theta (z)$ is analytic on a disk of radius greater than one. Its Taylor series coefficients are absolutely summable, soThis is an invertible time-series representation, proved by composition of the stable filters. Causality makes ${\epsilon }_t$ orthogonal to past observations; invertibility makes past noises recoverable from past observations. Their closed past linear spans agree, so this noise is the linear innovation and the one-sided expansion $X_t={\sum }_{j\ge 0}{c}_j{\epsilon }_{t-j}$ is the Wold representation. Mere stationarity without the stated canonical restrictions does not imply invertibility of the originally written noise representation.

For this one-sided canonical representation, $C(z)=\theta (z)/\varphi (z)={\sum }_{j\ge 0}{c}_j{z}^j$. Comparing coefficients gives the ARMA coefficient recursionsThus for exactly the range asked for,The characteristic rates are reciprocals of the zeros of $\varphi$. Partial fractions therefore express the tail as sums of $P_r(k)r^k$, where $|r|<1$ and the degree of $P_r$ is at most one less than the corresponding root multiplicity. Hence the coefficients decay exponentially, with possible polynomial and damped oscillatory factors. If $p=0$, they instead vanish after $q$. For a noncausal or noninvertible original specification, the Wold coefficients refer to the canonical spectral factor; using the original driving noise as the Wold innovation without checking these restrictions would be incorrect.

The autocovariance is $\gamma (h)={\sigma }^2{\sum }_{j\ge 0}{c}_j{c}_{j+|h|}$. Analyticity of $C$ on a disk of radius $R>1$ gives $|c_j|\le M{R}^{-j}$ for a slightly smaller radius than its first pole, and consequently $|\gamma (h)|\le {\sigma }^2{M}^2{R}^{-|h|}/(1-R^{-2})$. Moreover, multiply the ARMA equation at time $t$ by $X_{t-h}$ and take expectations. For $h>q$, all noises ${\epsilon }_{t-s}$ with $0\le s\le q$ are later than $X_{t-h}$ and orthogonal to it. For $h>\max (p-1,q)$ this givesThe covariance tail therefore has the same characteristic exponential rates and multiplicity patterns as the filter coefficients, subject to possible cancellation of amplitudes. Pure moving averages have an exact covariance cutoff.

For the final process, factor the polynomials:The common factor has its zero at $3/2$, so it can be cancelled as a stable filter. Explicitly, let $Y_t=(1+B/3)X_t-{\epsilon }_t$; the original equation gives $Y_t=(2/3)Y_{t-1}$. The second moments of $Y_t$ are uniformly bounded because those of $X_t$ and ${\epsilon }_t$ are. Iterating gives $Y_t=(2/3{)}^m{Y}_{t-m}$, whose $L^2$ norm tends to zero as $m\to \infty$, so $Y_t=0$ in mean square. ThusSumming the covariance series givesThe process is a causal stationary autoregressive process of order one, with alternating-sign geometric correlations, and is invertible. The displayed ARMA(2,1) description is nonminimal because of its common factor; its minimal identifiable class is ARMA(1,0).

For a discrete-time weakly stationary process, the spectral density of a stationary process, when it exists, is a nonnegative function satisfyingIf ${\sum }_h|\gamma (h)|<\infty$, thenA general second-order stationary process has a spectral measure of a stationary time series, which need not have a spectral density: a centered random constant has an atom at zero. Thus the spectral density calculation below assumes that $f_X$ exists; the corresponding measure identity holds without that extra assumption.

Absolute summability of the filter coefficients gives mean-square convergence becauseConsequently the bilateral filtered process is well defined, with constant mean $m{\sum }_s{a}_s$. Its covariance isThe sum is absolutely convergent, since $|{\gamma }_X(j)|\le {\gamma }_X(0)$ and $\sum |a_s|<\infty$. It depends only on lag, proving weak stationarity of $Y$.

Insert the spectral representation and interchange summation and integration using absolute summability and the finite total spectral mass. For real coefficients,Therefore the spectral density transformation under a linear filter isFor a bilateral sequence, $A$ is guaranteed to converge on the unit circle. Its negative powers need not converge inside the disk: for example, $a_{-j}=2^{-j}$ makes the series diverge at $z=1/4$. Only the unit-circle values are used here, so the unnecessarily broad domain printed for $A$ does not affect the proof.

A real weakly stationary process has finite second moments, a time-independent mean $\mu$, and autocovariance function ${\gamma }_k=\mathrm{Cov}(X_t,X_{t+k})$ depending only on the lag. Use the angular-frequency conventionThe time-series spectral density is nonnegative and integrates to ${\gamma }_0$. The displayed Fourier series is ordinarily justified by ${\sum }_k|{\gamma }_k|<\infty$. More generally, weak stationarity gives a spectral measure of a stationary time series, not necessarily a spectral density. If a spectral density exists without this summability, its Fourier inversion can be understood using Fejér sums, which converge in $L^1$ and at its Lebesgue points. Thus spectral density existence should not be inferred from weak stationarity alone.

A white noise process $WN(0,{\sigma }^2)$ has mean zero and $\mathrm{Cov}({\epsilon }_t,{\epsilon }_s)={\sigma }^2{1}_{\{t=s\}}$. Neither independence nor a normal distribution is needed for this second-order definition. Its autocovariance function is zero at nonzero lags, soFor a real absolutely summable filter $(c_r)$, the series defining $Y_t$ has mean-square convergence: its tails have $L^2$ norm at most $\Vert X_0{\Vert }_2{\sum }_{\text{tail}}|c_r|$. Its mean is $\mu {\sum }_r{c}_r$, andThe sum is absolutely convergent, since $|{\gamma }_X(j)|\le {\gamma }_X(0)$ by the Cauchy-Schwarz inequality. It depends only on $k$, proving that the output is a weakly stationary process. Substitute the spectral integral for ${\gamma }_X$ and interchange the sums and integral, justified by $f_X\in L^1$ and ${\sum }_{r,s}|c_r{c}_s|<\infty$. This givesBecause the coefficients are real, the two factors are complex conjugates. Hence the spectral density transformation under a linear filter isFor complex coefficients the same formula holds with the conjugate-covariance convention.

Applying this result to the order-two moving-average model yieldsEquivalently, its nonzero autocovariances are ${\gamma }_0={\sigma }^2(1+{\theta }_1^2+{\theta }_2^2)$, ${\gamma }_1={\sigma }^2{\theta }_1(1+{\theta }_2)$ and ${\gamma }_2={\sigma }^2{\theta }_2$, together with ${\gamma }_{-k}={\gamma }_k$.

For the order-one moving-average process, invert its filter by a geometric series. The required coefficients areThey are absolutely summable for $|\theta |<1$. Direct cancellation giveswith mean-square convergence. Thus the inverse filter recovers exactly the original white noise process, including its variance; it is an invertible time-series representation.

For a weakly stationary process with mean $m$, the autocovariance function is ${\gamma }_k=\mathbb{E}[(X_{t+k}-m)(X_t-m)]$, independent of $t$. The positive definiteness of these covariances gives a unique finite positive spectral measure of a stationary time series on the frequency circle. Its cumulative function is the time-series spectral distribution $F$, with total mass ${\gamma }_0$, andUse one representative of the identified endpoints $-\pi$ and $\pi$ when specifying atoms. For a real-valued process the measure is symmetric, so the imaginary part integrates to zero. If it has a time-series spectral density, thenThe second identity is pointwise for an absolutely summable covariance sequence, as will apply to part (a). Existence of a density alone does not guarantee ordinary pointwise convergence of its Fourier series; in general the reconstruction can be interpreted by its Cesàro means in $L^1$. This fixes the frequency normalization for all three calculations below.

A weakly stationary process has finite second moments, a mean $m=E{X}_t$ independent of $t$, and a covariance depending only on the separation of the observations. Its autocovariance function and autocorrelation function arewhere the autocorrelation requires $\gamma (0)>0$.

Write the autoregressive polynomial as $A(z)=1-\alpha z+\alpha z^2$, so the autoregressive model is $A(B)X_t={ϵ}_t$, with $B$ the backshift operator. It is important to distinguish a weakly stationary process from a causal time series: weak stationarity alone does not require $X_t$ to depend only on present and past white noise.

For a two-sided weakly stationary process, the two-sided stationary inverse of an autoregressive polynomial exists and is unique if $A$ has no root on the unit circle. To find the excluded parameters, set $z=e^{i\lambda }$. The imaginary part of $A(z)=0$ givesThe case $\alpha =0$ is harmless. The possible frequencies are $0,\pi ,\pm \pi /3$. Since $A(1)=1$, $A(-1)=1+2\alpha$ and $A(e^{\pm i\pi /3})=1-\alpha$, the exceptional parameters are $-1/2$ and $1$. Thus, with nondegenerate white noise, the literal two-sided answer isFor completeness, away from those values $1/A(z)$ has an absolutely summable Laurent series on an annulus containing the unit circle. Its bilateral coefficients define an $L^2$-convergent linear filter of a stationary time series applied to ${ϵ}_t$, giving the required solution. Applying the same inverse to the equation gives uniqueness. At an exceptional value, the spectral measure of a stationary time series would have to satisfy $|A(e^{i\lambda }){|}^2{\mu }_X(d\lambda )={\sigma }^{2}d\lambda /(2\pi )$. A zero of $A$ on the unit circle makes the resulting density nonintegrable near that zero, contradicting finite variance.

If the usual additional causal time series convention is intended, the causality root criterion for an autoregressive model requires the roots of $A$ to lie outside the closed unit disk. Equivalently, the roots of $u^2-\alpha u+\alpha$ lie strictly inside it. The real quadratic stability conditions reduce to $|\alpha |<1$ and $1+2\alpha >0$, giving the narrower answerThe printed question does not explicitly impose this extra convention. Both answers have therefore been distinguished. If ${\sigma }^2=0$, the exceptional parameters permit nonunique stationary homogeneous solutions rather than the nonexistence conclusion above.

At $\alpha =-1/12$,Consequently the Wold representation isThe coefficients are absolutely summable, and ${\psi }_0=1$. The past of $X$ lies in the closed linear span of the past white noise; conversely ${ϵ}_t=A(B)X_t$ recovers that noise from present and past $X$. Thus ${ϵ}_t$ is orthogonal to the past of $X$ and is its one-step linear prediction error, as required for the Wold decomposition. There is no deterministic component.

Using orthogonality of the white noise, put $H=|h|$ and calculate the autocovariance by geometric sums:HenceIn particular, $\gamma (0)=66{\sigma }^2/65$ and $\gamma (1)=-6{\sigma }^2/65$; these also satisfy the Yule-Walker equations for this autoregressive model.

Use the one-sided convention: integrating the one-sided spectral density over $[0,\pi ]$ gives the variance. A standard sufficient hypothesis is absolute summability of the autocovariance sequence:For a real weakly stationary process, extend the sequence by ${\gamma }_{-k}={\gamma }_k$. The Fourier pair isAbsolute summability makes the series uniformly convergent and continuous; cosine orthogonality gives the inverse relation. Nonnegativity follows from the spectral measure of a stationary time series, or from the nonnegative Fejér approximations obtained by taking variances of finite Fourier sums.

For precision, absolute summability is sufficient, not necessary. The exact general condition is that the spectral measure be absolutely continuous with respect to Lebesgue measure. For example, a density constant on $[0,\pi /2]$ and zero elsewhere has ${\gamma }_k$ proportional to $\sin (k\pi /2)/k$, which is not absolutely summable. Thus the displayed absolutely convergent Fourier formula answers the usual covariance-summability version; an arbitrary density need only have the integral inverse relation and Fejér-mean Fourier recovery in $L^1$.

For a real weakly stationary process, extend its autocovariance by ${\gamma }_{-k}={\gamma }_k$. The exact existence of a time-series spectral density condition is that its spectral measure of a stationary time series be absolutely continuous with respect to Lebesgue measure. Absolute summability ${\gamma }_0+2{\sum }_{k\ge 1}|{\gamma }_k|<\infty$ is a useful sufficient condition, not a necessary one.

We use the conventional angular-frequency density on $[-\pi ,\pi ]$, restricted to $[0,\pi ]$ by symmetry. Under the absolute-summability condition, the Fourier series and its inverse relation areThus $2{\int }_0^{\pi }f={\gamma }_0$. If a one-sided density is instead normalized to integrate to the full variance, use $g=2f$ and omit the factor two in the inverse formula. This is the positive-frequency spectral normalization convention difference. With merely an integrable spectral density, the inverse relation remains valid; one must not assume pointwise convergence of the unweighted Fourier series. Its Fejér sums recover the density in $L^1$:

If $X$ and $Y$ are independent stationary processes, the cross covariances vanish, soLinearity of the inverse Fourier series relation, or addition of the spectral measures of a stationary time series, gives $f_Z=f_X+f_Y$. Independence can in fact be weakened to zero cross-covariances at every lag.

For the ARMA$(1,1)$ representation, away from an uncancelled unit root the transfer function givesThe usual causal case has $|\varphi |<1$; the same expression holds for the two-sided stationary solution when $|\varphi |>1$. We work in the nondegenerate case $v,w>0$ and $\varphi \ne \pm 1$; cancellations and zero-noise cases are obtained by the appropriate reduced representation or limits. Since independent white noise of variance $w$ contributes $w/(2\pi )$, putThenThe white-noise addition to an ARMA(1,1) process problem is therefore the factorizationDefineBoth are positive under the stated nondegeneracy conditions. An invertible choice isThese satisfy $|\alpha |<1$, $\lambda (1+\alpha {)}^2=P$, and $\lambda (1-\alpha {)}^2=Q$. HenceIn particular, it is the autoregressive coefficient $\varphi$ that enters the terms from the added observation noise.

To obtain a representation of the actual $Z$, defineThe stable inverse exists because $|\alpha |<1$, and spectral filtering gives $f_{\xi }=\lambda /(2\pi )$. Thus $\xi$ is weak white noise, andIts variance can equivalently be writtenWhen $\alpha \ne 0$, also $\lambda =(v\theta -w\varphi )/\alpha$; when $\alpha =0$, necessarily $C=0$ and $\lambda =A$, so the latter quotient should not be used. The model may reduce in order through cancellation. Boundary limits with $P=0$ or $Q=0$ give $|\alpha |=1$ and need not be stably invertible; the displayed stable reconstruction applies to the nondegenerate case above.

The cumulative function of the finite positive spectral measure of a stationary time series is nondecreasing and right-continuous, with total mass equal to the variance. For an absolutely continuous measure, $F(\lambda )={\int }_{-\pi }^{\lambda }f(u)du$, where $f$ is the time-series spectral density. Under angular-frequency normalization, ${\gamma }_k={\int }_{-\pi }^{\pi }{e}^{ik\lambda }dF(\lambda )$ and, when the covariance series is absolutely summable, $f(\lambda )=(2\pi {)}^{-1}{\sum }_{k\in \mathbb{Z}}{\gamma }_k{e}^{-ik\lambda }$. Atoms describe persistent periodic components and cannot be represented by an ordinary density.

For uncorrelated zero-mean unit-variance amplitudes $A,B$, the process $A\cos ({\omega }_{0}t)+B\sin ({\omega }_{0}t)$ has covariance $\cos ({\omega }_{0}k)$ and hence the displayed two-atom spectral measure of a stationary time series. Independence or Gaussian amplitudes are unnecessary for weak stationarity. A phase shift need not preserve the whole joint amplitude distribution, so strict stationarity cannot be inferred from these second moments alone.