Codex @codex  0

For a weakly stationary process with mean $m$, the autocovariance function is ${\gamma }_k=\mathbb{E}[(X_{t+k}-m)(X_t-m)]$, independent of $t$. The positive definiteness of these covariances gives a unique finite positive spectral measure of a stationary time series on the frequency circle. Its cumulative function is the time-series spectral distribution $F$, with total mass ${\gamma }_0$, andUse one representative of the identified endpoints $-\pi$ and $\pi$ when specifying atoms. For a real-valued process the measure is symmetric, so the imaginary part integrates to zero. If it has a time-series spectral density, thenThe second identity is pointwise for an absolutely summable covariance sequence, as will apply to part (a). Existence of a density alone does not guarantee ordinary pointwise convergence of its Fourier series; in general the reconstruction can be interpreted by its Cesàro means in $L^1$. This fixes the frequency normalization for all three calculations below.

Define the autoregressive polynomial $\Phi (z)=1-{\sum }_{k=1}^p{\varphi }_k{z}^k$ and the moving-average polynomial $\Theta (z)=1+{\sum }_{k=1}^q{\theta }_k{z}^k$. The standard causality and invertibility root criteria for an ARMA model areIn the usual minimal autoregressive moving-average model, the two polynomials have no common roots. Under that convention, these are the necessary and sufficient root conditions for a causal and invertible representation with the given noise. Without minimality they remain sufficient; shared factors must be considered before claiming necessity.

To see why the conditions work, both ratios $C(z)=\Theta (z)/\Phi (z)$ and $D(z)=\Phi (z)/\Theta (z)$ are analytic on a disc of radius greater than one. Their power series coefficients therefore decay geometrically and are absolutely summable. The backshift operator $B{X}_t=X_{t-1}$ then gives a well-defined mean-square convergent causal time-series representationFiltering white noise by these fixed coefficients gives a weakly stationary process. The identity $\Phi (B)C(B)=\Theta (B)$ verifies the original equation. Multiplication by the absolutely summable inverse of $\Phi (B)$ also shows uniqueness among finite-variance stationary solutions. Similarly $D(B)X_t={ϵ}_t$ gives the invertible time-series representation.

Comparing coefficients in $\Phi (z)C(z)=\Theta (z)$ derives the first of the ARMA coefficient recursions:Here $c_j=0$ for negative $j$, ${\theta }_0=1$, and ${\theta }_j=0$ for $j>q$, so in particular $c_0=1$. For the inverse filter, compare coefficients in $\Theta (z)D(z)=\Phi (z)$. With $a_0=1$, $a_j=-{\varphi }_j$ for $1\le j\le p$, $a_j=0$ for $j>p$, and $d_j=0$ for negative indices, this givesThe minus sign in the inverse recursion comes from moving the nonconstant moving-average terms to the other side.

For the first-order model, the standard uncancelled conditions are $|\varphi |<1$ and $|\theta |<1$. The geometric power series expansions giveThus the ARMA(1,1) causal and inverse coefficients areAt $j=1$ these give $c_1=\varphi +\theta$ and $d_1=-(\varphi +\theta )$, including zero-valued parameters without any negative powers. If $\theta =-\varphi$, the filtered process reduces to $X_t={ϵ}_t$ and all later coefficients vanish. This common factor cancellation in an ARMA model is why the root inequalities should not be advertised as necessary for every nonminimal parameterization. The stated inequalities still ensure uniqueness for the original equation as well as the desired causal and invertible filters.

A weakly stationary process has finite second moments, a mean $m=E{X}_t$ independent of $t$, and a covariance depending only on the separation of the observations. Its autocovariance function and autocorrelation function arewhere the autocorrelation requires $\gamma (0)>0$.

Write the autoregressive polynomial as $A(z)=1-\alpha z+\alpha z^2$, so the autoregressive model is $A(B)X_t={ϵ}_t$, with $B$ the backshift operator. It is important to distinguish a weakly stationary process from a causal time series: weak stationarity alone does not require $X_t$ to depend only on present and past white noise.

For a two-sided weakly stationary process, the two-sided stationary inverse of an autoregressive polynomial exists and is unique if $A$ has no root on the unit circle. To find the excluded parameters, set $z=e^{i\lambda }$. The imaginary part of $A(z)=0$ givesThe case $\alpha =0$ is harmless. The possible frequencies are $0,\pi ,\pm \pi /3$. Since $A(1)=1$, $A(-1)=1+2\alpha$ and $A(e^{\pm i\pi /3})=1-\alpha$, the exceptional parameters are $-1/2$ and $1$. Thus, with nondegenerate white noise, the literal two-sided answer isFor completeness, away from those values $1/A(z)$ has an absolutely summable Laurent series on an annulus containing the unit circle. Its bilateral coefficients define an $L^2$-convergent linear filter of a stationary time series applied to ${ϵ}_t$, giving the required solution. Applying the same inverse to the equation gives uniqueness. At an exceptional value, the spectral measure of a stationary time series would have to satisfy $|A(e^{i\lambda }){|}^2{\mu }_X(d\lambda )={\sigma }^{2}d\lambda /(2\pi )$. A zero of $A$ on the unit circle makes the resulting density nonintegrable near that zero, contradicting finite variance.

If the usual additional causal time series convention is intended, the causality root criterion for an autoregressive model requires the roots of $A$ to lie outside the closed unit disk. Equivalently, the roots of $u^2-\alpha u+\alpha$ lie strictly inside it. The real quadratic stability conditions reduce to $|\alpha |<1$ and $1+2\alpha >0$, giving the narrower answerThe printed question does not explicitly impose this extra convention. Both answers have therefore been distinguished. If ${\sigma }^2=0$, the exceptional parameters permit nonunique stationary homogeneous solutions rather than the nonexistence conclusion above.

At $\alpha =-1/12$,Consequently the Wold representation isThe coefficients are absolutely summable, and ${\psi }_0=1$. The past of $X$ lies in the closed linear span of the past white noise; conversely ${ϵ}_t=A(B)X_t$ recovers that noise from present and past $X$. Thus ${ϵ}_t$ is orthogonal to the past of $X$ and is its one-step linear prediction error, as required for the Wold decomposition. There is no deterministic component.

Using orthogonality of the white noise, put $H=|h|$ and calculate the autocovariance by geometric sums:HenceIn particular, $\gamma (0)=66{\sigma }^2/65$ and $\gamma (1)=-6{\sigma }^2/65$; these also satisfy the Yule-Walker equations for this autoregressive model.

Use the one-sided convention: integrating the one-sided spectral density over $[0,\pi ]$ gives the variance. A standard sufficient hypothesis is absolute summability of the autocovariance sequence:For a real weakly stationary process, extend the sequence by ${\gamma }_{-k}={\gamma }_k$. The Fourier pair isAbsolute summability makes the series uniformly convergent and continuous; cosine orthogonality gives the inverse relation. Nonnegativity follows from the spectral measure of a stationary time series, or from the nonnegative Fejér approximations obtained by taking variances of finite Fourier sums.

For precision, absolute summability is sufficient, not necessary. The exact general condition is that the spectral measure be absolutely continuous with respect to Lebesgue measure. For example, a density constant on $[0,\pi /2]$ and zero elsewhere has ${\gamma }_k$ proportional to $\sin (k\pi /2)/k$, which is not absolutely summable. Thus the displayed absolutely convergent Fourier formula answers the usual covariance-summability version; an arbitrary density need only have the integral inverse relation and Fejér-mean Fourier recovery in $L^1$.

For a real weakly stationary process, extend its autocovariance by ${\gamma }_{-k}={\gamma }_k$. The exact existence of a time-series spectral density condition is that its spectral measure of a stationary time series be absolutely continuous with respect to Lebesgue measure. Absolute summability ${\gamma }_0+2{\sum }_{k\ge 1}|{\gamma }_k|<\infty$ is a useful sufficient condition, not a necessary one.

We use the conventional angular-frequency density on $[-\pi ,\pi ]$, restricted to $[0,\pi ]$ by symmetry. Under the absolute-summability condition, the Fourier series and its inverse relation areThus $2{\int }_0^{\pi }f={\gamma }_0$. If a one-sided density is instead normalized to integrate to the full variance, use $g=2f$ and omit the factor two in the inverse formula. This is the positive-frequency spectral normalization convention difference. With merely an integrable spectral density, the inverse relation remains valid; one must not assume pointwise convergence of the unweighted Fourier series. Its Fejér sums recover the density in $L^1$:

If $X$ and $Y$ are independent stationary processes, the cross covariances vanish, soLinearity of the inverse Fourier series relation, or addition of the spectral measures of a stationary time series, gives $f_Z=f_X+f_Y$. Independence can in fact be weakened to zero cross-covariances at every lag.

For the ARMA$(1,1)$ representation, away from an uncancelled unit root the transfer function givesThe usual causal case has $|\varphi |<1$; the same expression holds for the two-sided stationary solution when $|\varphi |>1$. We work in the nondegenerate case $v,w>0$ and $\varphi \ne \pm 1$; cancellations and zero-noise cases are obtained by the appropriate reduced representation or limits. Since independent white noise of variance $w$ contributes $w/(2\pi )$, putThenThe white-noise addition to an ARMA(1,1) process problem is therefore the factorizationDefineBoth are positive under the stated nondegeneracy conditions. An invertible choice isThese satisfy $|\alpha |<1$, $\lambda (1+\alpha {)}^2=P$, and $\lambda (1-\alpha {)}^2=Q$. HenceIn particular, it is the autoregressive coefficient $\varphi$ that enters the terms from the added observation noise.

To obtain a representation of the actual $Z$, defineThe stable inverse exists because $|\alpha |<1$, and spectral filtering gives $f_{\xi }=\lambda /(2\pi )$. Thus $\xi$ is weak white noise, andIts variance can equivalently be writtenWhen $\alpha \ne 0$, also $\lambda =(v\theta -w\varphi )/\alpha$; when $\alpha =0$, necessarily $C=0$ and $\lambda =A$, so the latter quotient should not be used. The model may reduce in order through cancellation. Boundary limits with $P=0$ or $Q=0$ give $|\alpha |=1$ and need not be stably invertible; the displayed stable reconstruction applies to the nondegenerate case above.

A noncausal stationary autoregression exists. On the two-sided time axis defineThis series converges in the sense of mean-square convergence, and direct subtraction gives $X_t-2X_{t-1}={\epsilon }_t$. The expected value and autocovariance areConsequently this is a weakly stationary process and, by Gaussianity, a strictly stationary process. It is an example of noncausal stationary autoregression, since $X_t$ uses future noise. In particular, $X_{t-1}$ is correlated with ${\epsilon }_t$, so the usual causal variance recursion is inapplicable. If an additional assumption required the driving noise to be independent of past observations, this solution would be excluded; that assumption is not stated here.

No weakly stationary process solves the model with nondegenerate noise. Iteration would giveFor a weakly stationary process with finite variance $V$, the Cauchy-Schwarz inequality instead gives $\mathrm{Var}(X_t-X_{t-m})=2V-2\gamma (m)\le 4V$. These two bounds contradict each other for large $m$. This argument does not assume that $X_{t-m}$ is independent of the intervening noise, so it excludes noncausal solutions too. The corresponding unit-root autoregressive process has persistent random walk behavior rather than stationary fluctuations.

A stationary causal time series exists. The infinite moving-average representationconverges in the sense of mean-square convergence because the driving variables are independent random variables and ${\sum }_j{4}^{-j}<\infty$. Shifting the series gives $X_t=\frac{1}{2}{X}_{t-1}+{\epsilon }_t$. Its expected value is zero and its autocovariance isThus it is a weakly stationary process; since the driving variables have a normal distribution, it is also a Gaussian process and a strictly stationary process. Throughout the stationarity discussion, take ${\sigma }^2>0$; degenerate zero noise permits trivial constant solutions.

A weakly stationary process has finite second moments, a constant expected value $\mu$, and covariance depending only on the time difference:A strictly stationary process, also called strongly stationary, has every finite-dimensional probability distribution invariant under a common shift: for every finite choice of times and every shift $h$, $(X_{t_1+h},\dots ,X_{t_m+h})$ and $(X_{t_1},\dots ,X_{t_m})$ have the same law. strict stationarity with finite second moments implies weak stationarity. weak stationarity alone does not determine the full joint law, and strict stationarity alone does not guarantee finite moments.

For a centered weakly stationary process with $\gamma (0)>0$, the autocorrelation function isFor $h\ge 2$, let $P$ be orthogonal projection in L2 space onto the linear span of $X_{t-1},\dots ,X_{t-h+1}$. The lag-$h$ partial autocorrelation function is the correlation coefficient of $X_t-P{X}_t$ and $X_{t-h}-P{X}_{t-h}$. It removes the linear contribution of the intervening observations. At lag one it is just $\rho (1)$. For nonsingular prediction covariance matrices, it is equivalently the last coefficient $a_{hh}$ in the order-$h$ linear predictor of $X_t$ from $X_{t-1},\dots ,X_{t-h}$. Equal residual variances, by stationarity, identify that coefficient with the residual correlation. If a residual has zero variance, this correlation is undefined; the nondegeneracy condition matters.

For the spectral questions take the unique weakly stationary process solving the equation, as is customary for a stable autoregressive moving-average model. The equation alone also admits nonstationary solutions differing by $C{4}^{-t}$, for which a spectral density of a stationary process need not exist. Let $B$ be the backshift operator. Since the autoregressive root is $4$, the stationary solution is causal andUsing the convention $\gamma (h)={\int }_{-\pi }^{\pi }{e}^{ih\lambda }f(\lambda )d\lambda$, the spectral density of a stationary process isThe stationary mean is zero, since $\mu -\mu /4=0$.

For a weakly stationary process, every observation has the same finite expectation $\mu$. Linearity of expectation, without any assumption of independence, givesThe variance is a sum of all pairwise covariances:For a real-valued weakly stationary process, the autocovariance is even. There are $n$ pairs at lag zero and $n-h$ pairs in each direction at positive lag $h$. ThereforePositive autocovariances inflate the uncertainty relative to the independent-observation formula; negative terms may reduce it. An unbiased estimator need not have variance tending to zero: a process consisting of the same random variable at every time is weakly stationary, but averaging cannot reduce its variance.

For a known mean zero, a common sample autocovariance function convention isThe weakly stationary process property gives $\mathbb{E}[Y_t{Y}_{t+h}]=\gamma (h)$, soThe lag-zero estimate is unbiased, but at nonzero lags the divisor $n$ causes a finite-sample multiplicative bias. An alternative convention uses $(n-h{)}^{-1}$ instead; for that convention the expectation is exactly $\gamma (h)$. These are two normalizations of the sample autocovariance function, so the divisor must be specified. Here no estimated-mean centering term occurs because zero is known. Replacing zero by a sample mean would introduce additional bias terms. Extend to negative lags by symmetry; there are no observed lagged pairs for $h\ge n$.

A zero-mean autoregressive process of order one satisfieswhere ${\epsilon }_t$ is white noise with variance ${\sigma }_{\epsilon }^2>0$. A causal time series uses present and past innovations only. For $|\alpha |<1$ the serieshas mean-square convergence because ${\sum }_j{\alpha }^{2j}<\infty$, and it supplies the causal weakly stationary process. Its variance is ${\sigma }_{\epsilon }^2/(1-{\alpha }^2)$. For $h\ge 0$, only the common innovations contribute to the autocovariance:Evenness then givesWhen $\alpha <0$, the signs alternate; when $\alpha =0$, the process is white noise. With nondegenerate innovations, $|\alpha |\ge 1$ does not give a causal finite-variance stationary solution. These covariance calculations need uncorrelated innovations, not Gaussianity; Gaussian innovations additionally yield a Gaussian process.

A weakly stationary process has a constant finite mean and covariance $\mathrm{Cov}(X_t,X_{t+h})$ depending only on lag $h$. The plotted process is not stationary: it has a declining trend and a pronounced oscillation of period about $25$, so its mean depends on time.

A process $(X_t{)}_{t\in \mathbb{Z}}$ is weakly stationary when it has finite second moments, a time-independent mean $\mathbb{E}{X}_t=\mu$, and an autocovariance functionthat depends only on the lag.

A process has period-$S$ second-order statistics when $E{X}_{t+S}=E{X}_t$ and $\mathrm{Cov}(X_{t+S},X_{s+S})=\mathrm{Cov}(X_t,X_s)$ for all times. Its variance can vary by season, so it need not be a weakly stationary process. A periodic covariance period need not be the fundamental period.

For a real weakly stationary process, restricting its even angular-frequency density $f$ to $[0,\pi ]$ gives $\gamma (k)=2{\int }_0^{\pi }f(\omega )\cos (k\omega )d\omega$. Folding both frequency halves into one density gives $g=2f$ and removes that factor two. Consequently white noise of variance $v$ has restricted density $v/(2\pi )$ and folded density $v/\pi$.

For a known zero mean, ${\hat{\gamma }}_n(h)=n^{-1}{\sum }_{t=1}^{n-h}{Y}_t{Y}_{t+h}$ for $0\le h<n$. Weakly stationary process assumptions give expectation $(1-h/n)\gamma (h)$. Dividing by $n-h$ instead gives an unbiased estimate at each individual lag, although the resulting collection need not form a positive semidefinite covariance sequence.

In a second-order sense a weakly stationary process has short memory when its autocovariance is absolutely summable. Then its spectral density of a stationary process is continuous and its sample-mean variance has the usual finite long-run variance of a stationary process. A central limit theorem still needs additional conditions.