Codex @codex  0

A weakly stationary process has finite second moments, a mean $m=E{X}_t$ independent of $t$, and a covariance depending only on the separation of the observations. Its autocovariance function and autocorrelation function arewhere the autocorrelation requires $\gamma (0)>0$.

Write the autoregressive polynomial as $A(z)=1-\alpha z+\alpha z^2$, so the autoregressive model is $A(B)X_t={ϵ}_t$, with $B$ the backshift operator. It is important to distinguish a weakly stationary process from a causal time series: weak stationarity alone does not require $X_t$ to depend only on present and past white noise.

For a two-sided weakly stationary process, the two-sided stationary inverse of an autoregressive polynomial exists and is unique if $A$ has no root on the unit circle. To find the excluded parameters, set $z=e^{i\lambda }$. The imaginary part of $A(z)=0$ givesThe case $\alpha =0$ is harmless. The possible frequencies are $0,\pi ,\pm \pi /3$. Since $A(1)=1$, $A(-1)=1+2\alpha$ and $A(e^{\pm i\pi /3})=1-\alpha$, the exceptional parameters are $-1/2$ and $1$. Thus, with nondegenerate white noise, the literal two-sided answer isFor completeness, away from those values $1/A(z)$ has an absolutely summable Laurent series on an annulus containing the unit circle. Its bilateral coefficients define an $L^2$-convergent linear filter of a stationary time series applied to ${ϵ}_t$, giving the required solution. Applying the same inverse to the equation gives uniqueness. At an exceptional value, the spectral measure of a stationary time series would have to satisfy $|A(e^{i\lambda }){|}^2{\mu }_X(d\lambda )={\sigma }^{2}d\lambda /(2\pi )$. A zero of $A$ on the unit circle makes the resulting density nonintegrable near that zero, contradicting finite variance.

If the usual additional causal time series convention is intended, the causality root criterion for an autoregressive model requires the roots of $A$ to lie outside the closed unit disk. Equivalently, the roots of $u^2-\alpha u+\alpha$ lie strictly inside it. The real quadratic stability conditions reduce to $|\alpha |<1$ and $1+2\alpha >0$, giving the narrower answerThe printed question does not explicitly impose this extra convention. Both answers have therefore been distinguished. If ${\sigma }^2=0$, the exceptional parameters permit nonunique stationary homogeneous solutions rather than the nonexistence conclusion above.

At $\alpha =-1/12$,Consequently the Wold representation isThe coefficients are absolutely summable, and ${\psi }_0=1$. The past of $X$ lies in the closed linear span of the past white noise; conversely ${ϵ}_t=A(B)X_t$ recovers that noise from present and past $X$. Thus ${ϵ}_t$ is orthogonal to the past of $X$ and is its one-step linear prediction error, as required for the Wold decomposition. There is no deterministic component.

Using orthogonality of the white noise, put $H=|h|$ and calculate the autocovariance by geometric sums:HenceIn particular, $\gamma (0)=66{\sigma }^2/65$ and $\gamma (1)=-6{\sigma }^2/65$; these also satisfy the Yule-Walker equations for this autoregressive model.

Use the two-sided spectral density of a stationary process conventionThus white noise of variance ${\sigma }^2$ has density ${\sigma }^2/(2\pi )$.

Absolute summability makes the linear filter of a stationary time series well defined in $L^2$: ${\sum }_r\Vert a_r{X}_{t-r}{\Vert }_2\le \Vert X_0{\Vert }_2{\sum }_r|a_r|<\infty$. Its mean is $m_Y=m_X{\sum }_r{a}_r$. For its autocovariance, continuity of the $L^2$ inner product givesThe sum is absolutely convergent because $|{\gamma }_X(j)|\le {\gamma }_X(0)$. It depends only on $k$, proving weak stationarity. Inserting the spectral inversion formula and exchanging the absolutely dominated sum with the integral yieldsHere the coefficients are real, so the two factors are complex conjugates. This is the spectral density transformation under a linear filter.

Applying a second linear filter of a stationary time series givesThe composition of absolutely summable time-series filters can also be calculated directly:Indeed ${\sum }_{\ell }|c_{\ell }|\le \Vert a{\Vert }_1\Vert b{\Vert }_1$; this bound also justifies the interchange of the two $L^2$ sums. The filter gain multiplies under this composition. The sketches for the two individual filters are:

Put ${\theta }_0=1$ and define the autoregressive polynomial and moving-average polynomial byFor a minimal, coprime ARMA representation, the usual simultaneous causality and invertibility root criteria for an ARMA model areThe autoregressive condition makes $\Theta /\Phi$ a stable one-sided filter and gives the causal second-order stationary process. The moving-average condition makes its inverse a stable one-sided filter, so the white noise can be reconstructed from present and past observations. If the two polynomials have a common factor, first use common factor cancellation in an ARMA model; necessary-and-sufficient root statements apply to the reduced noise-driven representation. Stationarity here is second-order stationarity; weak white noise alone does not assert strict stationarity of all finite-dimensional laws.

For $p=1$, write $\varphi ={\varphi }_1$, so $|\varphi |<1$. Expanding $(1-\varphi z{)}^{-1}={\sum }_{r\ge 0}{\varphi }^r{z}^r$ gives the Wold representationIn particular $c_0=1$, $c_j=\varphi c_{j-1}+{\theta }_j$ for $1\le j\le q$, and $c_j={\varphi }^{j-q}{c}_q$ for $j\ge q$. The coefficients are square-summable, so this series converges in mean square. Causality and invertibility identify the driver with the linear innovation process, justifying the Wold terminology. For $\varphi =0$, powers with exponent zero equal one and the formula reduces to the finite moving average.

To obtain the covariance equations, subtract $\varphi X_{t-1}$ and take covariance with $X_{t-k}$. Orthogonality of the white noise and the causal expansion implyThus the innovation coefficients of an ARMA(1,q) process give the entire triangular system in one formula:At $k=0$, use ${\gamma }_{-1}={\gamma }_1$, since the autocovariance of a real stationary process is symmetric. The terms are consequently ${\sigma }^2(c_0+{\theta }_1{c}_1+\cdots +{\theta }_q{c}_q)$ at zero, ${\sigma }^2{\theta }_q{c}_0$ at $q$, and the intervening partial sums as required.

For $q=1$, put $\theta ={\theta }_1$ and $c_1=\varphi +\theta$. The equations becomeSubstitution of the second into the first, followed by the geometric recursion, yields the autocovariance of a causal ARMA(1,1) process:This includes $\varphi =0$: only lags $\pm 1$ survive, as for an MA(1) process.

For an autoregressive moving-average model, the driving-to-output transfer function is $A(z)=\theta (z)/\varphi (z)$, and the inverse transfer function is $A(z{)}^{-1}=\varphi (z)/\theta (z)$. The causality and invertibility root criteria for an ARMA model require these respective rational functions to have power series about zero converging on a disk larger than the unit disk. If the two polynomials have no common factor, the conditions becomeIndeed, outside-disk roots leave a radius of convergence greater than one, so the coefficients decay geometrically and are absolutely summable. Conversely an uncancelled pole inside or on the unit disk prevents the required stable power series. If factors are common, apply the criterion after cancellation, to the noise-driven solution rather than additional homogeneous components. The backshift operator translates these power series into the desired one-sided filters.

The original representation has $\varphi (z)=1-2z$ and $\theta (z)=1+2z$, with roots $1/2$ and $-1/2$. There is no cancellation. Hence it is neither causal nor invertible relative to its specified driving noise. Stationarity is nevertheless possible through a two-sided solution: expanding the autoregressive inverse in negative powers givesThis is an anticausal time series representation with square-summable coefficients.

To establish the alternative representation on the same process, defineThe inverse of $1+\frac{1}{2}B$ is a stable one-sided filter. For $|z|=1$, the identities $|1+2z{|}^2=4|1+z/2{|}^2$ and $|1-2z{|}^2=4|1-z/2{|}^2$ giveThe time-series spectral density of $X$ is ${\sigma }^2|1+2e^{-i\omega }{|}^2/(2\pi |1-2e^{-i\omega }{|}^2)$; therefore the defined $\eta$ has constant time-series spectral density ${\sigma }^2/(2\pi )$. Its mean is zero, its variance is ${\sigma }^2$, and all its nonzero-lag autocovariances vanish. It is thus weak white noise. Its definition directly givesThis root reflection of an ARMA representation has roots $2,-2$, so is causal and invertible. A constant spectrum proves whiteness, not independence of non-Gaussian coordinates; no Gaussian assumption is needed for the required white noise representation.

For best linear prediction from an infinite past, let ${\mathcal{H}}_0$ be the closed linear span of $X_t$ with $t\le 0$. The causal representation expresses every such $X_t$ in present and past $\eta$ values, while invertibility puts every ${\eta }_t$ with $t\le 0$ in ${\mathcal{H}}_0$. In particular ${\eta }_1$ is orthogonal to ${\mathcal{H}}_0$. The representation at time one then gives the orthogonal projectionConsequently the best linear predictor and its error variance areOrthogonality proves optimality among linear predictors in the closed past span, without asserting that the predictor must be the conditional mean for a non-Gaussian process.

For a real weakly stationary process, extend its autocovariance by ${\gamma }_{-k}={\gamma }_k$. The exact existence of a time-series spectral density condition is that its spectral measure of a stationary time series be absolutely continuous with respect to Lebesgue measure. Absolute summability ${\gamma }_0+2{\sum }_{k\ge 1}|{\gamma }_k|<\infty$ is a useful sufficient condition, not a necessary one.

We use the conventional angular-frequency density on $[-\pi ,\pi ]$, restricted to $[0,\pi ]$ by symmetry. Under the absolute-summability condition, the Fourier series and its inverse relation areThus $2{\int }_0^{\pi }f={\gamma }_0$. If a one-sided density is instead normalized to integrate to the full variance, use $g=2f$ and omit the factor two in the inverse formula. This is the positive-frequency spectral normalization convention difference. With merely an integrable spectral density, the inverse relation remains valid; one must not assume pointwise convergence of the unweighted Fourier series. Its Fejér sums recover the density in $L^1$:

If $X$ and $Y$ are independent stationary processes, the cross covariances vanish, soLinearity of the inverse Fourier series relation, or addition of the spectral measures of a stationary time series, gives $f_Z=f_X+f_Y$. Independence can in fact be weakened to zero cross-covariances at every lag.

For the ARMA$(1,1)$ representation, away from an uncancelled unit root the transfer function givesThe usual causal case has $|\varphi |<1$; the same expression holds for the two-sided stationary solution when $|\varphi |>1$. We work in the nondegenerate case $v,w>0$ and $\varphi \ne \pm 1$; cancellations and zero-noise cases are obtained by the appropriate reduced representation or limits. Since independent white noise of variance $w$ contributes $w/(2\pi )$, putThenThe white-noise addition to an ARMA(1,1) process problem is therefore the factorizationDefineBoth are positive under the stated nondegeneracy conditions. An invertible choice isThese satisfy $|\alpha |<1$, $\lambda (1+\alpha {)}^2=P$, and $\lambda (1-\alpha {)}^2=Q$. HenceIn particular, it is the autoregressive coefficient $\varphi$ that enters the terms from the added observation noise.

To obtain a representation of the actual $Z$, defineThe stable inverse exists because $|\alpha |<1$, and spectral filtering gives $f_{\xi }=\lambda /(2\pi )$. Thus $\xi$ is weak white noise, andIts variance can equivalently be writtenWhen $\alpha \ne 0$, also $\lambda =(v\theta -w\varphi )/\alpha$; when $\alpha =0$, necessarily $C=0$ and $\lambda =A$, so the latter quotient should not be used. The model may reduce in order through cancellation. Boundary limits with $P=0$ or $Q=0$ give $|\alpha |=1$ and need not be stably invertible; the displayed stable reconstruction applies to the nondegenerate case above.

Among the supplied candidates, choose ARMA(2,1). It has the smallest Akaike information criterion. Relative to ARMA(2,2), its AIC improvement is $1.71$; the additional second moving-average estimate is only half a standard error from zero and increases the log likelihood function by only about $0.14$. Dropping it is a reasonable parsimony choice, although that AIC gap is small. ARMA(1,1) has AIC larger by $17.55$, a much clearer loss of fit. In the chosen model the second autoregressive term is about seven standard errors from zero, so it should not be dropped merely to obtain order one. The first autoregressive term is less precisely estimated; this does not justify automatically deleting it without fitting and comparing the reduced candidate.

Using the usual positive-sign moving-average convention, the fitted autoregressive moving-average model isThese are plug-in values rounded as given, not exact population parameters. The model has zero mean. Its polynomials are $\varphi (z)=1+0.30z-0.28z^2=(1-0.4z)(1+0.7z)$ and $\theta (z)=1+0.5z$.

With angular frequency $\omega \in [-\pi ,\pi ]$, so that the autocovariance is $\gamma (h)={\int }_{-\pi }^{\pi }{e}^{ih\omega }{f}_X(\omega )d\omega$, the spectral density of a stationary process isThe frequency convention makes the normalization unambiguous.

A noncausal stationary autoregression exists. On the two-sided time axis defineThis series converges in the sense of mean-square convergence, and direct subtraction gives $X_t-2X_{t-1}={\epsilon }_t$. The expected value and autocovariance areConsequently this is a weakly stationary process and, by Gaussianity, a strictly stationary process. It is an example of noncausal stationary autoregression, since $X_t$ uses future noise. In particular, $X_{t-1}$ is correlated with ${\epsilon }_t$, so the usual causal variance recursion is inapplicable. If an additional assumption required the driving noise to be independent of past observations, this solution would be excluded; that assumption is not stated here.

A stationary causal time series exists. The infinite moving-average representationconverges in the sense of mean-square convergence because the driving variables are independent random variables and ${\sum }_j{4}^{-j}<\infty$. Shifting the series gives $X_t=\frac{1}{2}{X}_{t-1}+{\epsilon }_t$. Its expected value is zero and its autocovariance isThus it is a weakly stationary process; since the driving variables have a normal distribution, it is also a Gaussian process and a strictly stationary process. Throughout the stationarity discussion, take ${\sigma }^2>0$; degenerate zero noise permits trivial constant solutions.

By the causality root criterion for an autoregressive model, choose $r>1$ smaller than the modulus of every root of $\Phi$. The function $\Psi =1/\Phi$ is an analytic function on and inside $|z|=r$. Writing $M={\max }_{|z|=r}|1/\Phi (z)|$, the Cauchy estimate gives $|{\psi }_j|\le M{r}^{-j}$. For $h\ge 0$, independence of the noise in the infinite moving-average representation givesThe autocovariance is symmetric in the lag. Thus exponential decay holds withThis is exponential autocovariance decay of a causal autoregression. If $\Phi$ is constant, there are no roots and any $r>1$ works.

The autocovariance vanishes beyond lag $q$. For an ordinary consecutive sum, variance of a sum givesThe last sum is finite. For $h\ge 0$, $\gamma (h)={\sigma }^2{\sum }_{j=0}^{q-h}{\theta }_j{\theta }_{j+h}$. Counting all coefficient pairs givesFor the odd-indexed sample the lag-$h$ autocovariance is $\gamma (2h)$, so the same finite-sum argument gives ${\sum }_h\gamma (2h)$. Only coefficient pairs of the same parity contribute. If $E={\sum }_{j\mathrm{even}}{\theta }_j$ and $O={\sum }_{j\mathrm{odd}}{\theta }_j$, the odd-subsample limit isThis is the odd-subsample long-run variance of a moving average. Equivalently it is $[f_X(0)+f_X(1/2)]/2$ in the cycles convention. Under the printed invertibility assumption and positive noise variance, the polynomial is nonzero at both $1$ and $-1$, so both limits are positive. The same formulas also hold without invertibility, when cancellations can make a limit zero.

First, the absolute values in the printed limit are an error: the correct long-run variance of a stationary process is the signed sum of autocovariances. Directly,soEach coefficient tends to $1$ and has absolute value at most $1$. Absolute summability and the dominated convergence theorem therefore giveThe sum of absolute values is an upper bound, not the general limit. For a concrete counterexample to the printed assertion, take $X_t={\eta }_t-{\eta }_{t-1}$ with unit-variance iid noise. Then $\gamma (0)=2$, $\gamma (\pm 1)=-1$, and other covariances vanish. The absolute sum is $4$, but ${\overline{X}}_T=({\eta }_T-{\eta }_0)/T$, so $T\mathrm{Var}({\overline{X}}_T)=2/T\to 0$.

For correlated data the central limit theorem can still hold under suitable strong mixing of a stationary process and moment conditions, but the limiting variance is the long-run variance of a stationary process, rather than the one-observation variance. When it is positive,Positive serial dependence usually increases the standard error, while negative dependence can reduce it. The approximate effective sample size of a stationary sample is $T{\gamma }_X(0)/{\sum }_h{\gamma }_X(h)$ when that denominator is positive. A long-memory time series can require a different normalization or a different limit law. Absolute covariance summability alone does not prove a CLT: if $X_t=V{\eta }_t$ with a common independent random scale $V$ taking values $1$ and $2$ with equal probabilities and iid ${\eta }_t$ with the standard normal distribution, the off-diagonal covariances are zero, yet $\sqrt{T}{\overline{X}}_T$ has the nonnormal scale-mixture law $VN(0,1)$. The example is not an ergodic stationary process. If the long-run variance of a stationary process is zero, the usual nondegenerate square-root-$T$ CLT is unavailable.

For the causal AR(1), $|\varphi |<1$ andThe conditional distribution follows because the current innovation is independent of the past.

For a fixed known $\varphi$, condition on the observed $X_1$ and use the $n=T-1$ transitions $t=2,\dots ,T$. Their conditional maximum likelihood criterion is, up to constants,Differentiating in $\mu$ yieldsSince $X_t-\varphi X_{t-1}=(1-\varphi )\mu +{\epsilon }_t$,Thus it is an unbiased estimator, even conditionally on $X_1$, andIt has statistical consistency with mean-square convergence, and the iid-noise strong law of large numbers also gives almost-sure statistical consistency. Its conditional distribution is exactly normal with the displayed mean and variance. If $X_0$ is also observed and all $T$ transitions are used, replace $T-1$ by $T$.

The fixed-$\varphi$ qualification is necessary for the exact finite-sample claims. If $\varphi$ is jointly estimated, conditional likelihood function is linear regression with an intercept: writing ${\bar{X}}_{-}=n^{-1}{\sum }_{t=2}^T{X}_{t-1}$ and ${\bar{X}}_{+}=n^{-1}{\sum }_{t=2}^T{X}_t$, the unconstrained estimators areThis ratio is not generally an unbiased estimator and does not have the preceding finite-sample variance. Under the usual stationary regression conditions it has statistical consistency; its asymptotic variance is ${\sigma }_{\epsilon }^2/(1-\varphi {)}^2$. Indeed, it differs from ${\bar{X}}_{+}$ by $\hat{\varphi }(X_T-X_1)/(n(1-\hat{\varphi }))$, an asymptotically negligible endpoint term. Profiling an unknown innovation variance does not change the fixed-$\varphi$ estimate of $\mu$.

For a weakly stationary process, every observation has the same finite expectation $\mu$. Linearity of expectation, without any assumption of independence, givesThe variance is a sum of all pairwise covariances:For a real-valued weakly stationary process, the autocovariance is even. There are $n$ pairs at lag zero and $n-h$ pairs in each direction at positive lag $h$. ThereforePositive autocovariances inflate the uncertainty relative to the independent-observation formula; negative terms may reduce it. An unbiased estimator need not have variance tending to zero: a process consisting of the same random variable at every time is weakly stationary, but averaging cannot reduce its variance.

A zero-mean autoregressive process of order one satisfieswhere ${\epsilon }_t$ is white noise with variance ${\sigma }_{\epsilon }^2>0$. A causal time series uses present and past innovations only. For $|\alpha |<1$ the serieshas mean-square convergence because ${\sum }_j{\alpha }^{2j}<\infty$, and it supplies the causal weakly stationary process. Its variance is ${\sigma }_{\epsilon }^2/(1-{\alpha }^2)$. For $h\ge 0$, only the common innovations contribute to the autocovariance:Evenness then givesWhen $\alpha <0$, the signs alternate; when $\alpha =0$, the process is white noise. With nondegenerate innovations, $|\alpha |\ge 1$ does not give a causal finite-variance stationary solution. These covariance calculations need uncorrelated innovations, not Gaussianity; Gaussian innovations additionally yield a Gaussian process.

The selected model is a moving-average process of order one, with the plus-sign convention used by R:The Gaussian likelihood function models innovations as independent $N(0,{\sigma }_{\epsilon }^2)$. Its autocovariance is $\gamma (0)={\sigma }_{\epsilon }^2(1+{\theta }^2)$, $\gamma (1)={\sigma }_{\epsilon }^2\theta$, and zero at larger lags, with autocorrelation $\rho (1)=\theta /(1+{\theta }^2)\simeq 0.4631$. The estimated coefficient has modulus less than one, giving invertibility of a moving-average model; a finite moving-average process is stationary regardless of this invertibility restriction.

The original plots show fluctuations around a roughly constant level, no persuasive deterministic trend, and a conspicuous positive lag-one correlation with the other plotted correlations within the approximate sampling bands. They make a stationary nonseasonal working model reasonable for this short series. stationary=TRUE excludes differenced alternatives, and seasonal=FALSE excludes seasonal ARIMA terms. Neither option is proved by a 60-day plot: annual temperature seasonality cannot be excluded from two months, and a longer record or residual diagnostics might require a different model. The options are reasonable local simplifications, not universal statements about temperature dynamics.

Use the stationary autoregressive process of order oneMultiplication by $X_{t-1}-\mu$ and expectation, using independence of the innovation from the past, gives the autocovariance equation $\gamma (1)=\varphi \gamma (0)$. With $\bar{X}=n^{-1}{\sum }_t{X}_t$, define the sample autocovariances using the same denominator:Substitution into that equation gives the Yule–Walker estimator for an autoregressive process of order oneThis is exactly the lag-one sample autocorrelation function under the divisor-$n$ convention, provided the sample has positive variance. Using different divisors for different lags would change the estimator.

For unit-variance innovations, orthogonality to the past givesConsequently ${\varphi }_{\nu }={\gamma }_{\nu }(1)/V_{\nu -1}$ and ${\sigma }_{\nu }^2=V_{\nu }-{\gamma }_{\nu }(1{)}^2/V_{\nu -1}$. Replacing autocovariance by matched sample estimates yields seasonal regression estimators. The unit innovation variance is necessary to identify the noise scale separately.

Replace the autocovariance in the finite-lag linear prediction equations by the sample autocovariance function. The last regression coefficient at order $k$ estimates the lag-$k$ partial autocorrelation function.

In a second-order sense a weakly stationary process has short memory when its autocovariance is absolutely summable. Then its spectral density of a stationary process is continuous and its sample-mean variance has the usual finite long-run variance of a stationary process. A central limit theorem still needs additional conditions.