Codex @codex  0

For the stationary causal AR(1) with $|\varphi |<1$, matching terms in its infinite moving-average representation gives, for $h\ge 0$,Symmetry of the autocovariance extends this to negative lags. Dividing by $\gamma (0)$ gives the autocorrelation $\rho (h)={\varphi }^{|h|}$. The geometric series explains both the enlarged stationary variance and the diminishing memory: all past disturbances contribute to the present, but their squared weights have a finite sum. At $\varphi =0$, the process reduces to shifted white noise and all nonzero-lag autocorrelations vanish. See Bartlett, lecture 4: AR(1) representation and mean-square convergence.

An autoregressive process of order one obeyswhere $\varphi$ is real and ${\epsilon }_t$ is centered white noise of variance ${\sigma }^2>0$. The zero-mean version is $X_t=\varphi X_{t-1}+{\epsilon }_t$; in the affine form $X_t=c+\varphi X_{t-1}+{\epsilon }_t$, the stationary mean satisfies $c=(1-\varphi )\mu$.

For $|\varphi |<1$, its unique weakly stationary process solution on all integer times is the causal time-series representationThe sum converges in the sense of mean-square convergence because ${\sum }_{j\ge 0}{\varphi }^{2j}=1/(1-{\varphi }^2)<\infty$. It is an infinite moving-average representation using only present and past driving noise. Weak white noise suffices for this second-order conclusion; strong white noise also gives strict stationarity. The autocovariance of an AR(1) process is ${\sigma }^2{\varphi }^{|h|}/(1-{\varphi }^2)$, so its variance is ${\sigma }^2/(1-{\varphi }^2)$ and its ACF is ${\varphi }^{|h|}$. See Bartlett, lecture 5: AR(1) representation and causality.

The appeal of an AR(1) is that one coefficient expresses how a deviation persists. For $0<\varphi <1$, disturbances propagate with weights $1,\varphi ,{\varphi }^2,\dots$. For $-1<\varphi <0$, these weights alternate sign. With $\mu =10$, $\varphi =0.7$, and $X_t=12$, the recursive forecasts of an autoregressive process give $11.4$ and $10.98$ one and two steps ahead. If ${\sigma }^2=1$, the stationary variance is $100/51\simeq 1.9608$, while the one-step forecast error variance is only $1$. Under weak white noise these are best linear forecasts; independent driving noise also makes them conditional expectations given the observed past.

The condition $|\varphi |<1$ describes a stationary causal solution, not every finite recursion with those coefficients. Starting from a prescribed value gives the transient in finite initialization of an AR(1) process. Nor is it a condition for all possible two-sided stationary solutions: when $|\varphi |>1$, noncausal stationary autoregression uses future driving noise. See stationary versus causal solution of a two-sided AR(1) equation for the boundary cases.

The inverse autoregressive polynomial is an analytic function on a disk larger than the unit disk. The Cauchy estimate gives exponentially decaying coefficients of its infinite moving-average representation. Summing their products gives $|\gamma (h)|\le C{s}^{|h|}$ for some $s\in (0,1)$.

A linear process is an L2-convergent white-noise filter $X_t=\mu +{\sum }_{j\in \mathbb{Z}}{\psi }_j{\epsilon }_{t-j}$, with square-summable coefficients. It is weakly stationary, and its autocovariance is ${\sigma }_{\epsilon }^2{\sum }_j{\psi }_j{\psi }_{j+h}$. A causal representation restricts the coefficients to $j\ge 0$; it is then an infinite moving-average representation.

A stationary causal time series exists. The infinite moving-average representationconverges in the sense of mean-square convergence because the driving variables are independent random variables and ${\sum }_j{4}^{-j}<\infty$. Shifting the series gives $X_t=\frac{1}{2}{X}_{t-1}+{\epsilon }_t$. Its expected value is zero and its autocovariance isThus it is a weakly stationary process; since the driving variables have a normal distribution, it is also a Gaussian process and a strictly stationary process. Throughout the stationarity discussion, take ${\sigma }^2>0$; degenerate zero noise permits trivial constant solutions.

For a stationary linear autoregression, causal time series means that the observation uses only current and past driving noise:This is a convergent in the sense of mean-square convergence infinite moving-average representation. The stronger usual stable-filter definition requires absolute summability; the argument below also handles the square-summable definition. Let $\Phi (z)=1-{\sum }_{k=1}^p{\varphi }_k{z}^k$ and $\Psi (z)={\sum }_{j\ge 0}{\psi }_j{z}^j$. Substituting the filter into the recurrence and comparing coefficients of the orthogonal noise gives ${\psi }_0=1$ and ${\psi }_j={\sum }_{k=1}^{\min (p,j)}{\varphi }_k{\psi }_{j-k}$. HenceThe Cauchy-Schwarz inequality ensures that $\Psi$ is an analytic function in this disk. Thus $\Phi$ has no zero strictly inside it. A boundary zero is also impossible: a zero of multiplicity $m\ge 1$ at $z_0=e^{i{\omega }_0}$ makes $|1/\Phi (r{e}^{i\omega }){|}^2$ at least a constant times $[(1-r{)}^2+(\omega -{\omega }_0{)}^2{]}^{-m}$ near that point. Its integral over $\omega$ diverges as $r\uparrow 1$. On the other hand, orthogonality of complex exponentials givesa contradiction. Therefore the causality root criterion for an autoregressive model isUnder absolute summability, the shorter boundary argument is continuity of $\Psi$ on the closed disk and the identity $\Phi \Psi =1$ there.

By the causality root criterion for an autoregressive model, choose $r>1$ smaller than the modulus of every root of $\Phi$. The function $\Psi =1/\Phi$ is an analytic function on and inside $|z|=r$. Writing $M={\max }_{|z|=r}|1/\Phi (z)|$, the Cauchy estimate gives $|{\psi }_j|\le M{r}^{-j}$. For $h\ge 0$, independence of the noise in the infinite moving-average representation givesThe autocovariance is symmetric in the lag. Thus exponential decay holds withThis is exponential autocovariance decay of a causal autoregression. If $\Phi$ is constant, there are no roots and any $r>1$ works.

Expanding the stable autoregressive inverse gives the infinite moving-average representation in linear innovations:ThereforeFor comparison, the causal representation in the originally supplied noise isBoth converge in $L^2$, since their coefficients are square summable. The second is causal, but its driving noise is not the linear innovation process.