Autoregressive process of order one ($\mathrm{AR}(1)$)

An autoregressive process of order one obeyswhere $\varphi$ is real and ${\epsilon }_t$ is centered white noise of variance ${\sigma }^2>0$. The zero-mean version is $X_t=\varphi X_{t-1}+{\epsilon }_t$; in the affine form $X_t=c+\varphi X_{t-1}+{\epsilon }_t$, the stationary mean satisfies $c=(1-\varphi )\mu$.

For $|\varphi |<1$, its unique weakly stationary process solution on all integer times is the causal time-series representationThe sum converges in the sense of mean-square convergence because ${\sum }_{j\ge 0}{\varphi }^{2j}=1/(1-{\varphi }^2)<\infty$. It is an infinite moving-average representation using only present and past driving noise. Weak white noise suffices for this second-order conclusion; strong white noise also gives strict stationarity. The autocovariance of an AR(1) process is ${\sigma }^2{\varphi }^{|h|}/(1-{\varphi }^2)$, so its variance is ${\sigma }^2/(1-{\varphi }^2)$ and its ACF is ${\varphi }^{|h|}$. See Bartlett, lecture 5: AR(1) representation and causality.

The appeal of an AR(1) is that one coefficient expresses how a deviation persists. For $0<\varphi <1$, disturbances propagate with weights $1,\varphi ,{\varphi }^2,\dots$. For $-1<\varphi <0$, these weights alternate sign. With $\mu =10$, $\varphi =0.7$, and $X_t=12$, the recursive forecasts of an autoregressive process give $11.4$ and $10.98$ one and two steps ahead. If ${\sigma }^2=1$, the stationary variance is $100/51\simeq 1.9608$, while the one-step forecast error variance is only $1$. Under weak white noise these are best linear forecasts; independent driving noise also makes them conditional expectations given the observed past.

The condition $|\varphi |<1$ describes a stationary causal solution, not every finite recursion with those coefficients. Starting from a prescribed value gives the transient in finite initialization of an AR(1) process. Nor is it a condition for all possible two-sided stationary solutions: when $|\varphi |>1$, noncausal stationary autoregression uses future driving noise. See stationary versus causal solution of a two-sided AR(1) equation for the boundary cases.

For the stationary causal AR(1) with $|\varphi |<1$, matching terms in its infinite moving-average representation gives, for $h\ge 0$,Symmetry of the autocovariance extends this to negative lags. Dividing by $\gamma (0)$ gives the autocorrelation $\rho (h)={\varphi }^{|h|}$. The geometric series explains both the enlarged stationary variance and the diminishing memory: all past disturbances contribute to the present, but their squared weights have a finite sum. At $\varphi =0$, the process reduces to shifted white noise and all nonzero-lag autocorrelations vanish. See Bartlett, lecture 4: AR(1) representation and mean-square convergence.

For $t\ge 1$, iterating an AR(1) from a fixed value $X_0=x_0$ gives the displayed finite sum. With centered white noise of variance ${\sigma }^2$ and $|\varphi |<1$,For $h\ge 0$, its covariance is $\mathrm{Cov}(X_{t+h},X_t)={\varphi }^h\mathrm{Var}(X_t)$, which depends on $t$. Even setting $x_0=\mu$ leaves a changing variance. At $\varphi =0.7$, ${\sigma }^2=1$, and $x_0=\mu$, the variances at times zero through three are $0,1,1.49,1.7301$, approaching $100/51$; this finite initialized record is not weakly stationary.

To initialize in weak stationarity, give $X_0$ mean $\mu$, variance ${\sigma }^2/(1-{\varphi }^2)$, and zero covariance with every future driving noise value. The full stationary construction sets $X_0-\mu ={\sum }_{j\ge 0}{\varphi }^j{\epsilon }_{-j}$. For discrete Gaussian white noise, equivalently draw $X_0$ from $N(\mu ,{\sigma }^2/(1-{\varphi }^2))$ independently of future noise. This gives exact strict stationarity from the first plotted point; it is the initialization used in the figure in time series. Matching these first two moments alone does not guarantee strict stationarity for non-Gaussian inputs. See also stationary initialization of a scalar linear state-space model.

For $|\varphi |<1$, the bilateral sum ${\sum }_{k\in \mathbb{Z}}{\varphi }^{|k|}{z}^k$ equals $(1-{\varphi }^2)/((1-\varphi z)(1-\varphi z^{-1}))$ on the unit circle. Multiplying by ${\sigma }^2/(\pi (1-{\varphi }^2))$ gives the one-sided autoregressive spectrum and reads off its covariance coefficients.

For a causal AR(1) plus uncorrelated observation white noise, cross covariances vanish by L2 convergence of the autoregressive noise expansion. Its covariance is ${\sigma }_z^2{\varphi }^{|h|}/(1-{\varphi }^2)+{\sigma }_w^2{1}_{\{h=0\}}$. The additional noise changes only lag zero, attenuating the normalized positive-lag correlations.

Filtering the observations by $1-\varphi B$ gives covariance $A={\sigma }_z^2+(1+{\varphi }^2){\sigma }_w^2$ at zero, $C=-\varphi {\sigma }_w^2$ at lag one and zero elsewhere. Set $\nu =(A+\sqrt{A^2-4C^2})/2$ and $\vartheta =C/\nu$. Then $\nu (1+{\vartheta }^2)=A$, $\nu \vartheta =C$ and $|\vartheta |<1$. Applying $(1+\vartheta B{)}^{-1}$ defines actual white-noise innovations with variance $\nu$, proving an at-most-(1,1) ARMA representation without assuming Gaussianity.

With nondegenerate white noise, a two-sided AR(1) equation has a unique weakly stationary solution exactly when $|\varphi |\ne 1$. The solution is causal for $|\varphi |<1$ and anticausal for $|\varphi |>1$. At $\varphi =\pm 1$, an n-term noise sum has variance $n{\sigma }^2$, while its difference-of-stationary-values representation has variance at most four times the stationary variance, a contradiction.

For a stationary autoregressive process of order one with mean $\mu$, autoregressive coefficient $\varphi$ and innovation variance ${\sigma }^2$, an interior observation conditional on its neighbors hasThe Markov property makes the same law valid when all other observations are also conditioned on.

For $|\varphi |<1$, independent $N(0,{\sigma }^2)$ innovations and stationary initial variance ${\sigma }^2/(1-{\varphi }^2)$, the full likelihood function isConditioning on $X_1$ gives a different likelihood.

Conditioning on the first observation of a centered autoregressive process of order one makes the maximum-likelihood estimator of its coefficient the least-squares ratio above, when the optimum is interior. The stationary initial normal distribution contributes an additional factor $\sqrt{1-{\varphi }^2}$ and quadratic term $(1-{\varphi }^2)X_1^2$. These change the exact maximizer, but for a fixed interior stationary parameter their score is order one while the conditional score has order $T$, giving the same leading estimator.

For $X_t-\mu =\varphi (X_{t-1}-\mu )+{\epsilon }_t$, the autocovariance equation is $\gamma (1)=\varphi \gamma (0)$. Replacing both autocovariances by sample sums divided by the same $n$ yields ${\hat{\varphi }}_{YW}=\hat{\gamma }(1)/\hat{\gamma }(0)$, the lag-one sample autocorrelation function.

For a stationary unit-variance normal distribution autoregression with coefficient $a\in (-1,1)$, its vector of length $T\ge 2$ has precision matrix $J$ that is tridiagonal: the two endpoint diagonal entries are $1/(1-a^2)$, interior entries are $(1+a^2)/(1-a^2)$, and adjacent off-diagonal entries are $-a/(1-a^2)$. An independent observation $r=\lambda Z+\eta$, with covariance $vI$, updates precision and information vector toThe conditional law is multivariate normal distribution $N(Q^{-1}h,Q^{-1})$.