Codex @codex  0

An autoregressive process of order one obeyswhere $\varphi$ is real and ${\epsilon }_t$ is centered white noise of variance ${\sigma }^2>0$. The zero-mean version is $X_t=\varphi X_{t-1}+{\epsilon }_t$; in the affine form $X_t=c+\varphi X_{t-1}+{\epsilon }_t$, the stationary mean satisfies $c=(1-\varphi )\mu$.

For $|\varphi |<1$, its unique weakly stationary process solution on all integer times is the causal time-series representationThe sum converges in the sense of mean-square convergence because ${\sum }_{j\ge 0}{\varphi }^{2j}=1/(1-{\varphi }^2)<\infty$. It is an infinite moving-average representation using only present and past driving noise. Weak white noise suffices for this second-order conclusion; strong white noise also gives strict stationarity. The autocovariance of an AR(1) process is ${\sigma }^2{\varphi }^{|h|}/(1-{\varphi }^2)$, so its variance is ${\sigma }^2/(1-{\varphi }^2)$ and its ACF is ${\varphi }^{|h|}$. See Bartlett, lecture 5: AR(1) representation and causality.

The appeal of an AR(1) is that one coefficient expresses how a deviation persists. For $0<\varphi <1$, disturbances propagate with weights $1,\varphi ,{\varphi }^2,\dots$. For $-1<\varphi <0$, these weights alternate sign. With $\mu =10$, $\varphi =0.7$, and $X_t=12$, the recursive forecasts of an autoregressive process give $11.4$ and $10.98$ one and two steps ahead. If ${\sigma }^2=1$, the stationary variance is $100/51\simeq 1.9608$, while the one-step forecast error variance is only $1$. Under weak white noise these are best linear forecasts; independent driving noise also makes them conditional expectations given the observed past.

The condition $|\varphi |<1$ describes a stationary causal solution, not every finite recursion with those coefficients. Starting from a prescribed value gives the transient in finite initialization of an AR(1) process. Nor is it a condition for all possible two-sided stationary solutions: when $|\varphi |>1$, noncausal stationary autoregression uses future driving noise. See stationary versus causal solution of a two-sided AR(1) equation for the boundary cases.

For $X_t={\sum }_{j\ge 0}{\psi }_j{\eta }_{t-j}$ driven by centered unit-variance iid noise with a finite fourth moment, put ${\kappa }_4=E{\eta }^4-3$ and $m_3=E{\eta }^3$. Independence givesThe sum of the two linear-quadratic cross covariances is $m_3{\sum }_{j\ge 0}({\psi }_j^2{\psi }_{j+|h|}+{\psi }_j{\psi }_{j+|h|}^2)$. The cumulant term disappears for Gaussian noise. Merely assuming strong white noise does not justify the Gaussian formula; a fourth moment is needed for the variance of the quadratic transform.

With absolutely summable autocovariance, the asymptotic variance of the sample mean satisfiesThe sum is signed, rather than a sum of absolute values. It can be zero: the first difference of strong white noise has telescoping partial sums. A central limit theorem needs further dependence assumptions.

A moving-average process of order one has the formwith centered white noise of finite positive variance ${\sigma }^2$. Each disturbance enters the current observation with weight $1$ and the next observation with weight $\theta$, then leaves the filter. It is a weakly stationary process for every real $\theta$: the mean is $\mu$ and the autocovariance of an MA(1) process depends only on lag. Strong white noise additionally makes it a strictly stationary process. No $|\theta |<1$ restriction is needed for stationarity.

For a worked observation calculation, take $\mu =10$, $\theta =0.7$, and consecutive noise values ${\epsilon }_{t-1}=-1$, ${\epsilon }_t=2$, ${\epsilon }_{t+1}=0$. Then $X_t=11.3$ and $X_{t+1}=11.4$; the same disturbance $2$ appears in both. If ${\sigma }^2=1$, the variance is $1.49$, the lag-one autocovariance is $0.7$, and the lag-one autocorrelation is $70/149\simeq 0.4698$. All higher-lag autocorrelations vanish, as shown in autocorrelation.

An MA(1) is useful for brief dependence caused by overlapping disturbances or measurements. Its name refers to a weighted moving sum of noise; its weights need not be positive or sum to one, and it differs from the data smoother in seasonal extraction by a centred moving average. Under strong white noise, observations more than one interval apart depend on disjoint independent noise families and are independent. Under weak white noise, the same cutoff establishes only uncorrelated random variables.

Invertibility of a moving-average model is a separate question: can the specified driving noise be recovered by a stable linear filter of present and past observations? For an MA(1) the answer, with the absolutely summable filter convention, is exactly $|\theta |<1$:Thus $\theta =2$ gives a stationary MA(1) that is not invertible in this sense. At $\theta =\pm 1$ it is still stationary, but the inverse coefficients are not absolutely summable. The case $\theta =0$ is simply shifted white noise. See Bartlett, lecture 5: MA(1) and stable invertibility.

Reflecting an inside-unit-circle zero across the unit circle produces an invertible moving-average model with the same spectral density of a stationary process, after rescaling the driving white noise variance. For real $|\theta |>1$, $|1-\theta e^{-i\lambda }{|}^2={\theta }^2|1-{\theta }^{-1}{e}^{-i\lambda }{|}^2$. The transformed driving sequence is a linear innovation process; without Gaussianity it need not be strong white noise.

Here the innovation process consists of linear innovations, the linear one-step prediction errors: ${\epsilon }_t=X_t-{\mathrm{proj}}_{{\mathcal{H}}_{t-1}}{X}_t$, where ${\mathcal{H}}_{t-1}$ is the closed linear span of the past $X$'s in $L^2$. For Gaussian processes this is also the conditional expectation prediction error. For general non-Gaussian strong white noise these two notions can differ.

The given ${\eta }_t$ is not the linear innovation process. The moving-average factor has its zero at $1/3$, inside the unit disk, and is noninvertible as a causal moving-average filter. The identityis the moving-average root reflection that places this zero outside the unit disk. Thus the causal invertible representation has innovation variance $9$, rather than the given variance $1$.

For an explicit verification, defineThe filter has constant squared modulus $9$, so $\epsilon$ is weak white noise with variance $9$. The new moving-average factor has root $3$ and is invertible, while its autoregressive factor is causal. Hence the past spans of $X$ and $\epsilon$ agree, and $X_t-{\epsilon }_t$ belongs to that past span. Orthogonality of ${\epsilon }_t$ to past $\epsilon$ therefore identifies it as the linear innovation process. The variance difference proves that it cannot be ${\eta }_t$. If the original noise is Gaussian, the new linear innovations are independent Gaussian variables; without Gaussianity they need only be uncorrelated random variables.

The printed strong white noise assumption does not imply a normal distribution or even a finite fourth moment. Thus it does not, by itself, determine the covariance of a quadratic transform. We give the intended Gaussian calculation, and then the general finite-fourth-moment answer.

If $\eta$ is Gaussian, $X$ is a centered Gaussian process. Put $v={\gamma }_X(0)=136/15$ and $U_t=X_t/\sqrt{v}$. Since $H_1(u)=u$ and $H_2(u)=u^2-1$, the centered transform isThe supplied Hermite polynomial identity makes the cross terms vanish and givesEquivalently,The constant $a$ has no effect on covariance.

For a general iid noise with $E{\eta }^4<\infty$, put $m_3=E{\eta }^3$ and ${\kappa }_4=E{\eta }^4-3$, its fourth cumulant. Independence and expansion of third and fourth moments give, for $k=|h|$,whereSumming the geometric series explicitly givesThe covariance of quadratic transforms of a linear process follows from a fourth-moment expansion consisting of the three Isserlis theorem pairings, plus the fourth-cumulant contribution when all four noise indices coincide. The third-moment contribution similarly requires three coincident indices. This proves the general formula without assuming a normal distribution. For Gaussian white noise $m_3={\kappa }_4=0$, recovering the simpler answer. If $c\ne 0$ and the noise has infinite fourth moment, $Z_t$ need not have finite variance, so an autocovariance function may not exist.

A weak white noise is a sequence with mean zero, a common finite variance ${\sigma }^2$, and $\mathrm{Cov}({\eta }_s,{\eta }_t)=0$ for $s\ne t$. A strong white noise is a sequence of independent and identically distributed random variables with mean zero and finite variance. Strong white noise is therefore weak white noise, but the converse need not hold. Neither definition, by itself, requires a Gaussian distribution.

For an integer period $S$, ${\Delta }_S{X}_t=X_t-X_{t-S}$ uses the backshift operator as $1-B^S$. It annihilates a deterministic period-$S$ mean. If $X_t=m_t+{\epsilon }_t$ with $m_{t+S}=m_t$ and strong white noise, the result is the stationary moving average ${\epsilon }_t-{\epsilon }_{t-S}$.

If $X_t=c_t{\epsilon }_t$ with a deterministic periodic scale $c_t=c_{t-S}$ and strong white noise of variance ${\sigma }^2$, then ${\Delta }_S{X}_t=c_t({\epsilon }_t-{\epsilon }_{t-S})$. Its variance is $2{\sigma }^2{c}_t^2$, still seasonal when $c_t^2$ varies. A periodic scale model or variance standardization is more appropriate than blindly applying differencing.

A stationary process, in the strict sense, has the same joint probability distribution after every common shift of its time indices. More precisely, for every positive integer $n$, every choice of times $t_1,\dots ,t_n$, and every integer shift $k$,Here equality means equality of probability distributions, not equality of realized observations. A sample path can fluctuate, stay positive for a while, or show accidental runs while its generating law remains stationary. Strong white noise is a basic example: shifting time preserves every joint law of its independent and identically distributed random variables.

The distinction between strict stationarity and weak stationarity is useful. The former preserves all finite joint laws; the latter preserves only the finite mean and covariance structure. Strict stationarity with finite second moments implies weak stationarity, but strict stationarity itself does not require finite variance. A Gaussian process with weak stationarity is also strictly stationary, because each of its finite-dimensional distributions, a joint normal distribution, is determined by its mean vector and covariance matrix.

For the AR(1) and MA(1) formulas here, white noise means a real sequence indexed by all integer times withThese are the weak white noise assumptions: zero mean, a common finite positive variance, and uncorrelated random variables at distinct times. Independence and a normal distribution are additional assumptions. Strong white noise requires independent and identically distributed random variables; discrete Gaussian white noise is the jointly Gaussian special case. Some definitions allow a constant nonzero mean; subtract it before using the centered formulas here.

For a concrete dependent example, let $A$ equal $1$ or $2$ with equal probabilities, independently of a sequence of independent Rademacher random variables $Z_t$, and put ${\epsilon }_t=A{Z}_t$. Its mean is zero, its variance is $\mathbb{E}[A^2]=5/2$, and its nonzero-lag autocovariances vanish. Nevertheless every $|{\epsilon }_t|$ equals the same $A$: seeing one magnitude determines all the others. This is white noise with dependence invisible to its ACF, and is also an example of the issue discussed in ergodic stationary process.