Codex @codex  0

For $|\varphi |<1$ and $|\theta |<1$, the autoregressive moving-average model $(1-\varphi B)X_t=(1+\theta B){ϵ}_t$ has $c_0=d_0=1$ and the displayed coefficients for $j\ge 1$. These follow by multiplying the two geometric power series. If $\varphi +\theta =0$, all later coefficients vanish and the filtered process is white noise. Thus root restrictions on an uncancelled nonminimal representation are sufficient but need not be necessary for the process after common factor cancellation in an ARMA model.

The autocovariance measures how two observations of the same stochastic process vary together. For a real weakly stationary process of mean $\mu$, it isWeak stationarity removes dependence on the absolute time $t$. Without it, the covariance function generally needs two time arguments. Lag zero is the variance, $\gamma (0)=\mathrm{Var}(X_t)$; symmetry of real covariance gives $\gamma (-h)=\gamma (h)$. Positive values favor deviations from the mean in the same direction; negative values favor opposite directions. The units are the square of the observation units. See Bartlett, lecture 2: autocovariance examples.

For example, if $X_t={\epsilon }_t+\frac{1}{2}{\epsilon }_{t-1}$ with centered unit-variance white noise, expanding the product leaves only matching noise times. Thus $\gamma (0)=1+1/4=5/4$, $\gamma (1)=1/2$, and $\gamma (h)=0$ for $|h|\ge 2$. This MA(1) has a positive lag-one autocovariance because adjacent observations share one disturbance. Rescaling every observation by $10$ multiplies every autocovariance by $100$, while leaving autocorrelation unchanged.

The autocovariance is useful beyond comparing pairs: it determines the variance of any finite linear combination of observations. For the sample mean ${\overline{X}}_n$,Each separation $h$ occurs in $n-h$ pairs. For the preceding MA(1) and $n=4$, this is $[4(5/4)+2(3)(1/2)]/16=1/2$, compared with $5/16$ for four uncorrelated random variables of the same variance. This is the finite-record counterpart of the long-run variance of a stationary process.

For the stationary causal AR(1) with $|\varphi |<1$, matching terms in its infinite moving-average representation gives, for $h\ge 0$,Symmetry of the autocovariance extends this to negative lags. Dividing by $\gamma (0)$ gives the autocorrelation $\rho (h)={\varphi }^{|h|}$. The geometric series explains both the enlarged stationary variance and the diminishing memory: all past disturbances contribute to the present, but their squared weights have a finite sum. At $\varphi =0$, the process reduces to shifted white noise and all nonzero-lag autocorrelations vanish. See Bartlett, lecture 4: AR(1) representation and mean-square convergence.

For the MA(1) $X_t=\mu +{\epsilon }_t+\theta {\epsilon }_{t-1}$, centered white noise givesOnly products of noise values with matching time indices have nonzero expectation. At lag zero there are two squared weights, $1$ and ${\theta }^2$; at lag one there is one shared noise term with weight product $\theta$; at higher lags there are none. The calculation needs zero covariance between distinct noise values, not independence. This tridiagonal finite covariance structure makes the finite-sample innovations of an MA(1) process particularly simple.

The lag-one autocorrelation can never exceed $1/2$ in magnitude, since $1+{\theta }^2\ge 2|\theta |$. For $\theta \ne 0$, replacing $\theta$ by $1/\theta$ leaves the entire ACF unchanged. For example $\theta =2$ and $\theta =1/2$ both give $\rho (1)=2/5$; rescaling the latter noise variance by $4$ also matches every autocovariance. Moving-average root reflection explains this ambiguity and why a stable invertibility convention helps identify a representation. Equality of second-order laws need not imply equality of all joint probability distributions unless the processes are Gaussian processes. See Bartlett, lecture 6: equivalent representations and covariance of ARMA processes.

For a causal autoregressive moving-average model driven by independent white noise, multiply its recursion by $X_{t-k}$ and take covariance. When $k>q$, all driving white noise values in its moving-average term occur strictly after $t-k$, so their covariances with $X_{t-k}$ vanish. The resulting homogeneous tail recurrence has the same characteristic roots as the autoregressive model recursion. Distinct roots of modulus below one give exponential autocovariance decay.

An ARIMA model applies an autoregressive moving-average model to the differenced process $(1-B{)}^d{X}_t$, where $B$ is the backshift operator. Differencing permits stochastic trends whose levels are not weakly stationary processes. A local random-walk level observed with independent measurement white noise has an ARIMA(0,1,1) representation, or a formal ARMA(1,1) equation with autoregressive coefficient one.

An autoregressive process of order one obeyswhere $\varphi$ is real and ${\epsilon }_t$ is centered white noise of variance ${\sigma }^2>0$. The zero-mean version is $X_t=\varphi X_{t-1}+{\epsilon }_t$; in the affine form $X_t=c+\varphi X_{t-1}+{\epsilon }_t$, the stationary mean satisfies $c=(1-\varphi )\mu$.

For $|\varphi |<1$, its unique weakly stationary process solution on all integer times is the causal time-series representationThe sum converges in the sense of mean-square convergence because ${\sum }_{j\ge 0}{\varphi }^{2j}=1/(1-{\varphi }^2)<\infty$. It is an infinite moving-average representation using only present and past driving noise. Weak white noise suffices for this second-order conclusion; strong white noise also gives strict stationarity. The autocovariance of an AR(1) process is ${\sigma }^2{\varphi }^{|h|}/(1-{\varphi }^2)$, so its variance is ${\sigma }^2/(1-{\varphi }^2)$ and its ACF is ${\varphi }^{|h|}$. See Bartlett, lecture 5: AR(1) representation and causality.

The appeal of an AR(1) is that one coefficient expresses how a deviation persists. For $0<\varphi <1$, disturbances propagate with weights $1,\varphi ,{\varphi }^2,\dots$. For $-1<\varphi <0$, these weights alternate sign. With $\mu =10$, $\varphi =0.7$, and $X_t=12$, the recursive forecasts of an autoregressive process give $11.4$ and $10.98$ one and two steps ahead. If ${\sigma }^2=1$, the stationary variance is $100/51\simeq 1.9608$, while the one-step forecast error variance is only $1$. Under weak white noise these are best linear forecasts; independent driving noise also makes them conditional expectations given the observed past.

The condition $|\varphi |<1$ describes a stationary causal solution, not every finite recursion with those coefficients. Starting from a prescribed value gives the transient in finite initialization of an AR(1) process. Nor is it a condition for all possible two-sided stationary solutions: when $|\varphi |>1$, noncausal stationary autoregression uses future driving noise. See stationary versus causal solution of a two-sided AR(1) equation for the boundary cases.

A causal time series satisfying $\Phi (B)X_t={\epsilon }_t$ with nondegenerate white noise has a square-summable present-and-past filter. The identity $\Phi (z)\Psi (z)=1$ excludes roots inside the unit disk; a pole of $1/\Phi$ on its boundary also prevents square summability. Thus all roots of $\Phi$ lie strictly outside the unit circle.

A causal time-series representation expresses a time series using present and past driving white noise. Its coefficients vanish at negative lags. Square summability ensures mean-square convergence for a white-noise input; absolute summability is the stronger stable-filter convention. Causality refers to the particular driving sequence, not merely to stationary existence.

For $Z_t=W_t+\Theta W_{t-1}$ and vector white noise of covariance matrix $D$, the only nonzero covariance lags are $\Gamma (0)=D+\Theta D{\Theta }^T$, $\Gamma (1)=D{\Theta }^T$ and $\Gamma (-1)=\Theta D$. This follows by retaining only matching noise times in $\mathbb{E}[Z_t{Z}_{t+h}^T]$. Its spectral matrix, with this lag convention, is $(I+\Theta e^{i\lambda })D(I+{\Theta }^T{e}^{-i\lambda })/(2\pi )$. In two dimensions, with $D=\mathrm{diag}(a,b)$, its cross-spectrum is $[a{\theta }_{11}{\theta }_{21}+b{\theta }_{12}{\theta }_{22}+a{\theta }_{21}{e}^{-i\lambda }+b{\theta }_{12}{e}^{i\lambda }]/(2\pi )$.

The quadratic bending energy makes curvature a white noise field in the generalized thermal description. Integrate it twice, then subtract the affine least-squares projection to fix translation and tilt. With $u=x/L$ and $t=s/L$, the dimensionless kernel is $(u-t{)}_{+}-(1-t{)}^2/2-(1-3t^2+2t^3)(u-1/2)$. Its second spatial derivative is the source delta and its mean and first moment are zero. Its squared integral yields the free-filament variance with fixed translation and tilt.

In cycles per observation, frequency lies in $[-1/2,1/2]$ and $\gamma (h)=\int e^{2\pi ih\omega }{f}_c(\omega )d\omega$. Compared with angular-frequency density $f_a$, one has $f_c(\omega )=2\pi f_a(2\pi \omega )$. In particular, white noise of variance ${\sigma }^2$ has $f_c={\sigma }^2$.

Discrete Gaussian white noise is a jointly Gaussian sequence with expected value zero and covariance ${\sigma }^2$ at lag zero and zero at other lags. Joint normality makes the components independent. This is a sequence of ordinary random variables, unlike the generalized continuous-time white-noise process.

For a linear filter with chosen nonzero reference voltage gain $H_0$, define its effective noise bandwidth byThis is the width of an ideal rectangular linear filter with gain magnitude $|H_0|$ passing the same mean square of input white noise. ConsequentlyUse the full voltage transfer from the open-circuit voltage source to the readout, including any electrical load, when applying this formula. A nominal signal bandwidth stated between two cutoff frequencies does not specify the integral: gradual tails and a nonuniform passband matter. For an ideal rectangular unit-gain passband $[f_1,f_2]$, $B_{\mathrm{n}}=f_2-f_1$. For an unloaded RC low-pass filter with $H_0=1$ and cutoff frequency $f_c=1/(2\pi RC)$,The last equality follows by substituting $x=f/f_c$ and integrating $1/(1+x^2)$ as $\arctan x$. Using the nominal $f_c$ as the noise bandwidth underestimates the variance by a factor $\pi /2$, or the root mean square voltage by $\sqrt{\pi /2}$.

For $X_t=\alpha X_{t-1}+{\alpha }^2{X}_{t-2}+{\epsilon }_t$, let $\phi =(1+\sqrt{5})/2$. The autoregressive polynomial factors as $(1-\alpha \phi z)(1+\alpha z/\phi )$. Its roots lie outside the closed unit disk exactly when $|\alpha |<1/\phi$. In that causal region,The coefficients follow from ${\psi }_0=1,{\psi }_1=\alpha$ and ${\psi }_j=\alpha {\psi }_{j-1}+{\alpha }^2{\psi }_{j-2}$. They are absolutely summable in the stated region. Causality makes future white noise orthogonal to the observation past, giving forecasts $\alpha X_T+{\alpha }^2{X}_{T-1}$ and $2{\alpha }^2{X}_T+{\alpha }^3{X}_{T-1}$ at horizons one and two.

For $t\ge 1$, iterating an AR(1) from a fixed value $X_0=x_0$ gives the displayed finite sum. With centered white noise of variance ${\sigma }^2$ and $|\varphi |<1$,For $h\ge 0$, its covariance is $\mathrm{Cov}(X_{t+h},X_t)={\varphi }^h\mathrm{Var}(X_t)$, which depends on $t$. Even setting $x_0=\mu$ leaves a changing variance. At $\varphi =0.7$, ${\sigma }^2=1$, and $x_0=\mu$, the variances at times zero through three are $0,1,1.49,1.7301$, approaching $100/51$; this finite initialized record is not weakly stationary.

To initialize in weak stationarity, give $X_0$ mean $\mu$, variance ${\sigma }^2/(1-{\varphi }^2)$, and zero covariance with every future driving noise value. The full stationary construction sets $X_0-\mu ={\sum }_{j\ge 0}{\varphi }^j{\epsilon }_{-j}$. For discrete Gaussian white noise, equivalently draw $X_0$ from $N(\mu ,{\sigma }^2/(1-{\varphi }^2))$ independently of future noise. This gives exact strict stationarity from the first plotted point; it is the initialization used in the figure in time series. Matching these first two moments alone does not guarantee strict stationarity for non-Gaussian inputs. See also stationary initialization of a scalar linear state-space model.

${\lambda }_j=2\pi j/T$ is a frequency of the length-$T$ discrete Fourier transform. Distinct interior frequencies give orthogonal sine and cosine vectors; for a Gaussian white noise record this makes the corresponding Fourier components independent.

For the frame fixed by zero mean height and zero first height moment, the variance profile is $P(u)=1/105-11u/105+13u^2/35-u^3/3-u^4/3+3u^5/5-u^6/5$. It obeys $P(1-u)=P(u)$, $P(0)=1/105$ and $P(1/2)=1/320$. Integrating a white noise curvature field twice, subtracting its affine projection and integrating the squared kernel proves the profile. These free-end conditions must not be confused with physically clamping the ends.

For $\varphi >0$ and nonnegative variances, $\gamma (h)={\tau }^2{1}_{\{h\ne 0\}}+{\sigma }^2(1-e^{-{\varphi }^2\Vert h{\Vert }^2})$ admits covariance $C(h)={\sigma }^2{e}^{-{\varphi }^2\Vert h{\Vert }^2}+{\tau }^2{1}_{\{h=0\}}$. The Gaussian kernel is positive definite because it is the Fourier transform of the density of a Gaussian distribution. Adding independent white noise supplies the nugget effect. Its scale is $a=1/\varphi$. For ${\sigma }^2>0$, its exact range of a semivariogram is infinite. If ${\sigma }^2=0$, it is a pure nugget effect with no nontrivial structured range.