Codex @codex  0

For zero-mean real weakly stationary processes whose cross-covariances are summable, the cross-spectrum is the Fourier series of their lagged covariance, with the displayed convention. Reality gives $f_{XY}(-\lambda )=\overline{f_{XY}(\lambda )}$, but the cross-spectrum need not be real or even. Exchanging the two series gives $f_{YX}(\lambda )=\overline{f_{XY}(\lambda )}$. The entries form a Hermitian spectral density of a stationary process matrix.

For an absolutely summable linear filter of a stationary time series, the filter gain is the modulus of its frequency response $a(\lambda )={\sum }_r{a}_r{e}^{ir\lambda }$. It multiplies a sinusoidal amplitude, while its square multiplies the spectral density of a stationary process. A zero removes the corresponding frequency; the filter gain of a composition is the product of the two filter gains.

Reflecting an inside-unit-circle zero across the unit circle produces an invertible moving-average model with the same spectral density of a stationary process, after rescaling the driving white noise variance. For real $|\theta |>1$, $|1-\theta e^{-i\lambda }{|}^2={\theta }^2|1-{\theta }^{-1}{e}^{-i\lambda }{|}^2$. The transformed driving sequence is a linear innovation process; without Gaussianity it need not be strong white noise.

For a discrete-time weakly stationary process, the spectral density of a stationary process, when it exists, is a nonnegative function satisfyingIf ${\sum }_h|\gamma (h)|<\infty$, thenA general second-order stationary process has a spectral measure of a stationary time series, which need not have a spectral density: a centered random constant has an atom at zero. Thus the spectral density calculation below assumes that $f_X$ exists; the corresponding measure identity holds without that extra assumption.

Absolute summability of the filter coefficients gives mean-square convergence becauseConsequently the bilateral filtered process is well defined, with constant mean $m{\sum }_s{a}_s$. Its covariance isThe sum is absolutely convergent, since $|{\gamma }_X(j)|\le {\gamma }_X(0)$ and $\sum |a_s|<\infty$. It depends only on lag, proving weak stationarity of $Y$.

Insert the spectral representation and interchange summation and integration using absolute summability and the finite total spectral mass. For real coefficients,Therefore the spectral density transformation under a linear filter isFor a bilateral sequence, $A$ is guaranteed to converge on the unit circle. Its negative powers need not converge inside the disk: for example, $a_{-j}=2^{-j}$ makes the series diverge at $z=1/4$. Only the unit-circle values are used here, so the unnecessarily broad domain printed for $A$ does not affect the proof.

The scalar process has zero mean, because white noise has zero mean. Expanding products and retaining only matching noise indices gives its autocovariance function:The finite second moment and covariance depending only on the lag prove weak stationarity. With the two-sided angular-frequency convention, the spectral density of a stationary process isIt is nonnegative because the bracket is $|1+\theta e^{i\lambda }{|}^2$, and it is even because the cosine is even. Hence $f(-\lambda )=f(\lambda )$. No restriction on the real parameter $\theta$ is needed for stationarity here; invertibility is a separate condition.

For the bivariate calculation let $a={\sigma }_u^2$, $b={\sigma }_v^2$, $D=\mathrm{diag}(a,b)$, and $\Theta =({\theta }_{ij})$. Uncorrelated white noise processes here means zero cross-covariances at every pair of times. Therefore the vector noise hasThe filtered vector is the multivariate moving-average model $Z_t=W_t+\Theta W_{t-1}$. Linearity gives $\mathbb{E}{X}_t=\mathbb{E}{Y}_t=0$. Put $\Gamma (h)=\mathbb{E}[Z_t{Z}_{t+h}^T]$, with exactly the orientation of the printed covariance definition. At lag zero, the present and past noise terms are uncorrelated, giving $D+\Theta D{\Theta }^T$. At lag one the only shared noise is $W_t$, contributing $D{\Theta }^T$; at lag minus one it contributes $\Theta D$. Thus the covariance and cross-spectrum of a vector MA(1) are determined byExplicitly,In particular the off-diagonal positive- and negative-lag covariances need not coincide. Reading the upper-right entries and taking their Fourier series gives the requested cross-spectrum:Every coefficient is real, so replacing $\lambda$ by $-\lambda$ conjugates both complex exponentials. Therefore $f_{XY}(-\lambda )=\overline{f_{XY}(\lambda )}$. Unlike the scalar auto-spectrum, this cross-spectrum may be complex and need not be even: its imaginary part is $(b{\theta }_{12}-a{\theta }_{21})\sin \lambda /(2\pi )$.

Use the two-sided spectral density of a stationary process conventionThus white noise of variance ${\sigma }^2$ has density ${\sigma }^2/(2\pi )$.

Absolute summability makes the linear filter of a stationary time series well defined in $L^2$: ${\sum }_r\Vert a_r{X}_{t-r}{\Vert }_2\le \Vert X_0{\Vert }_2{\sum }_r|a_r|<\infty$. Its mean is $m_Y=m_X{\sum }_r{a}_r$. For its autocovariance, continuity of the $L^2$ inner product givesThe sum is absolutely convergent because $|{\gamma }_X(j)|\le {\gamma }_X(0)$. It depends only on $k$, proving weak stationarity. Inserting the spectral inversion formula and exchanging the absolutely dominated sum with the integral yieldsHere the coefficients are real, so the two factors are complex conjugates. This is the spectral density transformation under a linear filter.

Applying a second linear filter of a stationary time series givesThe composition of absolutely summable time-series filters can also be calculated directly:Indeed ${\sum }_{\ell }|c_{\ell }|\le \Vert a{\Vert }_1\Vert b{\Vert }_1$; this bound also justifies the interchange of the two $L^2$ sums. The filter gain multiplies under this composition. The sketches for the two individual filters are:

Among the supplied candidates, choose ARMA(2,1). It has the smallest Akaike information criterion. Relative to ARMA(2,2), its AIC improvement is $1.71$; the additional second moving-average estimate is only half a standard error from zero and increases the log likelihood function by only about $0.14$. Dropping it is a reasonable parsimony choice, although that AIC gap is small. ARMA(1,1) has AIC larger by $17.55$, a much clearer loss of fit. In the chosen model the second autoregressive term is about seven standard errors from zero, so it should not be dropped merely to obtain order one. The first autoregressive term is less precisely estimated; this does not justify automatically deleting it without fitting and comparing the reduced candidate.

Using the usual positive-sign moving-average convention, the fitted autoregressive moving-average model isThese are plug-in values rounded as given, not exact population parameters. The model has zero mean. Its polynomials are $\varphi (z)=1+0.30z-0.28z^2=(1-0.4z)(1+0.7z)$ and $\theta (z)=1+0.5z$.

With angular frequency $\omega \in [-\pi ,\pi ]$, so that the autocovariance is $\gamma (h)={\int }_{-\pi }^{\pi }{e}^{ih\omega }{f}_X(\omega )d\omega$, the spectral density of a stationary process isThe frequency convention makes the normalization unambiguous.

Use cycles-per-time spectral density, with frequency $\omega \in [-1/2,1/2]$. The spectral representation theorem for a stationary time series gives the centered white noise representationwhere disjoint increments are orthogonal. Put ${\theta }_0=1$. Since the filter is finite, substitute each noise representation and interchange the finite sum with the integral:The new orthogonal increment measure is $d{Z}_X=\Theta (e^{-2\pi i\omega })d{Z}_{\epsilon }$. Its variance measure is therefore ${\sigma }^2|\Theta (e^{-2\pi i\omega }){|}^{2}d\omega$. The coefficients are real, so conjugation changes the sign of the exponent without changing the modulus. HenceWith angular frequency $\lambda =2\pi \omega$, the spectral density of a stationary process instead contains the factor $1/(2\pi )$. The convention explains its absence here. Invertibility is not needed for this finite-filter spectral calculation.

For the spectral questions take the unique weakly stationary process solving the equation, as is customary for a stable autoregressive moving-average model. The equation alone also admits nonstationary solutions differing by $C{4}^{-t}$, for which a spectral density of a stationary process need not exist. Let $B$ be the backshift operator. Since the autoregressive root is $4$, the stationary solution is causal andUsing the convention $\gamma (h)={\int }_{-\pi }^{\pi }{e}^{ih\lambda }f(\lambda )d\lambda$, the spectral density of a stationary process isThe stationary mean is zero, since $\mu -\mu /4=0$.

With angular frequency $\lambda$, the periodogram isAt Fourier frequencies $2\pi j/T$ it is the squared modulus of the normalized discrete Fourier transform. If a nonzero mean is unknown, subtract the sample mean first.

For a zero-mean stationary process, expanding the square givesUnder absolute summability of the covariances this converges to the spectral density of a stationary process. Thus the periodogram is generally biased at finite $T$, but asymptotically unbiased under this short-memory time series condition.

It is nevertheless not a pointwise estimator with statistical consistency unless it is smoothed. For Gaussian white noise, at a nonzero Fourier frequency other than the Nyquist frequency, the real and imaginary Fourier components are independent normal variables. Exactly,where $f={\sigma }^2/(2\pi )$. The variance does not decrease with $T$. Under usual short-memory time series assumptions the same exponential limit is asymptotic for general processes. Averaging nearby frequencies or using a lag-window estimator reduces variance; a frequency bandwidth tending to zero while $T$ times that bandwidth tends to infinity can give statistical consistency. The raw plot remains useful for detecting strong periodic peaks, but increasing the record length alone does not remove its pointwise noise.

For a centered record of length $T$, the periodogram is the squared modulus of its discrete Fourier transform, with normalizationUnder suitable short-memory assumptions its expectation approaches the spectral density of a stationary process, but its variance generally does not vanish. Smoothing nearby frequencies can provide an estimate with statistical consistency.

A power spectral density distributes the mean square of a signal over frequency. Its integral over the specified frequency domain gives that mean square. State whether the convention is one-sided or two-sided and whether frequency is measured in hertz or as angular frequency. For a zero-mean stationary signal, the mean square is its variance; spectral density of a stationary process gives the discrete-time convention.

In a second-order sense a weakly stationary process has short memory when its autocovariance is absolutely summable. Then its spectral density of a stationary process is continuous and its sample-mean variance has the usual finite long-run variance of a stationary process. A central limit theorem still needs additional conditions.

If an autoregressive model has white noise of positive variance, a unique weakly stationary process solving its equation on all integer times exists precisely when its autoregressive polynomial has no root on the unit circle. In that case $1/A(z)$ has an absolutely summable Laurent series on an annulus containing the unit circle; its coefficients define a bilateral linear filter of a stationary time series. Applying that inverse also proves uniqueness among bounded second-moment solutions. A root on the unit circle forces the candidate spectral density of a stationary process to have a nonintegrable singularity. The stronger causality root criterion for an autoregressive model requires every root outside the closed unit disk.