Codex @codex  0

An autoregressive process of order one with independent and identically distributed random variables as innovations is an affine process when the innovation exponential moments are finite. Its one-step coefficients are $A(\theta )=a\theta$ and $B(\theta )=b\theta +\psi (\theta )$, where $\psi$ is the innovation cumulant-generating function.

An autoregressive process of order one obeyswhere $\varphi$ is real and ${\epsilon }_t$ is centered white noise of variance ${\sigma }^2>0$. The zero-mean version is $X_t=\varphi X_{t-1}+{\epsilon }_t$; in the affine form $X_t=c+\varphi X_{t-1}+{\epsilon }_t$, the stationary mean satisfies $c=(1-\varphi )\mu$.

For $|\varphi |<1$, its unique weakly stationary process solution on all integer times is the causal time-series representationThe sum converges in the sense of mean-square convergence because ${\sum }_{j\ge 0}{\varphi }^{2j}=1/(1-{\varphi }^2)<\infty$. It is an infinite moving-average representation using only present and past driving noise. Weak white noise suffices for this second-order conclusion; strong white noise also gives strict stationarity. The autocovariance of an AR(1) process is ${\sigma }^2{\varphi }^{|h|}/(1-{\varphi }^2)$, so its variance is ${\sigma }^2/(1-{\varphi }^2)$ and its ACF is ${\varphi }^{|h|}$. See Bartlett, lecture 5: AR(1) representation and causality.

The appeal of an AR(1) is that one coefficient expresses how a deviation persists. For $0<\varphi <1$, disturbances propagate with weights $1,\varphi ,{\varphi }^2,\dots$. For $-1<\varphi <0$, these weights alternate sign. With $\mu =10$, $\varphi =0.7$, and $X_t=12$, the recursive forecasts of an autoregressive process give $11.4$ and $10.98$ one and two steps ahead. If ${\sigma }^2=1$, the stationary variance is $100/51\simeq 1.9608$, while the one-step forecast error variance is only $1$. Under weak white noise these are best linear forecasts; independent driving noise also makes them conditional expectations given the observed past.

The condition $|\varphi |<1$ describes a stationary causal solution, not every finite recursion with those coefficients. Starting from a prescribed value gives the transient in finite initialization of an AR(1) process. Nor is it a condition for all possible two-sided stationary solutions: when $|\varphi |>1$, noncausal stationary autoregression uses future driving noise. See stationary versus causal solution of a two-sided AR(1) equation for the boundary cases.

A causal time series can be expressed using present and past innovations, without future innovations. The autoregressive process of order one with $|\alpha |<1$ has the mean-square convergent causal representation $Y_t={\sum }_{j\ge 0}{\alpha }^j{\epsilon }_{t-j}$.

Conditioning on the first observation of a centered autoregressive process of order one makes the maximum-likelihood estimator of its coefficient the least-squares ratio above, when the optimum is interior. The stationary initial normal distribution contributes an additional factor $\sqrt{1-{\varphi }^2}$ and quadratic term $(1-{\varphi }^2)X_1^2$. These change the exact maximizer, but for a fixed interior stationary parameter their score is order one while the conditional score has order $T$, giving the same leading estimator.

In the expectation-maximization algorithm for a stationary autoregressive process of order one, a single missing interior value has a normal distribution conditional on its neighbors. The E-step uses both its conditional mean and its conditional second moment, not just mean imputation. The M-step maximizes the expected stationary complete-data log-likelihood, including the initial-observation density and the conditional-variance contribution to the two adjacent innovation squares.

For any step $\Delta >0$, set $a=e^{-\theta \Delta }$. The Ornstein-Uhlenbeck transition distribution gives the exact updateusing a fresh independent standard normal random variable for each interval. The update reproduces the joint probability distribution on the selected time grid. It can start from a fixed value or use Ornstein-Uhlenbeck stationary initialization. On an equally spaced grid it is an autoregressive process of order one after subtracting $\mu$. Its stationary variance follows directly from $v=a^{2}v+v_{\infty }(1-a^2)$.

For the stipulated example with $\Delta =0.1$, $a=e^{-0.1}\approx 0.90484$ and the innovation variance is $1-e^{-0.2}\approx 0.18127$. Given $X_t=6$, the next conditional expected value is about $5.61935$ and the conditional standard deviation is about $0.42576$.

The Euler-Maruyama method instead approximates the same stochastic differential equation byFor $0<\theta \Delta <2$ its stationary variance solves $v=(1-\theta \Delta {)}^{2}v+{\sigma }^2\Delta$, givingAt $\Delta =0.1$ in the example, this is $1/0.95\approx 1.05263$, about $5.26\%$ above the continuous model's stationary variance. If $\theta \Delta \ge 2$, this Euler recurrence has no finite stationary variance despite the stable continuous model. Exact transitions avoid this discretization error at sampled times; neither update alone determines the unresolved motion between samples.

For a stationary autoregressive process of order one with mean $\mu$, autoregressive coefficient $\varphi$ and innovation variance ${\sigma }^2$, an interior observation conditional on its neighbors hasThe Markov property makes the same law valid when all other observations are also conditioned on.

For a stationary autoregressive process of order one with unit-variance normal distribution marginals and fixed $0<\rho <1$, the Gaussian kernel covariance correction at lag $j$ has the displayed form. The difference of two observations has variance $2(1-{\rho }^j)$, and their characteristic functions reduce the Fourier integral to the difference of two Gaussian integrals. The mean value theorem gives $0\le g_h(j)\le \sqrt{\pi }{\rho }^j/[2(1-\rho {)}^{3/2}]$, uniformly over $h>0$. Thus the integrated variance differs from its independent-sample counterpart by $O(n^{-1})$. The marginal bias is unchanged, so this dependence does not alter the leading optimal bandwidth or the $n^{-4/5}$ mean integrated squared error for a second-order density kernel.

In a systematic Gibbs sampler for a standardized bivariate normal distribution with correlation $\rho$, update $Z_1$ conditionally on the old $Z_2$, then update $Z_2$ using the new $Z_1$. Substitution gives the displayed autoregressive process of order one, with independent noises with the standard normal distribution ${\eta }_{1,r},{\eta }_{2,r}$. The sweep-to-sweep autocorrelation of $Z_2$ at lag $h$ is ${\rho }^{2h}$, so the chain mixes slowly as $|\rho |$ approaches one.

A logistic-normal count model takes $Y_i\mid {\theta }_i\sim \mathrm{Bin}(n_i,{\mathrm{logit}}^{-1}{\theta }_i)$ and ${\theta }_i=x_i^T\beta +\lambda Z_i+{\eta }_i$, where $Z$ is a stationary unit-variance autoregressive process of order one with coefficient $a$ and the independent ${\eta }_i$ have normal distribution with variance $v$. The marginal latent variance is ${\lambda }^2+v$, and its off-diagonal covariance is ${\lambda }^2{a}^{|i-j|}$. Random success probabilities produce overdispersion relative to the binomial distribution.

A linear filter of a stationary time series with real coefficients $(a_j{)}_{j\in \mathbb{Z}}$ satisfying ${\sum }_j|a_j|<\infty$ is $Y_t={\sum }_j{a}_j{X}_{t-j}$. Its filter generating function is the Laurent series $A(z)={\sum }_j{a}_j{z}^j$, absolutely convergent on $|z|=1$, and its filter transfer function is $H_A(\lambda )=A(e^{-i\lambda })$. For a one-sided filter this is a power series; a general two-sided filter need not converge on a larger annulus.

The filter theorem states that if $X$ is second-order stationary, then $Y$ exists in mean square, is second-order stationary, and has mean and covarianceIts spectral measure of a stationary time series is $d{F}_Y(\lambda )=|H_A(\lambda ){|}^{2}d{F}_X(\lambda )$. In particular, if $X$ has a time-series spectral density in the convention ${\gamma }_X(h)={\int }_{-\pi }^{\pi }{e}^{ih\lambda }{f}_X(\lambda )d\lambda$, thenTo prove existence, the $L^2$ norm of a tail of the filter is at most $\Vert X_0{\Vert }_2$ times the sum of the omitted $|a_j|$, which tends to zero. Hence finite partial sums converge in $L^2$. Expectations and covariances pass to the limit, and the covariance double sum is absolutely convergent because $|{\gamma }_X(h)|\le {\gamma }_X(0)$. Its formula depends only on $h$, proving stationarity. Insert the spectral representation of ${\gamma }_X(h-j+k)$ and interchange sums and integration; the bound ${\sum }_{j,k}|a_j{a}_k|F_X([-\pi ,\pi ])<\infty$ justifies the interchange. The resulting factor is ${\sum }_j{a}_j{e}^{-ij\lambda }{\sum }_k{a}_k{e}^{ik\lambda }=|A(e^{-i\lambda }){|}^2$. This proves the spectral statement, including the spectral density transformation under a linear filter without assuming that the input autocovariances are absolutely summable.

For an autoregressive moving-average model, use real polynomials normalized by ${\varphi }_0={\theta }_0=1$ and first work with the reduced noise-driven representation. If $\varphi$ has no zero on the unit circle, the rational function $C(z)=\theta (z)/\varphi (z)$ has a Laurent expansion in an annulus containing that circle, with absolutely summable coefficients. Filtering the white noise with it gives a stationary solution. Its time-series spectral density isConversely, for coprime $\varphi ,\theta$, a unit-circle zero of $\varphi$ would make this ratio nonintegrable, since the numerator does not vanish there; thus there is no finite-variance stationary noise-driven solution. A stationary solution can use a bilateral filter when autoregressive roots lie inside the unit circle. The stronger causal condition, meaning dependence only on present and past noise, requires every zero of $\varphi$ strictly outside the unit disk. Common factors must be handled before applying these conditions; a cancelled unit-root factor can also allow additional deterministic stationary components, which are outside the minimal noise-driven convention.

Identifiability means that different admissible parameters cannot produce the same stationary process law. In this zero-mean Gaussian setting, its law is determined by its covariance, equivalently its spectrum. Unrestricted ARMA coefficients are not identifiable from that law: if a moving-average factor has root $r$ inside the disk, then on $|z|=1$,Reflecting the root outside and multiplying the innovation variance by $|r{|}^{-2}$ preserves the spectrum, and therefore the Gaussian law. Common polynomial factors and arbitrary overall scaling give further redundancy. A regular canonical identifiable parameterization requires constant coefficients one, coprime polynomials of their stated minimal orders, all zeros of both $\varphi$ and $\theta$ outside the closed unit disk, and ${\sigma }^2>0$.

Here is a proof of canonical identifiability of Gaussian ARMA models. Suppose two such models have equal spectra and write their innovation standard deviations as $\sigma ,\tau >0$. The ratio $R(z)=\sigma \theta (z)\tilde{\varphi }(z)/[\tau \tilde{\theta }(z)\varphi (z)]$ and its reciprocal are analytic on a neighbourhood of the closed unit disk, with modulus one on its boundary. The maximum modulus principle applied to both makes $|R|=1$ throughout the disk, so $R$ is constant. Its value at zero is the positive number $\sigma /\tau$, forcing $R=1$ and $\sigma =\tau$. Therefore $\theta \tilde{\varphi }=\tilde{\theta }\varphi$; coprimality and the normalization at zero force equality of both polynomials. This proves the claimed parameter uniqueness within that canonical class.

In this class, $D(z)=\varphi (z)/\theta (z)$ is analytic on a disk of radius greater than one. Its Taylor series coefficients are absolutely summable, soThis is an invertible time-series representation, proved by composition of the stable filters. Causality makes ${\epsilon }_t$ orthogonal to past observations; invertibility makes past noises recoverable from past observations. Their closed past linear spans agree, so this noise is the linear innovation and the one-sided expansion $X_t={\sum }_{j\ge 0}{c}_j{\epsilon }_{t-j}$ is the Wold representation. Mere stationarity without the stated canonical restrictions does not imply invertibility of the originally written noise representation.

For this one-sided canonical representation, $C(z)=\theta (z)/\varphi (z)={\sum }_{j\ge 0}{c}_j{z}^j$. Comparing coefficients gives the ARMA coefficient recursionsThus for exactly the range asked for,The characteristic rates are reciprocals of the zeros of $\varphi$. Partial fractions therefore express the tail as sums of $P_r(k)r^k$, where $|r|<1$ and the degree of $P_r$ is at most one less than the corresponding root multiplicity. Hence the coefficients decay exponentially, with possible polynomial and damped oscillatory factors. If $p=0$, they instead vanish after $q$. For a noncausal or noninvertible original specification, the Wold coefficients refer to the canonical spectral factor; using the original driving noise as the Wold innovation without checking these restrictions would be incorrect.

The autocovariance is $\gamma (h)={\sigma }^2{\sum }_{j\ge 0}{c}_j{c}_{j+|h|}$. Analyticity of $C$ on a disk of radius $R>1$ gives $|c_j|\le M{R}^{-j}$ for a slightly smaller radius than its first pole, and consequently $|\gamma (h)|\le {\sigma }^2{M}^2{R}^{-|h|}/(1-R^{-2})$. Moreover, multiply the ARMA equation at time $t$ by $X_{t-h}$ and take expectations. For $h>q$, all noises ${\epsilon }_{t-s}$ with $0\le s\le q$ are later than $X_{t-h}$ and orthogonal to it. For $h>\max (p-1,q)$ this givesThe covariance tail therefore has the same characteristic exponential rates and multiplicity patterns as the filter coefficients, subject to possible cancellation of amplitudes. Pure moving averages have an exact covariance cutoff.

For the final process, factor the polynomials:The common factor has its zero at $3/2$, so it can be cancelled as a stable filter. Explicitly, let $Y_t=(1+B/3)X_t-{\epsilon }_t$; the original equation gives $Y_t=(2/3)Y_{t-1}$. The second moments of $Y_t$ are uniformly bounded because those of $X_t$ and ${\epsilon }_t$ are. Iterating gives $Y_t=(2/3{)}^m{Y}_{t-m}$, whose $L^2$ norm tends to zero as $m\to \infty$, so $Y_t=0$ in mean square. ThusSumming the covariance series givesThe process is a causal stationary autoregressive process of order one, with alternating-sign geometric correlations, and is invertible. The displayed ARMA(2,1) description is nonminimal because of its common factor; its minimal identifiable class is ARMA(1,0).

Iterating the recurrence yieldsThus every finite collection of the $X_j$ is a linear transformation of independent standard normal random variables, and has a multivariate Gaussian distribution. The expected values are zero. Independence of the $Y_k$ givesFor $i<j$, the recurrence writes $X_j={\rho }^{j-i}{X}_i$ plus a sum of future innovations independent of $X_i$. Hence $\mathrm{Cov}(X_i,X_j)={\rho }^{j-i}$. The required vector law isThe covariance matrix is positive definite: the lower-triangular transformation from $(Y_1,\dots ,Y_n)$ has nonzero diagonal entries $1,\sqrt{1-{\rho }^2},\dots ,\sqrt{1-{\rho }^2}$. This also identifies a stationary autoregressive process of order one, with marginal density $f=\varphi$.

With the standard normal density as the kernel for density estimation, the two characteristic functions are $\psi (t)=e^{-t^2/2}$ and ${\psi }_K(ht)=e^{-h^2{t}^2/2}$. The Gaussian random variable $X_1-X_{j+1}$ has mean zero and variance $2(1-{\rho }^j)$, soThis expression is real. Substitution in the given integrated covariance expression, followed by the Gaussian integral ${\int }_{\mathbb{R}}{e}^{-a{t}^2}dt=\sqrt{\pi /a}$ for $a>0$, gives

To bound the total Gaussian autoregressive kernel variance correction, apply the mean value theorem to $u\mapsto u^{-1/2}$. For $j\ge 1$, the whole interval between $1+h^2-{\rho }^j$ and $1+h^2$ is bounded below by $1-\rho >0$. ThereforeSumming the geometric series gives the uniform boundIt follows from the supplied variance identity thatThe constant depends on the fixed $\rho$, but not on $h$ or $n$. In particular this applies to the requested shrinking-bandwidth regime; the covariance bound itself does not need $nh\to \infty$.

All observations have the same marginal density $f$, so dependence changes the variance but not the bias of a kernel density estimator: $\mathbb{E}{\hat{f}}_h=K_h\ast f$. Write $R(q)={\int }_{\mathbb{R}}{q}^2$ and ${\mu }_2(K)=\int u^{2}K(u)du$. For a smooth density with $f^{\prime \prime }\in L^2$, the integrated variance of a kernel density estimator in the independent case is $R(K)/(nh)+O(n^{-1})$. A second-order Taylor expansion of the convolution gives $K_h\ast f-f=(h^2{\mu }_2(K)/2)f^{\prime \prime }+o(h^2)$ in the L2 norm. Thus the bias-variance decomposition of mean squared error givesThe first two terms are the asymptotic mean integrated squared error. The additional dependence term is smaller than $n^{-4/5}$, so the leading optimal bandwidth and optimal rate are the same as for independent data:Here both $K$ and $f$ are the standard normal density. Since $f^{\prime \prime }(x)=(x^2-1)\varphi (x)$, direct Gaussian integrals giveConsequently the explicit leading bandwidth isAt this bandwidth the leading integrated variance is four times the leading integrated squared bias, which is also a check on the minimization constant.

Every $X_n$ is integrable: induction gives $|X_n|\le a^n|x|+{\sum }_{j=0}^{n-1}{a}^j$. Since the innovation has expectation zero, the recursion impliesStarting from $\mathbb{E}{X}_0=x$, induction therefore givesThis holds for every $a>0$, including $a=1$. The independence of the signs also identifies this as an autoregressive process of order one, although zero mean of each sign already suffices for the displayed expectation recursion.

For the autoregressive process of order one, the formula at $t=0$ reads $r_0=r_0$, with an empty sum. If it holds at time $t$, substitution into the recursion givesThus mathematical induction provesNo restriction such as $|\beta |<1$ is needed for this finite-time identity.

Put $n=T-t$ and define the finite geometric seriesStarting the autoregressive process of order one at time $t$ gives $r_{t+k}={\beta }^k{r}_t+{\sum }_{j=1}^k{\beta }^{k-j}{\xi }_{t+j}$. ConsequentlyThe future innovations are independent and identically distributed random variables, independent of ${\mathcal{F}}_t$. Factorizing their exponential conditional expectation and using the cumulant-generating function $K$ yieldsThus the exponential-affine bond pricing coefficients areThey vanish at maturity, giving $P(T,T)=1$. Everywhere finiteness of $K$ ensures that each zero-coupon bond price is positive and finite, including when $\beta$ is zero, negative, or one.

A zero-mean autoregressive process of order one satisfieswhere ${\epsilon }_t$ is white noise with variance ${\sigma }_{\epsilon }^2>0$. A causal time series uses present and past innovations only. For $|\alpha |<1$ the serieshas mean-square convergence because ${\sum }_j{\alpha }^{2j}<\infty$, and it supplies the causal weakly stationary process. Its variance is ${\sigma }_{\epsilon }^2/(1-{\alpha }^2)$. For $h\ge 0$, only the common innovations contribute to the autocovariance:Evenness then givesWhen $\alpha <0$, the signs alternate; when $\alpha =0$, the process is white noise. With nondegenerate innovations, $|\alpha |\ge 1$ does not give a causal finite-variance stationary solution. These covariance calculations need uncorrelated innovations, not Gaussianity; Gaussian innovations additionally yield a Gaussian process.

Assume the innovations are independent of the previous history and have finite exponential moments at every real argument, as required by the finite-valued affine definition. Conditioning on $X_{t-1}=x$ givesHence the autoregressive process of order one is an affine process, withHere and in the remaining parts, an affine process uses a time-homogeneous Markov process: the one-step transform is the same at every time. The displayed definition at time $1$ alone would not determine later transitions of an arbitrary time-inhomogeneous Markov process.

Iterating the autoregressive process of order one givesFor $q_r={\sum }_{j=r}^t{\theta }_j{a}^{j-r}$, regrouping the sum of the exponents givesThe independent innovations factorize the moment-generating function. ConsequentlyFinite sums avoid a division by $a-1$, so these formulas include $a=0$ and $a=1$; zeroth powers in these sums are $1$.

Write $\lambda ={\sigma }^2$, $v={\sigma }_0^2$, and $a=\sqrt{\rho }$. We keep the model's printed coefficient ${\sigma }^2{Z}_i$ literally. Its latent log odds can be writtenThe independence of the ${\theta }_i$ is conditional on $Z$: they are not marginally independent when the random effects $Z_i$ are correlated.

Holding the other predictors and the random-effect realization fixed, increasing predictor $j$ by one shifts the latent log odds by ${\beta }_j$, so it multiplies the conditional odds of a late departure by $e^{{\beta }_j}$. For a binary snow indicator, this compares the snow and no-snow days with other predictors held fixed. This is a conditional odds ratio, not generally the same as a marginal odds ratio after integrating the random effects.

Randomizing ${\theta }_i$ allows unmeasured daily conditions to change the success probability beyond the systematic linear predictor. In this logistic-normal regression with autoregressive random effects, let $P_i={\mathrm{logit}}^{-1}({\theta }_i)$ and ${\bar{p}}_i=\mathbb{E}[P_i]$. The law of total variance givesso the mixture allows overdispersion beyond a binomial distribution with fixed probability. The correlated part also allows related outcomes on nearby days.

The stationary autoregressive process of order one has $\mathrm{Var}(Z_i)=1$ and $\mathrm{Cov}(Z_i,Z_j)=a^{|i-j|}$. HenceThe ${\sigma }_0^2$ term is unstructured daily variability, whereas ${\sigma }^4$ is the temporally correlated component under the printed parameterization. Larger $\rho$ gives greater persistence: the lag-$h$ correlation of the latent log odds is ${\sigma }^4{\rho }^{h/2}/({\sigma }^4+{\sigma }_0^2)$.

The PDF itself prints both the coefficient ${\sigma }^2{Z}_i$ and the requested combination ${\sigma }^2+{\sigma }_0^2$. They are inconsistent as a marginal variance. If the intended coefficient were $\sigma Z_i$, the variance and off-diagonal covariance would instead be ${\sigma }^2+{\sigma }_0^2$ and ${\sigma }^2{\rho }^{|i-j|/2}$. The prior's argument list also repeats ${\sigma }^2$; its displayed density indicates the intended second variance parameter is ${\sigma }_0^2$.

There is a further substantive issue with that improper prior. After integrating the latent variables, the observed-data likelihood function is positive and continuous at $\lambda =0$ for fixed finite $\beta$, positive $v$, and $\rho \in (0,1)$. On a compact positive-volume set of those parameters it therefore has a positive lower bound for small $\lambda$. The prior density is $1/(\lambda v)$, so integrating over $\lambda$ gives ${\int }_0^{\epsilon }d\lambda /\lambda =\infty$. Thus the stated joint posterior distribution is improper, an instance of improper posterior from a log-uniform random-effect scale prior. The requested fixed-parameter conditional laws below are nevertheless proper; their existence does not remedy the joint impropriety.

Use the stationary autoregressive process of order oneMultiplication by $X_{t-1}-\mu$ and expectation, using independence of the innovation from the past, gives the autocovariance equation $\gamma (1)=\varphi \gamma (0)$. With $\bar{X}=n^{-1}{\sum }_t{X}_t$, define the sample autocovariances using the same denominator:Substitution into that equation gives the Yule–Walker estimator for an autoregressive process of order oneThis is exactly the lag-one sample autocorrelation function under the divisor-$n$ convention, provided the sample has positive variance. Using different divisors for different lags would change the estimator.