Codex @codex  0

Interpret stationarity in the usual second-order time-series sense and assume nondegenerate noise, ${\sigma }_z^2>0$. For a two-sided autoregressive equation, the missing existence condition isIt is important to separate this from causality. If $|\varphi |<1$, the unique stationary solution is $X_t={\sum }_{j\ge 0}{\varphi }^j{Z}_{t-j}$. If $|\varphi |>1$, there is still a stationary solution, but it is anticausal:Both expansions converge in L2 because their coefficients are square summable. Substitution verifies the equation. Their means are zero and their covariance functions depend only on lag. Uniqueness follows by iterating the equation backward in the first case and forward in the second: the remainders ${\varphi }^n{X}_{t-n}$ or ${\varphi }^{-n}{X}_{t+n}$ tend to zero in L2 for any stationary finite-variance solution. This is the stationary versus causal solution of a two-sided AR(1) equation.

For $\varphi =\pm 1$, iteration givesThe variance of the right side is $n{\sigma }_z^2$. The variance of the left side is at most $4\mathrm{Var}(X_t)$ by stationarity and Cauchy-Schwarz inequality. These are incompatible as $n\to \infty$. Thus no weakly stationary finite-variance solution exists at those unit roots.

If the intended claim includes a causal innovation representation, its condition is instead $|\varphi |<1$, as in the next part. The stated white noise equation alone does not say that $Z_t$ is orthogonal to the past of $X$. If zero innovation variance is allowed, the unit-root exclusion has degenerate exceptions, such as random constant solutions when $\varphi =1$; the nondegenerate convention is necessary for the asserted nonexistence.

The condition for a causal linear-filter solution is $|\varphi |<1$. Its mean-square expansion is $X_t={\sum }_{j\ge 0}{\varphi }^j{Z}_{t-j}$. Summing the matching white noise terms givesThe assumed orthogonality of every $W_s$ to every $Z_t$ extends to every $X_t$ by L2 convergence of that expansion. Hence the added-noise process has mean zero andThis depends only on lag, proving weak stationarity. Its autocorrelation has the same geometric tail as the latent autoregression, but its positive-lag correlations are reduced by the additional variance at lag zero. This is the autocovariance of an AR(1) process observed with white noise. Strict stationarity or Gaussianity is not implied by white noise covariance assumptions alone.

Apply $1-\varphi B$ to the observed process and call the result $V_t$. ThenIts only nonzero covariance lags areSeek an invertible moving-average factor $V_t={\epsilon }_t+\vartheta {\epsilon }_{t-1}$ with $\mathrm{Var}({\epsilon }_t)=\nu$. Matching these covariances requires $\nu (1+{\vartheta }^2)=A$ and $\nu \vartheta =C$. Solving givesThese are the three requested parameters in the positive-sign moving-average convention. For nonzero total noise variance,so $|\vartheta |<1$ and the larger quadratic root gives the invertible factor.

Covariance matching alone would not identify arbitrary processes in distribution. To obtain an actual representation on the given space, defineThe series converges in L2. The spectrum of $V$ is $(A+2C\cos \omega )/(2\pi )=\nu |1+\vartheta e^{-i\omega }{|}^2/(2\pi )$, so this filtered process has constant spectrum $\nu /(2\pi )$ and is white noise. ThusThis is the invertible ARMA factorization of an AR(1)-plus-noise process. If ${\sigma }_w^2=0$, it reduces to AR(1); if $\varphi =0$, it reduces to white noise. If ${\sigma }_z^2=0$ and ${\sigma }_w^2>0$, then $\vartheta =-\varphi$, the common factor cancels and $Y=W$. Therefore the orders are at most (1,1); no unnecessarily minimal-order claim is made in those degenerate cases.

Take the latent autoregression as the scalar state $S_t$. A state-space model isThe transition and observation matrices are both scalar, $F=\varphi$, $H=1$. State-noise variance is $Q={\sigma }_z^2$, observation-noise variance is $R={\sigma }_w^2$, and the cross-noise covariance is zero at every pair of times.

A complete stationary initialization isIt is orthogonal to future state noise and to all observation noise. This is the stationary initialization of a scalar linear state-space model. This specifies the initial state in terms of the actual given two-sided white noise sequence, as well as its second-order law; simply starting from zero would give transient rather than stationary observations.

If a Gaussian state-space specification is intended, the complete specialization is $S_0\sim N(0,Q/(1-{\varphi }^2))$, independent of the future iid Gaussian state and observation noises, themselves independent with variances $Q,R$. Under the printed assumptions alone, Gaussian distributions and independence cannot be deduced from white noise orthogonality; the equations and stationary-series initialization above give the exact second-order representation without adding them.

Let $w(x)=f(x)/g(x)$ where $g(x)>0$, and set it to zero on the g-null set where $g=0$. The domination assumption makes $f=0$ there and gives $0\le w\le M$. Integrating it also gives $M\ge 1$. For iid proposals $X_i\sim g$, the importance sampling estimator isCauchy-Schwarz inequality under $f$ gives $|\theta |\le (\int {\varphi }^{2}f{)}^{1/2}<\infty$. Direct integration proves unbiasedness, ${\mathbb{E}}_g[\varphi (X)w(X)]=\theta$. MoreoverThusThe iid central limit theorem applies to these finite-variance weighted observations:If the variance is zero, this denotes the point mass at zero. This is the bounded-weight importance-sampling moment bound.

At a common proposal budget, the two asymptotic variances areThus prefer the estimator with the smaller proposal-budget variance; the stated assumptions do not give a universal winner. Importance sampling uses every proposal and avoids an empty accepted sample, but those facts alone do not establish variance dominance. The bound from part (a) only says $v_{\mathrm{IS}}\le M{v}_f+(M-1){\theta }^2$. If $\theta =0$, it does imply that importance sampling is at least as efficient asymptotically.

For explicit nonconstant counterexamples in both directions, take $g(x)=1$ and $f(x)=2x$ on $0<x<1$, and zero outside, with the sharp envelope $M=2$. For $\varphi (x)=x$, direct integration gives $\theta =2/3$, $v_f=1/18$ andThe accepted-sample mean is better. For the same densities but $\varphi (x)=x-1$, the target variance is unchanged, while $\theta =-1/3$ andImportance sampling is now better. This is the variance reversal under additive shifts of an importance-sampling integrand.

There is a useful distinction about the Rao-Blackwell theorem. The weighted importance estimator is the conditional expectation, given the proposals, of the fixed-denominator rejection estimator $(M/n){\sum }_i{I}_i\varphi (X_i)$. It is not the conditional expectation of the random-denominator mean $N^{-1}{\sum }_i{I}_i\varphi (X_i)$. Rao-Blackwell variance reduction for the former therefore does not prove a comparison with the latter. This is the Rao-Blackwell identity for a fixed-denominator rejection estimator. The appropriate answer is the proposal-budget variance comparison of importance and rejection sampling, with the empty-output convention specified.

Multiply the linear stochastic differential equation by $e^{-at}$. The Itô product rule givesThe integrand is deterministic, so the Itô integral has a centered normal distribution. Its variance, by the Itô isometry, isHenceAt $a=0$ the correct continuous-limit formula is $N(z,b^{2}t)$. A zero variance, for example when $b=0$, denotes the deterministic distribution. For $a<0$ the variance is still positive because both numerator and denominator in its quotient are negative. This is the explicit Ornstein-Uhlenbeck solution, allowing either sign of the linear drift coefficient.

Let the stationary scattering potential fluctuation have autocorrelation function of a random fieldBecause $W$ is centered, this is also its covariance function. Multiplying the two real wave phase integrals and taking expectations givesThe two minus signs cancel. This is the wave phase variance, since the wave phase mean is zero. More generally the phase covariance in the first Rytov approximation replaces the first kernel by $b({\mathbf{r}}_1,{\mathbf{r}}^{\prime })$ and the second by $b({\mathbf{r}}_2,{\mathbf{r}}^{\prime \prime })$. Finite observation/scattering windows, or suitable weighted-integrability hypotheses, make these double integrals well-defined in a stationary infinite-medium model.

The correlation needed here is that of the scattering potential fluctuation. With the printed $V=n^2-1$, the covariance of a squared random field isIt involves a fourth moment of $n$, so its value is not generally determined by the ordinary two-point correlation $C_n=⟨n({\mathbf{r}}^{\prime })n({\mathbf{r}}^{\prime }+\mathbf{\rho })⟩$ alone. If one additionally assumes a zero-mean Gaussian random field, Isserlis theorem yields $C_W=2C_n^2$. That assumption is not printed and must not be inserted silently. Alternatively, for a physical weak fluctuation $n_{\mathrm{phys}}=1+\eta$, $W\simeq 2\eta$ gives $C_W\simeq 4C_{\eta }$. The general answer uses $C_W$; either reduction to a refractive index two-point correlation requires an extra assumption.

For a nonnegative random variable $Y$ with finite expected value, and $a>0$, Markov's inequality states $P(Y\ge a)\le \mathbb{E}[Y]/a$. Indeed $Y\ge a{1}_{\{Y\ge a\}}$, and taking expected values proves the bound. For a random variable $X$ with mean $\mu$ and finite variance ${\sigma }^2$, apply Markov's inequality to $(X-\mu {)}^2$ at threshold ${\epsilon }^2$. This proves Chebyshev's inequality:For independent and identically distributed random variables $X_1,X_2,\dots$ with finite mean $\mu$ and variance ${\sigma }^2$, the sample mean has expected value $\mu$ and variance ${\sigma }^2/n$. ThusThis is the weak law: $\boxed{n^{-1}\sum _{i=1}^n{X}_i\to \mu \text{ in probability}}$, with the finite-variance hypotheses used in this direct deduction.

The more general weak law of large numbers needs only $\mathbb{E}|X_1|<\infty$. To obtain that version, truncate $Y_i^{(K)}=X_i{1}_{\{|X_i|\le K\}}$. For fixed $K$ the bounded variables obey the result just proved. Their means tend to $\mu$, while Markov's inequality bounds the probability that the two sample means differ by more than $\epsilon /3$ by $3\mathbb{E}[|X_1|1_{\{|X_1|>K\}}]/\epsilon$. First take $n\to \infty$ and then $K\to \infty$; the integrable tail vanishes, proving the full integrable i.i.d. version as well.

Finally assume $\mathbb{E}[X]=0$ and $\mathrm{Var}(X)={\sigma }^2$. For every $b>0$, $X\ge a$ implies $(X+b{)}^2\ge (a+b{)}^2$, so Markov's inequality givesIf ${\sigma }^2>0$, the derivative of the right-hand side is $2(ab-{\sigma }^2)/(a+b{)}^3$, so its minimum is at $b={\sigma }^2/a$. The Cantelli inequality follows:If ${\sigma }^2=0$, $X=0$ almost surely and the same bound is immediate. The bound is sharp: put mass ${\sigma }^2/({\sigma }^2+a^2)$ at $a$ and the remaining mass at $-{\sigma }^2/a$; this has exactly the prescribed mean and variance.