Past exam of the mathematics course of the University of Cambridge / 2015 / iii / Paper 37 (Time Series and Monte Carlo Inference)

A stationary causal time series exists. The infinite moving-average representationconverges in the sense of mean-square convergence because the driving variables are independent random variables and ${\sum }_j{4}^{-j}<\infty$. Shifting the series gives $X_t=\frac{1}{2}{X}_{t-1}+{\epsilon }_t$. Its expected value is zero and its autocovariance isThus it is a weakly stationary process; since the driving variables have a normal distribution, it is also a Gaussian process and a strictly stationary process. Throughout the stationarity discussion, take ${\sigma }^2>0$; degenerate zero noise permits trivial constant solutions.

No weakly stationary process solves the model with nondegenerate noise. Iteration would giveFor a weakly stationary process with finite variance $V$, the Cauchy-Schwarz inequality instead gives $\mathrm{Var}(X_t-X_{t-m})=2V-2\gamma (m)\le 4V$. These two bounds contradict each other for large $m$. This argument does not assume that $X_{t-m}$ is independent of the intervening noise, so it excludes noncausal solutions too. The corresponding unit-root autoregressive process has persistent random walk behavior rather than stationary fluctuations.

A noncausal stationary autoregression exists. On the two-sided time axis defineThis series converges in the sense of mean-square convergence, and direct subtraction gives $X_t-2X_{t-1}={\epsilon }_t$. The expected value and autocovariance areConsequently this is a weakly stationary process and, by Gaussianity, a strictly stationary process. It is an example of noncausal stationary autoregression, since $X_t$ uses future noise. In particular, $X_{t-1}$ is correlated with ${\epsilon }_t$, so the usual causal variance recursion is inapplicable. If an additional assumption required the driving noise to be independent of past observations, this solution would be excluded; that assumption is not stated here.

A unit-root autoregressive process retains shocks permanently, whereas a causal time series with $|\varphi |<1$ reverts towards its mean. Testing the unit root determines whether stationary autoregressive analysis is appropriate or differencing is needed. For the model without an intercept or trend, use the Dickey–Fuller test against the lower-sided alternative $\varphi <1$ near the null. PutThis is the ordinary regression statistic for a zero coefficient when $\Delta X_t$ is regressed on $X_{t-1}$, but its null probability distribution is not the usual Student law. Under the standard unit-root initialization $X_1=o_P(\sqrt{n})$ and innovations independent of the starting value,where $W$ is standard Brownian motion. If $d_{\alpha }$ is the lower $\alpha$-quantile of $D$, the asymptotic level-$\alpha$ critical region isThe deterministic terms and null initialization must match the critical-value table. For exact finite-sample size of a statistical test, calibrate the statistic from its Gaussian random walk null with the specified initial condition and noise scale; for a zero starting value its distribution is scale-free. The printed two-sided recurrence alone specifies neither an initial law nor a universal finite-sample critical value. Ordinary normal quantiles do not give the intended size of a statistical test.

For a stationary linear autoregression, causal time series means that the observation uses only current and past driving noise:This is a convergent in the sense of mean-square convergence infinite moving-average representation. The stronger usual stable-filter definition requires absolute summability; the argument below also handles the square-summable definition. Let $\Phi (z)=1-{\sum }_{k=1}^p{\varphi }_k{z}^k$ and $\Psi (z)={\sum }_{j\ge 0}{\psi }_j{z}^j$. Substituting the filter into the recurrence and comparing coefficients of the orthogonal noise gives ${\psi }_0=1$ and ${\psi }_j={\sum }_{k=1}^{\min (p,j)}{\varphi }_k{\psi }_{j-k}$. HenceThe Cauchy-Schwarz inequality ensures that $\Psi$ is an analytic function in this disk. Thus $\Phi$ has no zero strictly inside it. A boundary zero is also impossible: a zero of multiplicity $m\ge 1$ at $z_0=e^{i{\omega }_0}$ makes $|1/\Phi (r{e}^{i\omega }){|}^2$ at least a constant times $[(1-r{)}^2+(\omega -{\omega }_0{)}^2{]}^{-m}$ near that point. Its integral over $\omega$ diverges as $r\uparrow 1$. On the other hand, orthogonality of complex exponentials givesa contradiction. Therefore the causality root criterion for an autoregressive model isUnder absolute summability, the shorter boundary argument is continuity of $\Psi$ on the closed disk and the identity $\Phi \Psi =1$ there.

By the causality root criterion for an autoregressive model, choose $r>1$ smaller than the modulus of every root of $\Phi$. The function $\Psi =1/\Phi$ is an analytic function on and inside $|z|=r$. Writing $M={\max }_{|z|=r}|1/\Phi (z)|$, the Cauchy estimate gives $|{\psi }_j|\le M{r}^{-j}$. For $h\ge 0$, independence of the noise in the infinite moving-average representation givesThe autocovariance is symmetric in the lag. Thus exponential decay holds withThis is exponential autocovariance decay of a causal autoregression. If $\Phi$ is constant, there are no roots and any $r>1$ works.

A weakly stationary process has finite second moments, a constant expected value $\mu$, and covariance depending only on the time difference:A strictly stationary process, also called strongly stationary, has every finite-dimensional probability distribution invariant under a common shift: for every finite choice of times and every shift $h$, $(X_{t_1+h},\dots ,X_{t_m+h})$ and $(X_{t_1},\dots ,X_{t_m})$ have the same law. strict stationarity with finite second moments implies weak stationarity. weak stationarity alone does not determine the full joint law, and strict stationarity alone does not guarantee finite moments.

Two useful features of autoregressive conditional heteroscedasticity are persistent changes in conditional scale and excess unconditional kurtosis. Its time-varying conditional variance can explain volatility clustering, where large absolute returns occur in groups even when signed returns have little autocorrelation. A homoscedastic autoregressive moving-average model has a fixed innovation variance.

Also, a conditional normal distribution with a random scale is a Gaussian scale mixture. Its unconditional kurtosis can exceed $3$, or its fourth moment can be infinite. A Gaussian autoregressive moving-average model remains jointly Gaussian and cannot reproduce this effect. In the particular lag-two ARCH process, persistence of the squared scale occurs within each parity subsequence; the two parity subsequences are independent.

First justify the dependence structure rather than assume that adjacent squares behave like an ordinary lag-one ARCH model. Write $Z_t=X_t^2$. Iterating its nonnegative recurrence givesThe product coefficient has expected value ${\alpha }_2^m$, so it tends to zero in probability. The stationary $Z_{t-2m}$ are a tight family; consequently the remainder tends to zero in probability, without needing independence between that remainder's factors. The increasing partial sums therefore give the representationThis proves the parity decomposition of a lag-two ARCH process: even and odd observations are functions of disjoint noise families. It also shows that the positive scale ${\sigma }_t$ is a function of past noise, independent of ${\epsilon }_t$. The conditional expectation of $X_t$ given past noise is zero, and the second-moment recursion is $m_2={\alpha }_0+{\alpha }_2{m}_2$. independence of the parity families gives the adjacent-square covariance. ThusTo establish finiteness of the fourth moment before using its recursion, note that the $L^2$ norm of the $k$th term in the positive series for $Z_t$ is ${\alpha }_0\sqrt{3}({\alpha }_2\sqrt{3}{)}^k$. The triangle inequality in L2 space makes the series square-integrable when $3{\alpha }_2^2<1$. independence of the current noise and past scale then yieldsand henceIf $3{\alpha }_2^2\ge 1$, a finite $m_4$ would make $(1-3{\alpha }_2^2)m_4=3{\alpha }_0^2+6{\alpha }_0{\alpha }_2{m}_2>0$, which is impossible. Its fourth moment is then infinite. The adjacent-square product remains integrable by parity independence, even in that case.

The useful autoregressive model is for the squares, not for signed observations. Put $Y_t=X_t^2$ andThenThe errors form a martingale difference sequence relative to the noise history, because the current standardized noise is independent of the past. When the fourth moment is finite they have finite variance and are uncorrelated across distinct times, although their conditional variance depends on the regressor. In centered form, $Y_t-m_2={\alpha }_2(Y_{t-2}-m_2)+u_t$.

For ordinary least squares, use the $n-2$ response-regressor pairs $y_t=X_t^2$, $r_t=X_{t-2}^2$, $t=3,\dots ,n$. With their separate means $\bar{y}$ and $\bar{r}$, minimize ${\sum }_{t=3}^n(y_t-a-b{r}_t{)}^2$. If the regressor sum of squares is positive, the estimators areThe two means use the matched pairs; replacing them indiscriminately by a single full-sample mean is not the exact least-squares formula. The noise-series representation supplies an ergodic stationary process. If $3{\alpha }_2^2<1$, finite regressor second moments and the error's zero conditional expectation justify the usual population regression and statistical consistency argument. The observations still define a finite-sample least-squares fit outside that moment range, but the ordinary finite-variance justification must not be claimed there. If parameter constraints are required, minimize the same criterion subject to $a>0$ and $0<b<1$, rather than assert that unconstrained estimates automatically satisfy them.

The covariance is zero. Let ${\mathcal{F}}_t=\sigma ({\epsilon }_s:s\le t)$. The stationary noise-series representation makes $f(X_t)$ measurable with respect to ${\mathcal{F}}_t$, whereas the martingale difference sequence property gives $\mathbb{E}[X_{t+h}\mid {\mathcal{F}}_{t+h-1}]=0$. The law of total expectation therefore givesSince $\mathbb{E}{X}_{t+h}=0$, this is the required covariance. All products are integrable by the Cauchy-Schwarz inequality, using finite second moments of $X$ and the stipulated square integrability of $f(X_t)$.

This covariance is also zero, but symmetry is essential to the proof. Write ${\epsilon }_t={\eta }_t|{\epsilon }_t|$, where ${\eta }_t$ is an independent fair sign. The normal distribution makes that sign independent of its magnitude and of every other driving variable. Conditional on the magnitudes and all noise except this sign, changing ${\eta }_t$ flips $X_t$ and leaves $X_t^2$ unchanged. Every future conditional scale uses only squared past observations, so $X_{t+h}$ is unchanged for $h>0$.

Thus $f(X_{t+h})$ is independent of the remaining fair sign in $X_t$. Averaging that sign gives $\mathbb{E}[X_{t}f(X_{t+h})]=0$, and henceThe Cauchy-Schwarz inequality again guarantees integrability. For odd $h$, the parity decomposition of a lag-two ARCH process also gives independence directly. For even $h$, the sign symmetry of an ARCH process supplies the argument; the martingale difference sequence property alone would not justify this reversed covariance.

For a centered weakly stationary process with $\gamma (0)>0$, the autocorrelation function isFor $h\ge 2$, let $P$ be orthogonal projection in L2 space onto the linear span of $X_{t-1},\dots ,X_{t-h+1}$. The lag-$h$ partial autocorrelation function is the correlation coefficient of $X_t-P{X}_t$ and $X_{t-h}-P{X}_{t-h}$. It removes the linear contribution of the intervening observations. At lag one it is just $\rho (1)$. For nonsingular prediction covariance matrices, it is equivalently the last coefficient $a_{hh}$ in the order-$h$ linear predictor of $X_t$ from $X_{t-1},\dots ,X_{t-h}$. Equal residual variances, by stationarity, identify that coefficient with the residual correlation. If a residual has zero variance, this correlation is undefined; the nondegeneracy condition matters.

Use the sample autocorrelation function and sample partial autocorrelation function to look for approximate cutoffs. For a minimal causal autoregressive model of order $p$, the partial autocorrelation function is zero beyond $p$, while the autocorrelation function usually tails off, possibly with damped oscillation. For an invertible moving-average model of order $q$, the autocorrelation function is zero beyond $q$, while the partial autocorrelation function tails off. In a mixed autoregressive moving-average model, both generally tail off.

Thus the last visibly nonzero partial correlation suggests an autoregressive order, and the last visibly nonzero autocorrelation suggests a moving-average order. Finite samples do not produce exact zeros. Approximate white-noise reference bands can help flag clearly nonzero lags, but they are not universal confidence intervals for arbitrary correlated processes. Fit a few suggested orders and check residual autocorrelation, rather than treating these plots as a proof of the model order. This is order identification by autocorrelation cutoffs.

For the moving-average process of order one, $\gamma (0)={\sigma }^2(1+{\theta }_1^2)$, $\gamma (1)={\sigma }^2{\theta }_1$, and $\gamma (2)=0$. Regressing each endpoint on the single intervening observation leaves residual covariance $\gamma (2)-\gamma (1{)}^2/\gamma (0)$ and residual variance $\gamma (0)-\gamma (1{)}^2/\gamma (0)$. Therefore its lag-two partial autocorrelation function isThe denominator is positive. In particular, the moving-average autocorrelation cutoff at lag one does not make the lag-two partial correlation zero unless ${\theta }_1=0$.

Use cycles-per-time spectral density, with frequency $\omega \in [-1/2,1/2]$. The spectral representation theorem for a stationary time series gives the centered white noise representationwhere disjoint increments are orthogonal. Put ${\theta }_0=1$. Since the filter is finite, substitute each noise representation and interchange the finite sum with the integral:The new orthogonal increment measure is $d{Z}_X=\Theta (e^{-2\pi i\omega })d{Z}_{\epsilon }$. Its variance measure is therefore ${\sigma }^2|\Theta (e^{-2\pi i\omega }){|}^{2}d\omega$. The coefficients are real, so conjugation changes the sign of the exponent without changing the modulus. HenceWith angular frequency $\lambda =2\pi \omega$, the spectral density of a stationary process instead contains the factor $1/(2\pi )$. The convention explains its absence here. Invertibility is not needed for this finite-filter spectral calculation.

The autocovariance vanishes beyond lag $q$. For an ordinary consecutive sum, variance of a sum givesThe last sum is finite. For $h\ge 0$, $\gamma (h)={\sigma }^2{\sum }_{j=0}^{q-h}{\theta }_j{\theta }_{j+h}$. Counting all coefficient pairs givesFor the odd-indexed sample the lag-$h$ autocovariance is $\gamma (2h)$, so the same finite-sum argument gives ${\sum }_h\gamma (2h)$. Only coefficient pairs of the same parity contribute. If $E={\sum }_{j\mathrm{even}}{\theta }_j$ and $O={\sum }_{j\mathrm{odd}}{\theta }_j$, the odd-subsample limit isThis is the odd-subsample long-run variance of a moving average. Equivalently it is $[f_X(0)+f_X(1/2)]/2$ in the cycles convention. Under the printed invertibility assumption and positive noise variance, the polynomial is nonzero at both $1$ and $-1$, so both limits are positive. The same formulas also hold without invertibility, when cancellations can make a limit zero.

The generalized quantile function isThe tail limits of a cumulative distribution function make the defining set nonempty and its infimum finite. Monotonicity and right continuity imply $F^{-1}(u)\le x$ exactly when $u\le F(x)$. To see the direction involving the infimum, let points in the defining set decrease towards it; right continuity gives $F(F^{-1}(u))\ge u$, so every larger $x$ also qualifies. Consequently, for a uniform distribution variable $U$,The probability-zero endpoints $U=0,1$ can be assigned arbitrary outputs. This proves inverse transform sampling even for distributions with atoms or flat portions; continuity or strict monotonicity of $F$ is not required.

Choose a proposal probability density function $q$ positive wherever $|h|f$ is nonzero, up to null sets, and draw independent and identically distributed random variables $Y_1,\dots ,Y_n$ from $q$. The ordinary importance sampling estimator isThe support condition for importance sampling and absolute integrability give ${\mathbb{E}}_q[h(Y)f(Y)/q(Y)]=\int hf=\mu$, so this is an unbiased estimator. Its variance, possibly infinite, isLet $A=\int |h|f$. If $A>0$, the Cauchy-Schwarz inequality yieldsEquality holds for $q$ proportional to $|h|f$. Thus the minimum-variance importance distribution and the minimum variance areFor a constant-sign integrand this is zero variance. If $A=0$, the integrand vanishes almost everywhere and the zero estimator already has zero variance. Requiring the proposal to cover all of the target's support, including where $h=0$, can exclude $q_{\ast }$; in that stricter class, $(1-ϵ)q_{\ast }+ϵf$ approaches the same infimum as $ϵ\downarrow 0$. This distinguishes coverage of the integral from coverage of every target event.

Integrating the Weibull distribution density gives $F(x)=1-\exp (-x^{\alpha }/\beta )$ for $x>0$. Inverse transform sampling therefore givesReplacing $1-U_1$ by $U_1$ produces the same probability distribution. Both uniforms belong to $(0,1)$ with probability one. Here the conventional Weibull scale is ${\beta }^{1/\alpha }$, rather than $\beta$.

The Box-Muller transform uses independent uniforms to setThe radius has the unit Rayleigh distribution, with density $r{e}^{-r^2/2}$ for $r>0$, and the angle is independently uniform on $[0,2\pi )$. Their joint density is $(2\pi {)}^{-1}r{e}^{-r^2/2}$. The Cartesian change of variables has absolute Jacobian determinant $r$, so the joint density of $(X_1,X_2)$ isThe factorization proves that the outputs are independent random variables, each with a standard normal distribution.

Recognize the integrand as a Gaussian scale mixture. If $R$ has the unit Rayleigh distribution and $Z$ is an independent standard normal distribution variable, then the conditional density of $RZ$ given $R=r$ is $(\sqrt{2\pi }r{)}^{-1}{e}^{-x^2/(2r^2)}$. Multiplying by the radius density givesThus use $U_1$ for the independent radius and $U_2,U_3$ for the normal output of the Box-Muller transform:The mixture argument also proves that $g$ integrates to one, by the Tonelli theorem. As a check, $R^2$ is exponential of rate $1/2$, so the characteristic function of $RZ$ is $\mathbb{E}{e}^{-t^2{R}^2/2}=1/(1+t^2)$. This is the Rayleigh-normal scale mixture, with Laplace distribution density $e^{-|x|}/2$.

Let $P$ be the transition kernel of the Markov chain. It is phi-irreducible if there is a nonzero sigma-finite measure $\phi$ such that, for every measurable $A$ and every starting state $x$,Equivalently, there is positive probability of eventually reaching every set of positive reference measure. For a finite state space with counting measure, this becomes ordinary irreducible Markov chain accessibility. On a continuous space, requiring access to each individual point would usually be inappropriate; positive-measure sets play that role. Irreducibility requires positive probability, rather than probability one, and does not itself assert recurrence or a finite mean return time.

A convenient general-state-space ergodic theorem is the following. For a positive Harris recurrent Markov chain with invariant probability measure $\pi$, and a measurable function $h$ with $\pi |h|<\infty$,Here $\pi h=\int hd\pi$ is the stationary expected value. Harris recurrence means that every set of positive irreducibility measure is visited almost surely from every state; positive recurrence supplies an invariant probability rather than only an infinite invariant measure. The ergodic theorem for a positive Harris recurrent Markov chain holds from any starting state under these Harris hypotheses. aperiodicity is not necessary just for averages.

A useful sufficient form of the central limit theorem for a geometrically ergodic Markov chain adds aperiodicity, geometric ergodicity, and $\pi (|h{|}^{2+\delta })<\infty$ for some $\delta >0$. It givesThese are sufficient hypotheses, not a claim that irreducibility alone ensures a central limit theorem. The Markov chain Monte Carlo asymptotic variance uses stationary covariances ${\gamma }_h(k)={\mathrm{Cov}}_{\pi }(h(X_0),h(X_k))$, with ${\gamma }_h(0)={\mathrm{Var}}_{\pi }h$. The series is absolutely convergent under the stated sufficient assumptions. If ${\gamma }_h(0)>0$, write ${\rho }_h(k)={\gamma }_h(k)/{\gamma }_h(0)$ and $v_h={\gamma }_h(0){\tau }_{\mathrm{int}}$, where the integrated autocorrelation time is ${\tau }_{\mathrm{int}}=1+2{\sum }_{k\ge 1}{\rho }_h(k)$. When $v_h>0$, the effective sample size of a Markov chain is approximately $n/{\tau }_{\mathrm{int}}$. Negative correlations can reduce the asymptotic variance; a zero asymptotic variance gives a degenerate normal limit. For constant $h$, variance is zero and the autocorrelation normalization is undefined.

First require a proper target: $0<Z={\int }_{{\mathbb{R}}^d}\pi (x)dx<\infty$ and $\bar{\pi }=\pi /Z$. The Metropolis–Hastings algorithm uses the Metropolis–Hastings acceptance probabilitySufficient general conditions are phi-irreducibility, aperiodicity, and a drift-minorisation condition establishing geometric ergodicity of the resulting chain, together with a stationary $(2+\delta )$th moment for the observable being averaged. These imply the central limit theorem for a geometrically ergodic Markov chain. The observable's moment condition must be included: conditions on the sampler alone cannot give the theorem for every arbitrary function.

A concrete stronger condition, directly in terms of the proposal, isTogether with a bounded observable, this is an especially simple sufficient answer. The accepted proposal density is $\min \{q(y\mid x),\bar{\pi }(y)q(x\mid y)/\bar{\pi }(x)\}$, so it is at least $ϵ\bar{\pi }(y)$. The kernel satisfies a global minorization condition with the target, hence uniform geometric ergodicity, positive Harris recurrence, and aperiodicity. An independent proposal whose importance weight $\bar{\pi }(y)/q(y)$ is uniformly bounded is one example. No claim of these properties follows merely from writing down a positive proposal.

The algorithm is a Random-scan Gibbs sampler. Start at a point where the target is positive, with a proper normalizing integral and well-defined full conditional distributions. Its state is the entire coordinate vector. Choosing the initial point fixes the initial law; it need not already be the invariant law. The transition rule below preserves the target irrespective of this choice.

At each transition, choose a coordinate uniformly and independently of the previous coordinate choices. Draw a fresh value from that coordinate's full conditional distribution given the current remaining coordinates, and keep all other coordinates fixed. This supplies the random-selection and conditional-draw operations lost in the TeX transcription. The update depends only on the current vector and fresh randomness, which proves the Markov property. The printed mathematical requests are the kernel and its detailed balance, answered in the following two subsections.

For the Random-scan Gibbs sampler, the correct transition kernel is a measure, since a one-coordinate update is singular with respect to full-dimensional Lebesgue measure when $p>1$. Write $x_{-i}$ for all coordinates except $i$. ThenEach Dirac measure fixes an unupdated coordinate. Equivalently, $P(x,A)=p^{-1}{\sum }_i\int 1_A(x_{-i},z){\pi }_i(z\mid x_{-i})dz$, with $z$ placed in coordinate $i$. Replacing this kernel by an ordinary density on the whole product space would omit those fixed-coordinate constraints.

Let $\bar{\pi }$ be the normalized target and ${\bar{\pi }}_{-i}$ its marginal distribution. A coordinate-$i$ kernel $P_i$ leaves $x_{-i}=x_{-i}^{\prime }$ and has the joint old-new measureThis is symmetric in the old and new state: the common remaining coordinates are fixed and the two conditional factors exchange places. Thus each coordinate kernel satisfies detailed balance. Averaging them with the fixed weights $1/p$ provesIntegrating out the old state proves invariance of $\bar{\pi }$. The common normalization cancels, so detailed balance can also be written with the unnormalized target. This establishes detailed balance of a random-scan Gibbs sampler and a reversible Markov chain. Fresh independent random choices make the update a Markov chain, as explained above; invariance is not an assertion that an arbitrarily initialized chain starts in stationarity.

Use the Bayes' theorem to multiply the Ising model prior by the conditionally independent normal likelihood factors. For fixed observed $x$, the posterior distribution isThere is no boundary wraparound: the neighbor sets use only actual horizontal and vertical edges inside the square. Each unordered edge is counted once. In particular, the prior retains the printed minus sign in front of $J$, and the likelihood retains the seventh power. The finite state space and finite observed values make every posterior weight positive and finite, with a finite positive normalizing sum.

The prior factors involving $s_i$ are $\exp (-J{s}_i{s}_j)$ for $j\in {\mathcal{N}}_i$. A likelihood factor centered at $k$ depends on $s_i$ precisely when $i\in {\mathcal{N}}_k$, equivalently $k\in {\mathcal{N}}_i$. Those factors areOnly incident prior edges and likelihoods centered at neighboring sites are affected. In particular, the likelihood centered at $i$ does not depend on $s_i$: its mean involves only the neighbors of $i$. These are the local likelihood factors in a hidden spin field. The resulting conditional dependence can extend two lattice steps, beyond the nearest-neighbor prior interactions. For $K=1$, both collections are empty.

For $k\in {\mathcal{N}}_i$, put $a_k^{(i)}={\sum }_{j\in {\mathcal{N}}_k\setminus \{i\}}{s}_j$ and $H_i={\sum }_{j\in {\mathcal{N}}_i}{s}_j$. Removing all factors independent of $s_i$ from the posterior distribution gives, for $u\in \{-1,1\}$,A stable implementation uses the difference of the log weights. The conditional log odds areso ${\pi }_i(+1\mid s_{-i},x)=1/(1+e^{-L_i})$, a logistic function. For very large $|L_i|$, evaluate this logistic expression using the sign of $L_i$ to avoid exponential overflow.