Choose a proposal probability density function positive wherever is nonzero, up to null sets, and draw independent and identically distributed random variables from . The ordinary importance sampling estimator is
The support condition for importance sampling and absolute integrability give , so this is an unbiased estimator. Its variance, possibly infinite, is
Let . If , the Cauchy-Schwarz inequality yields
Equality holds for proportional to . Thus the minimum-variance importance distribution and the minimum variance are
For a constant-sign integrand this is zero variance. If , the integrand vanishes almost everywhere and the zero estimator already has zero variance. Requiring the proposal to cover all of the target's support, including where , can exclude ; in that stricter class, approaches the same infimum as . This distinguishes coverage of the integral from coverage of every target event.
The generalized quantile function is
The tail limits of a cumulative distribution function make the defining set nonempty and its infimum finite. Monotonicity and right continuity imply exactly when . To see the direction involving the infimum, let points in the defining set decrease towards it; right continuity gives , so every larger also qualifies. Consequently, for a uniform distribution variable ,
The probability-zero endpoints can be assigned arbitrary outputs. This proves inverse transform sampling even for distributions with atoms or flat portions; continuity or strict monotonicity of is not required.
The autocovariance vanishes beyond lag . For an ordinary consecutive sum, variance of a sum gives
The last sum is finite. For , . Counting all coefficient pairs gives
For the odd-indexed sample the lag- autocovariance is , so the same finite-sum argument gives . Only coefficient pairs of the same parity contribute. If and , the odd-subsample limit is
This is the odd-subsample long-run variance of a moving average. Equivalently it is in the cycles convention. Under the printed invertibility assumption and positive noise variance, the polynomial is nonzero at both and , so both limits are positive. The same formulas also hold without invertibility, when cancellations can make a limit zero.
Use cycles-per-time spectral density, with frequency . The spectral representation theorem for a stationary time series gives the centered white noise representation
where disjoint increments are orthogonal. Put . Since the filter is finite, substitute each noise representation and interchange the finite sum with the integral:
The new orthogonal increment measure is . Its variance measure is therefore . The coefficients are real, so conjugation changes the sign of the exponent without changing the modulus. Hence
With angular frequency , the spectral density of a stationary process instead contains the factor . The convention explains its absence here. Invertibility is not needed for this finite-filter spectral calculation.
For the moving-average process of order one, , , and . Regressing each endpoint on the single intervening observation leaves residual covariance and residual variance . Therefore its lag-two partial autocorrelation function is
The denominator is positive. In particular, the moving-average autocorrelation cutoff at lag one does not make the lag-two partial correlation zero unless .
Use the sample autocorrelation function and sample partial autocorrelation function to look for approximate cutoffs. For a minimal causal autoregressive model of order , the partial autocorrelation function is zero beyond , while the autocorrelation function usually tails off, possibly with damped oscillation. For an invertible moving-average model of order , the autocorrelation function is zero beyond , while the partial autocorrelation function tails off. In a mixed autoregressive moving-average model, both generally tail off.
Thus the last visibly nonzero partial correlation suggests an autoregressive order, and the last visibly nonzero autocorrelation suggests a moving-average order. Finite samples do not produce exact zeros. Approximate white-noise reference bands can help flag clearly nonzero lags, but they are not universal confidence intervals for arbitrary correlated processes. Fit a few suggested orders and check residual autocorrelation, rather than treating these plots as a proof of the model order. This is order identification by autocorrelation cutoffs.
For a centered weakly stationary process with , the autocorrelation function is
For , let be orthogonal projection in L2 space onto the linear span of . The lag- partial autocorrelation function is the correlation coefficient of and . It removes the linear contribution of the intervening observations. At lag one it is just . For nonsingular prediction covariance matrices, it is equivalently the last coefficient in the order- linear predictor of from . Equal residual variances, by stationarity, identify that coefficient with the residual correlation. If a residual has zero variance, this correlation is undefined; the nondegeneracy condition matters.
The useful autoregressive model is for the squares, not for signed observations. Put and
Then
The errors form a martingale difference sequence relative to the noise history, because the current standardized noise is independent of the past. When the fourth moment is finite they have finite variance and are uncorrelated across distinct times, although their conditional variance depends on the regressor. In centered form, .
For ordinary least squares, use the response-regressor pairs , , . With their separate means and , minimize . If the regressor sum of squares is positive, the estimators are
The two means use the matched pairs; replacing them indiscriminately by a single full-sample mean is not the exact least-squares formula. The noise-series representation supplies an ergodic stationary process. If , finite regressor second moments and the error's zero conditional expectation justify the usual population regression and statistical consistency argument. The observations still define a finite-sample least-squares fit outside that moment range, but the ordinary finite-variance justification must not be claimed there. If parameter constraints are required, minimize the same criterion subject to and , rather than assert that unconstrained estimates automatically satisfy them.
First justify the dependence structure rather than assume that adjacent squares behave like an ordinary lag-one ARCH model. Write . Iterating its nonnegative recurrence gives
The product coefficient has expected value , so it tends to zero in probability. The stationary are a tight family; consequently the remainder tends to zero in probability, without needing independence between that remainder's factors. The increasing partial sums therefore give the representation
This proves the parity decomposition of a lag-two ARCH process: even and odd observations are functions of disjoint noise families. It also shows that the positive scale is a function of past noise, independent of . The conditional expectation of given past noise is zero, and the second-moment recursion is . independence of the parity families gives the adjacent-square covariance. Thus
To establish finiteness of the fourth moment before using its recursion, note that the norm of the th term in the positive series for is . The triangle inequality in L2 space makes the series square-integrable when . independence of the current noise and past scale then yields
and hence
If , a finite would make , which is impossible. Its fourth moment is then infinite. The adjacent-square product remains integrable by parity independence, even in that case.
Two useful features of autoregressive conditional heteroscedasticity are persistent changes in conditional scale and excess unconditional kurtosis. Its time-varying conditional variance can explain volatility clustering, where large absolute returns occur in groups even when signed returns have little autocorrelation. A homoscedastic autoregressive moving-average model has a fixed innovation variance.
Also, a conditional normal distribution with a random scale is a Gaussian scale mixture. Its unconditional kurtosis can exceed , or its fourth moment can be infinite. A Gaussian autoregressive moving-average model remains jointly Gaussian and cannot reproduce this effect. In the particular lag-two ARCH process, persistence of the squared scale occurs within each parity subsequence; the two parity subsequences are independent.
A weakly stationary process has finite second moments, a constant expected value , and covariance depending only on the time difference:
A strictly stationary process, also called strongly stationary, has every finite-dimensional probability distribution invariant under a common shift: for every finite choice of times and every shift , and have the same law. strict stationarity with finite second moments implies weak stationarity. weak stationarity alone does not determine the full joint law, and strict stationarity alone does not guarantee finite moments.
By the causality root criterion for an autoregressive model, choose smaller than the modulus of every root of . The function is an analytic function on and inside . Writing , the Cauchy estimate gives . For , independence of the noise in the infinite moving-average representation gives
The autocovariance is symmetric in the lag. Thus exponential decay holds with
This is exponential autocovariance decay of a causal autoregression. If is constant, there are no roots and any works.
For a stationary linear autoregression, causal time series means that the observation uses only current and past driving noise:
This is a convergent in the sense of mean-square convergence infinite moving-average representation. The stronger usual stable-filter definition requires absolute summability; the argument below also handles the square-summable definition. Let and . Substituting the filter into the recurrence and comparing coefficients of the orthogonal noise gives and . Hence
The Cauchy-Schwarz inequality ensures that is an analytic function in this disk. Thus has no zero strictly inside it. A boundary zero is also impossible: a zero of multiplicity at makes at least a constant times near that point. Its integral over diverges as . On the other hand, orthogonality of complex exponentials gives
a contradiction. Therefore the causality root criterion for an autoregressive model is
Under absolute summability, the shorter boundary argument is continuity of on the closed disk and the identity there.
A unit-root autoregressive process retains shocks permanently, whereas a causal time series with reverts towards its mean. Testing the unit root determines whether stationary autoregressive analysis is appropriate or differencing is needed. For the model without an intercept or trend, use the Dickey–Fuller test against the lower-sided alternative near the null. Put
This is the ordinary regression statistic for a zero coefficient when is regressed on , but its null probability distribution is not the usual Student law. Under the standard unit-root initialization and innovations independent of the starting value,
where is standard Brownian motion. If is the lower -quantile of , the asymptotic level- critical region is
The deterministic terms and null initialization must match the critical-value table. For exact finite-sample size of a statistical test, calibrate the statistic from its Gaussian random walk null with the specified initial condition and noise scale; for a zero starting value its distribution is scale-free. The printed two-sided recurrence alone specifies neither an initial law nor a universal finite-sample critical value. Ordinary normal quantiles do not give the intended size of a statistical test.

Pinned article: Introduction to the OurBigBook Project

Welcome to the OurBigBook Project! Our goal is to create the perfect publishing platform for STEM subjects, and get university-level students to write the best free STEM tutorials ever.
Everyone is welcome to create an account and play with the site: ourbigbook.com/go/register. We belive that students themselves can write amazing tutorials, but teachers are welcome too. You can write about anything you want, it doesn't have to be STEM or even educational. Silly test content is very welcome and you won't be penalized in any way. Just keep it legal!
We have two killer features:
  1. topics: topics group articles by different users with the same title, e.g. here is the topic for the "Fundamental Theorem of Calculus" ourbigbook.com/go/topic/fundamental-theorem-of-calculus
    Articles of different users are sorted by upvote within each article page. This feature is a bit like:
    • a Wikipedia where each user can have their own version of each article
    • a Q&A website like Stack Overflow, where multiple people can give their views on a given topic, and the best ones are sorted by upvote. Except you don't need to wait for someone to ask first, and any topic goes, no matter how narrow or broad
    This feature makes it possible for readers to find better explanations of any topic created by other writers. And it allows writers to create an explanation in a place that readers might actually find it.
    Figure 1.
    Screenshot of the "Derivative" topic page
    . View it live at: ourbigbook.com/go/topic/derivative
  2. local editing: you can store all your personal knowledge base content locally in a plaintext markup format that can be edited locally and published either:
    This way you can be sure that even if OurBigBook.com were to go down one day (which we have no plans to do as it is quite cheap to host!), your content will still be perfectly readable as a static site.
    Figure 5. . You can also edit articles on the Web editor without installing anything locally.
    Video 3.
    Edit locally and publish demo
    . Source. This shows editing OurBigBook Markup and publishing it using the Visual Studio Code extension.
  3. https://raw.githubusercontent.com/ourbigbook/ourbigbook-media/master/feature/x/hilbert-space-arrow.png
  4. Infinitely deep tables of contents:
    Figure 6.
    Dynamic article tree with infinitely deep table of contents
    .
    Descendant pages can also show up as toplevel e.g.: ourbigbook.com/cirosantilli/chordate-subclade
All our software is open source and hosted at: github.com/ourbigbook/ourbigbook
Further documentation can be found at: docs.ourbigbook.com
Feel free to reach our to us for any help or suggestions: docs.ourbigbook.com/#contact