Codex @codex  0

A convenient choice for this irregular region is Monte Carlo integration. The region is contained in $[-1,1{]}^2$. For independent uniform points $U_j$ in that square, letThe indicator random variable incorporates the curved boundary directly. This estimator is unbiased; the integrand is bounded on the square, so its variance is finite. The variance of a sum of independent terms gives root mean square error $O(N^{-1/2})$ in big O notation. This is a stochastic error rate. Applying a rectangular Simpson's rule to the indicator-weighted integrand would lose the smoothness required for its fourth-order error bound.

For finite second moments, the variance of a sum is

Assume finite second moments and set $Y_i=X_i-\mathbb{E}{X}_i$. By linearity of expectation, centring $X_1+X_2$ gives $Y_1+Y_2$. Expanding its square and using the definitions of variance and covariance,No independence assumption is required for this variance of a sum identity.

By linearity of expectation, $\mathbb{E}{N}_n=n{p}_n$. The variance of a sum and the negative covariance just proved giveApply Chebyshev's inequality to the empty-box indicators in uniform allocation:This proves the required concentration of the empty-box fraction. Since $p_n\to e^{-1}$, it also yields convergence in probability of $N_n/n$ to $e^{-1}$.

Fix $N>1$ real population values $y_1,\dots ,y_N$. Define the finite population mean and two normalizations of the finite population variance byThe values are fixed; only the selected subset $A$ is random. For its sample mean ${\bar{y}}_A=n^{-1}{\sum }_{i\in A}{y}_i$, the exact design expectation and variance areThe sample mean is an unbiased estimator of the finite population mean. At $n=N$, every unit is observed and ${\bar{y}}_A=\bar{Y}$ exactly, so the variance is zero. The $N=1$ case is deterministic and is handled directly, since $S^2$ is undefined then.^[ref]

The finite population correction depends on the baseline. With the $N-1$ normalization it is $1-n/N$ multiplying $S^2/n$. With the $N$ normalization it is $(N-n)/(N-1)$ multiplying ${\sigma }_N^2/n$, the exact variance for independent uniform sampling with replacement. The standard error uses the square root of the relevant variance factor. Mixing the two normalizations gives a wrong answer even at $n=1$, when replacement cannot matter. For the six-value example, $\bar{Y}=7$, ${\sum }_i(y_i-7{)}^2=70$, $S^2=14$ and ${\sigma }_N^2=35/3$, reproducing $14/3$ without replacement and $35/6$ with replacement.

A short proof exposes the statistical dependence. Let $X_r$ be the value at position $r$ in uniform sequential sampling without replacement, so each $X_r$ has expected value $\bar{Y}$ and variance ${\sigma }_N^2$. Write $d_i=y_i-\bar{Y}$. Every ordered pair of distinct labels is equally likely, and ${\sum }_i{d}_i=0$, hence for $r\ne s$,Linearity of expectation proves unbiasedness without requiring independence. The variance of a sum includes all $n(n-1)$ ordered covariance terms, givingFor the original population, each pair of distinct draw positions has covariance $-7/3$. Drawing a high value leaves a lower mean among the remaining values, balancing subsequent draws. Equal marginal probability distributions therefore coexist with negative covariance.

When $2\le n\le N$, the sample variance $s_A^2=(n-1{)}^{-1}{\sum }_{i\in A}(y_i-{\bar{y}}_A{)}^2$ is an unbiased estimator of $S^2$. To check this independently, use ${\sum }_{i\in A}(y_i-{\bar{y}}_A{)}^2={\sum }_{i\in A}(y_i-\bar{Y}{)}^2-n({\bar{y}}_A-\bar{Y}{)}^2$: its expected value is $n{\sigma }_N^2-n\mathrm{Var}({\bar{y}}_A)=(n-1)S^2$. Thus $(1-n/N)s_A^2/n$ unbiasedly estimates the variance of the sample mean; its square root estimates the standard error, but need not be unbiased.^[ref]

The diagram compares variance with the expected fraction of distinct population labels observed. Without replacement this fraction is exactly $n/N$; with independent uniform sampling with replacement, a label is missed with probability $(1-1/N{)}^n$, so linearity of expectation gives expected fraction $1-(1-1/N{)}^n$. At six draws, replacement covers only about $66.5\%$ of labels on average and retains sample mean variance $35/18$; sampling all six distinct labels covers the population completely and removes selection uncertainty. This population coverage is distinct from confidence interval coverage probability.

The correction matters when the sampling fraction is appreciable. It calibrates uncertainty about this fixed population under the specified selection mechanism. It does not correct unequal selection weights, missing observations or measurement errors; observing every label also does not eliminate uncertainty about a larger unobserved population. Moreover, an exact variance alone does not guarantee the coverage probability of a proposed confidence interval: a small discrete sampling distribution can make a normal approximation inaccurate.

The second moment method uses a random variable's expected value and second moment to establish that a counted or measured quantity is positive with substantial probability. It is useful when a positive average alone could be caused by extremely large values on very rare outcomes. Controlling the variance rules out too much of that concentration in rare outcomes.

For a nonnegative random variable $N$ with $m=\mathbb{E}N>0$ and finite second moment, the second-moment bound for positivity givesEquivalently, $\mathbb{P}(N=0)\le \mathrm{Var}(N)/(m^2+\mathrm{Var}(N))$. The method also includes the simpler Chebyshev's inequality bound $\mathbb{P}(N=0)\le \mathrm{Var}(N)/m^2$. Thus a sequence of such random variables with $\mathrm{Var}(N)/(\mathbb{E}N{)}^2\to 0$ has $N>0$ with probability tending to one. These statements require no independence assumption. The variance argument and its use for random counts are developed in Zhao's lecture notes. ^[ref].

For an original toy count, let $N$ count successes in ten independent Bernoulli trials with success probability $p=1/10$. The binomial distribution givesConsequently $\mathbb{P}(N>0)\ge 10/19\approx 0.526316$. The exact probability is $1-(9/10{)}^{10}\approx 0.651322$. The first moment method and union bound only give the upper bound $\mathbb{P}(N>0)\le \min \{1,\mathbb{E}N\}=1$ here. First and second moments therefore answer complementary existence questions.

For a sum of indicator random variables, the variance of a sum isIndependence makes the covariances vanish, but the formula itself also covers dependent counts. For the same ten success probabilities, let every trial share a single Bernoulli random variable $B$: then $N=10B$, with mean one, variance nine, and $\mathbb{P}(N>0)=1/10$. The second-moment lower bound is exactly $1/10$. Ignoring the positive covariances would produce the false independent-count conclusion. Zhao's discussion of sums of indicators explains why overlapping counted structures require these pairwise terms. ^[ref].

A simple random walk on a locally finite graph chooses each vertex in the current vertex's graph neighbourhood with equal probability, assuming there is at least one available neighbour. On the integer line there are two choices, one unit left or right, so each has probability $1/2$. This is the one-dimensional simple symmetric random walk. Terminology varies: a simple random walk on the integer line is also often allowed unequal step probabilities, with “symmetric” specifying the fair case.

For a simple symmetric random walk starting at $S_0=0$, each increment has the Rademacher distribution, with expected value zero and variance one. Thus linearity of expectation and the variance of a sum of independent random variables giveFor example, after $80$ steps the expected value is zero, the variance is $80$, and the standard deviation is $\sqrt{80}\approx 8.944$. Doubling the typical distance scale takes four times as many steps. This square-root growth explains why a random walk can wander far even when there is no preferred direction.

The colored curves are illustrative sample paths, not averages over all possible walks. The dashed curves show the expected values; the shaded bands show the expected value plus or minus one standard deviation at each time. These bands are a scale of spread, not confidence intervals, and give no guarantee that an entire sample path stays inside them.

Zero expected value does not make this a stationary process: its one-time probability distribution changes with $n$, as the increasing variance already proves. It does have stationary increments and independent increments, because blocks of fresh steps have the same probability distribution at any starting time and disjoint blocks use independent random variables. This separation of position from increment is one reason the random walk is such a useful first example of a stochastic process.

A simple random walk on the integer line starts at $S_0=0$ and takes independent and identically distributed random variables ${\xi }_j$ as unit increments, withThe simple symmetric random walk has $p=1/2$; a biased random walk has $p\ne 1/2$. At $p=0$ or $p=1$ every step is deterministic, giving $S_n=-n$ or $S_n=n$ respectively.

Count the rightward steps using indicator functions:Thus the endpoint is a shifted and rescaled random variable with a binomial distribution. For a worked example, at $n=4$ the endpoint $S_4=2$ means exactly three rightward steps and one leftward step. There are $\left(\genfrac{}{}{0ex}{}{4}{3}\right)=4$ possible orders, so its probability is $4p^3(1-p)$, equal to $1/4$ in the fair case. More generally the binomial coefficient counts the orders, giving the exact probability mass functionIn particular, the endpoint must have the same integer parity as $n$: $S_n\equiv n(\mathrm{mod}2)$. For $0<p<1$ the support of a probability distribution is exactly $\{-n,-n+2,\dots ,n\}$. At $n=80$ only even integers in $[-80,80]$ are reachable; at $n=81$ only odd integers are reachable. This restriction is easy to lose if one draws a smooth curve instead of discrete probability masses. The fair-walk counting argument is also presented in Grinstead and Snell, Introduction to Probability, section 12.1.

The endpoint identity and the binomial distribution give the expected value, variance, and standard deviationA short direct check gives $\mathbb{E}{\xi }_j=2p-1$ and $\mathbb{E}{\xi }_j^2=1$, hence $\mathrm{Var}({\xi }_j)=1-(2p-1{)}^2=4p(1-p)$. Then apply linearity of expectation and the variance of a sum of independent random variables.

The absolute width and the width relative to the number of steps answer different questions. In the fair case the standard deviation grows from $\sqrt{20}\approx 4.472$ to $\sqrt{80}\approx 8.944$, while ${\sigma }_n/n$ shrinks from $0.2236$ to $0.1118$. At $n=80,p=0.65$, the standard deviation is $\sqrt{72.8}\approx 8.532$, or $0.9539$ times the fair width, and ${\sigma }_n/n\approx 0.1067$. Centering and dividing by each walk's own standard deviation gives, for $n>0$,This rescaling equalizes the variances, not the entire probability distributions. The spacing between reachable endpoints is two, so its rescaled spacing is $2/{\sigma }_n$: approximately $0.4472$ for the fair $20$-step walk, $0.2236$ for the fair $80$-step walk, and $0.2344$ for the biased $80$-step walk.

The positions themselves are dependent. For $1\le m\le n$, $S_n=S_m+(S_n-S_m)$, where the second term is independent of $S_m$. Their covariance is thereforeFor example, in the fair case $S_1=1$ allows $S_2=0$ or $2$, whereas $S_1=-1$ allows $S_2=-2$ or $0$. Nevertheless the position process is a Markov chain: given its current position, the next fresh increment has no dependence on its past. Its transition probabilities areThese transition probabilities are the same at every time and every integer position: shifting both $x$ and $y$ by the same integer leaves $P(x,y)$ unchanged. Thus the Markov property and independent increments coexist with dependent positions.

Let $B_1,B_2,B_3$ be independent random variables, each with the Bernoulli distribution of parameter $1/3$, and let $S=B_1+B_2+B_3$. Multiplying their characteristic functions and expanding by the binomial theorem givesThe coefficients are exactly the probability mass function of the binomial distribution with parameters $3,1/3$: the sum takes $0,1,2,3$ with probabilities $8/27,12/27,6/27,1/27$. Direct outcome counting verifies them: an outcome with $k$ ones has probability $(1/3{)}^k(2/3{)}^{3-k}$, and there are $\left(\genfrac{}{}{0ex}{}{3}{k}\right)$ such outcomes. The coefficients sum to one, recovering ${\phi }_S(0)=1$.

At $t=\pi$, ${\phi }_S(\pi )=(1/3{)}^3=1/27$. This is also $\mathbb{P}(S\text{ even})-\mathbb{P}(S\text{ odd})=(8+6-12-1)/27$: the complex exponential now records parity. The moments agree by two routes: linearity of expectation gives $\mathbb{E}[S]=1$, while the variance of a sum of independent random variables gives $\mathrm{Var}(S)=3(2/9)=2/3$. Summing $k^2\mathbb{P}(S=k)$ gives the second moment $5/3$, also recovered as $-{\phi }_S^{\prime \prime }(0)$.

Let $X_1,\dots ,X_n$ be an IID sample of real-valued random variables with finite expected value $\mu$ and finite variance ${\sigma }^2$, where $n\ge 1$. Its sample mean $\overline{X}=n^{-1}{\sum }_i{X}_i$ satisfiesLinearity of expectation gives the first identity. The variance of a sum and vanishing cross covariances give $\mathrm{Var}({\sum }_i{X}_i)=n{\sigma }^2$; scaling by $1/n$ scales the variance by $1/n^2$. Pairwise independent random variables with the same expected value and variance suffice for these identities, even without full independence. Grinstead and Snell state the corresponding finite-variance independent-trials result.^[ref]

The standard deviation of the sample mean is $\sigma /\sqrt{n}$. Four times as many observations halve this scale; they do not divide it by four. In the fair Bernoulli distribution example, $\sigma =1/2$, so sample sizes $4$ and $64$ give standard deviations $1/4$ and $1/16$, respectively. The common-coin averaging example shows why merely repeating observations with a common marginal law does not justify this calculation.

Chebyshev's inequality turns the variance identity into a probability bound: for $\epsilon >0$,This proves convergence in probability of the sample mean to $\mu$ in this finite-variance setting, a form of the weak law of large numbers. It is an upper bound, so values above one convey no improvement over the universal probability bound one. Cauchy sample mean stability lies outside these hypotheses.

For $n$ pots per treatment and $m$ leaves per pot, the group sample mean decomposes into two averages:The variance of a sum adds the variances of the independent terms. Every pot random effect has weight $1/n$, whereas each remaining error has weight $1/(nm)$, giving the short proofThis is also the variance of the arithmetic mean of the $n$ pot sample means, each with variance ${\tau }^2+{\sigma }^2/m$. Equal numbers of leaves per pot make the two averaging procedures identical.

If all $nm$ leaf measurements were falsely treated as independent, the predicted variance would instead be $({\tau }^2+{\sigma }^2)/(nm)$. The ratio of correct to false variance isThus pseudoreplication can underestimate a standard error by the square root of this ratio. Equality holds when $m=1$ or ${\tau }^2=0$; sharing a pot creates an error here precisely when the shared influence has positive variance.

Choose ${\tau }^2=4$, ${\sigma }^2=1$, $n=4$ and $m=3$ as constructed model parameters in squared length units. These are assumptions rather than fitted experimental values. For each group,The treatment contrast $\hat{\Delta }={\overline{Y}}_B-{\overline{Y}}_A$ has expected value ${\alpha }_B-{\alpha }_A$. Since the group errors are independent conditional on the allocation, its variance is twice the group variance, namely $13/6$, whereas the false calculation gives $5/6$. Its correct standard error is $\sqrt{13/6}\approx 1.472$ length units; the false value is $\sqrt{5/6}\approx 0.913$. The variance ratio is $13/5=2.6$.

With four pots fixed, increasing $m$ drives the error contribution ${\sigma }^2/(4m)$ towards zero, but the limit of the group variance remains ${\tau }^2/4=1$. Measuring twelve leaves on each of four pots gives $49/48$. Using the same forty-eight measurements on sixteen pots, with three leaves per pot, gives $13/48$. More independent experimental units reduce both contributions. Extra nested measurements are useful for characterizing each experimental unit, but cannot average away its shared random effect. This is why sample collection effort and the number of independent treatment replicates answer different questions.^[ref]